Abstract
Listening to heart sounds is an important first step in evaluating the cardiovascular system and is important in the early detection of cardiovascular disease. We present and evaluate a framework for combining machine learning algorithms, crowd workers, and experts in the classification of heart sound recordings. The development of a hybrid human-machine framework is motivated by the past success in utilizing human computation to solve problems in medicine and the use of human-machine frameworks in other domains. We describe the methods that decide when and how to escalate the analysis of heart sounds to different resources and incorporate their decision into a final classification. Our framework was tested with a combination of machine classifiers and crowd workers from Amazon's Mechanical Turk. The results indicate a hybrid approach achieves greater performance than a baseline classifier alone, utilizing less expert resources while achieving similar performance, compared to a framework without the crowd.
- Avrim Blum and Tom Mitchell. 1998. Combining labeled and unlabeled data with co-training. In Proceedings of the 11th Annual Conference on Computational Learning Theory. ACM, 92--100. Google Scholar
Digital Library
- Ignacio J Diaz Bobillo. 2016. A tensor approach to heart sound classification. In Computing in Cardiology Conference (CinC), 2016. IEEE, 629--632.Google Scholar
- J. Christopher Brady, C. Andrea Villanti, L. Jennifer Pearson, R. Thomas Kirchner, P. Omesh Gupta, and P. Chirag Shah. 2014. Rapid Grading of Fundus Photographs for Diabetic Retinopathy Using Crowdsourcing. J Med Internet Res , Vol. 16, 10 (30 Oct 2014), e233.Google Scholar
Cross Ref
- Mark Cartwright, Ayanna Seals, Justin Salamon, Alex Williams, Stefanie Mikloska, Duncan MacConnell, Edith Law, Juan P. Bello, and Oded Nov. 2017. Seeing Sound: Investigating the Effects of Visualizations and Complexity on Crowdsourced Audio Annotations. Proc. ACM Hum.-Comput. Interact. , Vol. 1, CSCW, Article 29 (Dec. 2017), bibinfonumpages21 pages. Google Scholar
Digital Library
- Justin Cheng and Michael S Bernstein. 2015. Flock: Hybrid crowd-machine learning classifiers. In Proceedings of the 18th ACM Conference on Computer Supported Cooperative Work & Social Computing. ACM, 600--611. Google Scholar
Digital Library
- G. D. Clifford, C. Liu, B. Moody, D. Springer, I. Silva, Q. Li, and R. G. Mark. 2016. Classification of normal/abnormal heart sound recordings: The PhysioNet/Computing in Cardiology Challenge 2016. In 2016 Computing in Cardiology Conference (CinC) . 609--612.Google Scholar
- J.S. Coviello. 2013. Auscultation Skills: Breath & Heart Sounds 5 ed.). Wolters Kluwer Health. 2013023781Google Scholar
- P. Dawid, A. M. Skene, A. P. Dawidt, and A. M. Skene. 1979. Maximum likelihood estimation of observer error-rates using the EM algorithm. Applied Statistics (1979), 20--28.Google Scholar
- Francisco J. Candido dos Reis, Stuart Lynn, H. Raza Ali, Diana Eccles, Andrew Hanby, Elena Provenzano, Carlos Caldas, William J. Howat, Leigh-Anne McDuffus, Bin Liu, Frances Daley, Penny Coulson, Rupesh J. Vyas, Leslie M. Harris, Joanna M. Owens, Amy F.M. Carton, Janette P. McQuillan, Andy M. Paterson, Zohra Hirji, Sarah K. Christie, Amber R. Holmes, Marjanka K. Schmidt, Montserrat Garcia-Closas, Douglas F. Easton, Manjeet K. Bolla, Qin Wang, Javier Benitez, Roger L. Milne, Arto Mannermaa, Fergus Couch, Peter Devilee, Robert A.E.M. Tollenaar, Caroline Seynaeve, Angela Cox, Simon S. Cross, Fiona M. Blows, Joyce Sanders, Renate de Groot, Jonine Figueroa, Mark Sherman, Maartje Hooning, Hermann Brenner, Bernd Holleczek, Christa Stegmaier, Chris Lintott, and Paul D.P. Pharoah. 2015. Crowdsourcing the General Public for Large Scale Molecular Pathology Studies in Cancer. EBioMedicine , Vol. 2, 7 (2015), 681 -- 689.Google Scholar
Cross Ref
- Steven Dow, Anand Kulkarni, Scott Klemmer, and Björn Hartmann. 2012. Shepherding the Crowd Yields Better Work. In Proceedings of the ACM 2012 Conference on Computer Supported Cooperative Work (CSCW '12). ACM, New York, NY, USA, 1013--1022. Google Scholar
Digital Library
- David S. Gerbarg, Angelo Taranta, Mario Spagnuolo, and John J. Hofler. 1963. Computer analysis of phonocardiograms. Progress in Cardiovascular Diseases , Vol. 5, 4 (1963), 393 -- 405.Google Scholar
Cross Ref
- Benjamin M. Good and Andrew I. Su. 2013. Crowdsourcing for bioinformatics. Bioinformatics , Vol. 29, 16 (2013), 1925--1933.Google Scholar
Cross Ref
- Dilek Hakkani-Tür, Giuseppe Riccardi, and Allen Gorin. 2002. Active learning for automatic speech recognition. In 2002 IEEE International Conference on Acoustics, Speech, and Signal Processing (ICASSP), Vol. 4. IEEE, IV--3904--IV--3907.Google Scholar
Cross Ref
- D. Hakkani-Tur, G. Tur, M. Rahim, and G. Riccardi. 2004. Unsupervised and active learning in automatic speech recognition for call classification. In 2004 IEEE International Conference on Acoustics, Speech, and Signal Processing, Vol. 1. I--429.Google Scholar
- David W Hosmer and Stanley Lemeshow. 1980. Goodness of fit tests for the multiple logistic regression model . Communications in Statistics: Theory and Methods , Vol. 9, 10 (1980), 1043--1069.Google Scholar
Cross Ref
- A. J. Joshi, F. Porikli, and N. P. Papanikolopoulos. 2012. Scalable Active Learning for Multiclass Image Classification. IEEE Transactions on Pattern Analysis and Machine Intelligence , Vol. 34, 11 (Nov 2012), 2259--2273. Google Scholar
Digital Library
- Himabindu Lakkaraju, Stephen H Bach, and Jure Leskovec. 2016. Interpretable decision sets: A joint framework for description and prediction. In Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining. ACM, 1675--1684. Google Scholar
Digital Library
- Paul Lamere. 2008. Social tagging and music information retrieval. Journal of New Music Research , Vol. 37, 2 (2008), 101--114.Google Scholar
Cross Ref
- Edith Law and Luis von Ahn. 2011. Human Computation. Synthesis Lectures on Artificial Intelligence and Machine Learning , Vol. 5, 3 (2011), 1--121. https://doi.org/10.2200/S00371ED1V01Y201107AIM013Google Scholar
Digital Library
- Thérèse A Stukel. 1988. Generalized Logistic Models . J. Amer. Statist. Assoc. , Vol. 83, 402 (1988), 426--431.Google Scholar
Cross Ref
- Chong Sun, Narasimhan Rampalli, Frank Yang, and AnHai Doan. 2014. Chimera: Large-scale classification using machine learning, rules, and crowdsourcing. Proceedings of the VLDB Endowment , Vol. 7, 13 (2014), 1529--1540. Google Scholar
Digital Library
- A.J. Taylor. {n. d.}. Learning Cardiac Auscultation: From Essentials to Expert Clinical Interpretation .Google Scholar
- Simon Tong and Daphne Koller. 2001. Support vector machine active learning with applications to text classification. Journal of Machine Learning Research , Vol. 2, Nov (2001), 45--66. Google Scholar
Digital Library
- Gokhan Tur, Dilek Hakkani-Tür, and Robert E Schapire. 2005. Combining active and semi-supervised learning for spoken language understanding. Speech Communication , Vol. 45, 2 (2005), 171--186.Google Scholar
Cross Ref
- Douglas Turnbull, Ruoran Liu, Luke Barrington, and Gert RG Lanckriet. 2007. A Game-Based Approach for Collecting Semantic Annotations of Music. In ISMIR , Vol. 7. 535--538.Google Scholar
- Simon C Warby, Sabrina L Wendt, Peter Welinder, Emil G S Munk, Oscar Carrillo, Helge B D Sorensen, Poul Jennum, Paul E Peppard, Pietro Perona, and Emmanuel Mignot. 2014. Sleep-spindle detection: crowdsourcing and evaluating performance of experts, non-experts and automated methods . Nature Methods , Vol. 11 (Feb 2014), 385.Google Scholar
Cross Ref
- Jenna Wiens and John V Guttag. 2010. Active learning applied to patient-adaptive heartbeat classification. In Advances in Neural Information Processing Systems. 2442--2450. Google Scholar
Digital Library
- Frank Wilcoxon. 1945. Individual comparisons by ranking methods. Biometrics bulletin , Vol. 1, 6 (1945), 80--83.Google Scholar
- Bishan Yang, Jian-Tao Sun, Tengjiao Wang, and Zheng Chen. 2009. Effective multi-label active learning for text classification. In Proceedings of the 15th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining . ACM, 917--926. Google Scholar
Digital Library
- Dong Yu, Balakrishnan Varadarajan, Li Deng, and Alex Acero. 2010. Active learning and semi-supervised learning for speech recognition: A unified framework using the global entropy reduction maximization criterion. Computer Speech & Language , Vol. 24, 3 (2010), 433--444. Google Scholar
Digital Library
- Shan Zhang, Aditya Vempaty, Susan E Parks, and Pramod K Varshney. 2017. On classification of environmental acoustic data using crowds. In 2017 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP). IEEE, 5880--5884.Google Scholar
Cross Ref
- Xiaojin Zhu, Bryan Gibson, and Timothy Rogers. 2011. Co-Training as a Human Collaboration Policy. In AAAI Conference on Artificial Intelligence . Google Scholar
Digital Library
Index Terms
MechanicalHeart: A Human-Machine Framework for the Classification of Phonocardiograms
Recommendations
Automated Diagnosis of Heart Sounds Using Rule-Based Classification Tree
In order to assist the diagnosis procedure of heart sound signals, this paper presents a new automated method for classifying the heart status using a rule-based classification tree into normal and three abnormal cases; namely the aortic valve stenosis, ...
Phonocardiogram transfer learning-based CatBoost model for diastolic dysfunction identification using multiple domain-specific deep feature fusion
AbstractLeft ventricular diastolic dyfunction detection is particularly important in cardiac function screening. This paper proposed a phonocardiogram (PCG) transfer learning-based CatBoost model to detect diastolic dysfunction noninvasively. ...
Highlights- Transfer learning can provide objective aid in diastolic dysfunction detection using phonocardiogram (PCG).
Deep Learning for Heart Sounds Classification Using Scalograms and Automatic Segmentation of PCG Signals
Advances in Computational IntelligenceAbstractThis paper proposes a set of Deep Learning algorithms for classifying Phonocardiogram (PCG) scalogram. PCG signals contain valuable information about the heart health status, and they could help us in early detection and diagnosis of potential ...






Comments