skip to main content
research-article

Automatic Speech Classifier for Mild Cognitive Impairment and Early Dementia

Authors Info & Claims
Published:15 October 2021Publication History
Skip Abstract Section

Abstract

The World Health Organization estimates that 50 million people are currently living with dementia worldwide and this figure will almost triple by 2050. Current pharmacological treatments are only symptomatic, and drugs or other therapies are ineffective in slowing down or curing the neurodegenerative process at the basis of dementia. Therefore, early detection of cognitive decline is of the utmost importance to respond significantly and deliver preventive interventions. Recently, the researchers showed that speech alterations might be one of the earliest signs of cognitive defect, observable well in advance before other cognitive deficits become manifest. In this article, we propose a full automated method able to classify the audio file of the subjects according to the progress level of the pathology. In particular, we trained a specific type of artificial neural network, called autoencoder, using the visual representation of the audio signal of the subjects, that is, the spectrogram. Moreover, we used a data augmentation approach to overcome the problem of the large amount of annotated data usually required during the training phase, which represents one of the most major obstacles in deep learning. We evaluated the proposed method using a dataset of 288 audio files from 96 subjects: 48 healthy controls and 48 cognitively impaired participants. The proposed method obtained good classification results compared to the state-of-the-art neuropsychological screening tests and, with an accuracy of 90.57%, outperformed the methods based on manual transcription and annotation of speech.

REFERENCES

  1. [1] Abel Stefanie, Huber Walter, and Dell Gary S.. 2009. Connectionist diagnosis of lexical disorders in aphasia. Aphasiology 23, 11 (2009), 13531378.Google ScholarGoogle Scholar
  2. [2] Ambrosini Emilia, Caielli Matteo, Milis Marios, Loizou Christos, Azzolino Domenico, Damanti Sarah, Bertagnoli Laura, Cesari Matteo, Moccia Sara, Cid Manuel, et al. 2019. Automatic speech analysis to early detect functional cognitive decline in elderly population. In 2019 41st Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC’19). IEEE, 212216.Google ScholarGoogle Scholar
  3. [3] Beltrami Daniela, Calzà Laura, Gagliardi Gloria, Ghidoni Enrico, Marcello Norina, Favretti Rema Rossini, and Tamburini Fabio. 2016. Automatic identification of mild cognitive impairment through the analysis of Italian spontaneous speech productions. In Proceedings of the 10th International Conference on Language Resources and Evaluation (LREC’16). 20862093.Google ScholarGoogle Scholar
  4. [4] Beltrami Daniela, Gagliardi Gloria, Favretti Rema Rossini, Ghidoni Enrico, Tamburini Fabio, and Calzà Laura. 2018. Speech analysis by natural language processing techniques: A possible tool for very early detection of cognitive decline?Frontiers in Aging Neuroscience 10 (2018), 369.Google ScholarGoogle ScholarCross RefCross Ref
  5. [5] Bertini Flavio, Bergami Giacomo, Montesi Danilo, Veronese Giacomo, Marchesini Giulio, and Pandolfi Paolo. 2018. Predicting frailty condition in elderly using multidimensional socioclinical databases. Proceedings of the IEEE 106, 4 (2018), 723737.Google ScholarGoogle Scholar
  6. [6] Boschi Veronica, Catricala Eleonora, Consonni Monica, Chesi Cristiano, Moro Andrea, and Cappa Stefano F.. 2017. Connected speech in neurodegenerative language disorders: A review. Frontiers in Psychology 8 (2017), 269.Google ScholarGoogle Scholar
  7. [7] Budson Andrew E. and Solomon Paul R.. 2011. Memory Loss E-Book: A Practical Guide for Clinicians. Elsevier Health Sciences.Google ScholarGoogle Scholar
  8. [8] Calzà Laura, Beltrami Daniela, Gagliardi Gloria, Ghidoni Enrico, Marcello Norina, Rossini-Favretti Rema, and Tamburini Fabio. 2015. Should we screen for cognitive decline and dementia?Maturitas 82, 1 (2015), 2835.Google ScholarGoogle Scholar
  9. [9] Calzà Laura, Gagliardi Gloria, Favretti Rema Rossini, and Tamburini Fabio. 2020. Linguistic features and automatic classifiers for identifying mild cognitive impairment and dementia. Computer Speech & Language 65 (2020), 101113.Google ScholarGoogle ScholarCross RefCross Ref
  10. [10] Ciurli Paola, Marangolo Paola, and Basso Anna. 1996. Esame Del Linguaggio-II. OS. Retrieved on August 28, 2021 from https://www.giuntipsy.it/catalogo/test/esame-del-linguaggio-ii.Google ScholarGoogle Scholar
  11. [11] Clark David Glenn, McLaughlin Paula M., Woo Ellen, Hwang Kristy, Hurtz Sona, Ramirez Leslie, Eastman Jennifer, Dukes Reshil-Marie, Kapur Puneet, DeRamus Thomas P., et al. 2016. Novel verbal fluency scores and structural brain imaging for prediction of cognitive outcome in mild cognitive impairment. Alzheimer’s & Dementia: Diagnosis, Assessment & Disease Monitoring 2 (2016), 113122.Google ScholarGoogle Scholar
  12. [12] Etienne Caroline, Fidanza Guillaume, Petrovskii Andrei, Devillers Laurence, and Schmauch Benoit. 2018. CNN+LSTM architecture for speech emotion recognition with data augmentation. arXiv:1802.05630. https://arxiv.org/abs/1802.05630Google ScholarGoogle Scholar
  13. [13] Farias Sarah Tomaszewski, Mungas Dan, Reed Bruce R., Harvey Danielle, and DeCarli Charles. 2009. Progression of mild cognitive impairment to dementia in clinic- vs community-based cohorts. Archives of Neurology 66, 9 (2009), 11511157.Google ScholarGoogle ScholarCross RefCross Ref
  14. [14] Folstein Marshal F., Folstein Susan E., and McHugh Paul R.. 1975. “Mini-mental state”: A practical method for grading the cognitive state of patients for the clinician. Journal of Psychiatric Research 12, 3 (1975), 189198.Google ScholarGoogle ScholarCross RefCross Ref
  15. [15] Fors Kristina Lundholm, Fraser Kathleen C., and Kokkinakis Dimitrios. 2018. Automated syntactic analysis of language abilities in persons with mild and subjective cognitive impairment.. In MIE. 705709.Google ScholarGoogle Scholar
  16. [16] Fraser K., Fors K. Lundholm, Eckerström Marie, Themistocleous Charalambos, and Kokkinakis Dimitrios. 2018. Improving the sensitivity and specificity of MCI screening with linguistic information. In LREC Workshop: RaPID-2.Google ScholarGoogle Scholar
  17. [17] Fraser Kathleen C., Fors Kristina Lundholm, and Kokkinakis Dimitrios. 2019. Multilingual word embeddings for the assessment of narrative speech in mild cognitive impairment. Computer Speech & Language 53 (2019), 121139.Google ScholarGoogle Scholar
  18. [18] Fraser Kathleen C., Fors Kristina Lundholm, Kokkinakis Dimitrios, and Nordlund Arto. 2017. An analysis of eye-movements during reading for the detection of mild cognitive impairment. In Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing. 10161026.Google ScholarGoogle Scholar
  19. [19] Fraser Kathleen C., Fors Kristina Lundholm, Eckerström Marie, Öhman Fredrik, and Kokkinakis Dimitrios. 2019. Predicting MCI status from multimodal language data using cascaded classifiers. Frontiers in Aging Neuroscience 11 (2019), 205.Google ScholarGoogle Scholar
  20. [20] Freitag Michael, Amiriparian Shahin, Pugachevskiy Sergey, Cummins Nicholas, and Schuller Björn. 2017. auDeep: Unsupervised learning of representations from audio with deep recurrent neural networks. The Journal of Machine Learning Research 18, 1 (2017), 63406344. Google ScholarGoogle ScholarDigital LibraryDigital Library
  21. [21] Gosztolya Gábor, Vincze Veronika, Tóth László, Pákáski Magdolna, Kálmán János, and Hoffmann Ildikó. 2019. Identifying mild cognitive impairment and mild Alzheimer’s disease based on spontaneous speech using ASR and linguistic features. Computer Speech & Language 53 (2019), 181197.Google ScholarGoogle ScholarCross RefCross Ref
  22. [22] Hannun Awni, Case Carl, Casper Jared, Catanzaro Bryan, Diamos Greg, Elsen Erich, Prenger Ryan, Satheesh Sanjeev, Sengupta Shubho, Coates Adam, et al. 2014. Deep speech: Scaling up end-to-end speech recognition. arXiv:1412.5567. https://arxiv.org/abs/1412.5567Google ScholarGoogle Scholar
  23. [23] Jaitly Navdeep and Hinton Geoffrey E.. 2013. Vocal tract length perturbation (VTLP) improves speech recognition. In Proceedings of the ICML Workshop on Deep Learning for Audio, Speech and Language, Vol. 117.Google ScholarGoogle Scholar
  24. [24] Jarrold William, Peintner Bart, Wilkins David, Vergryi Dimitra, Richey Colleen, Gorno-Tempini Maria Luisa, and Ogar Jennifer. 2014. Aided diagnosis of dementia type through computer-based analysis of spontaneous speech. In Proceedings of the Workshop on Computational Linguistics and Clinical Psychology: From Linguistic Signal to Clinical Reality. 2737.Google ScholarGoogle Scholar
  25. [25] Kanda Naoyuki, Takeda Ryu, and Obuchi Yasunari. 2013. Elastic spectral distortion for low resource speech recognition with deep neural networks. In 2013 IEEE Workshop on Automatic Speech Recognition and Understanding. IEEE, 309314.Google ScholarGoogle Scholar
  26. [26] Kim Chanwoo, Misra Ananya, Chin Kean, Hughes Thad, Narayanan Arun, Sainath Tara, and Bacchiani Michiel. 2017. Generation of large-scale simulated utterances in virtual rooms to train deep-neural networks for far-field speech recognition in Google Home. Interspeech 2017 (2017), 379–383.Google ScholarGoogle Scholar
  27. [27] Ko Tom, Peddinti Vijayaditya, Povey Daniel, and Khudanpur Sanjeev. 2015. Audio augmentation for speech recognition. In 16th Annual Conference of the International Speech Communication Association.Google ScholarGoogle Scholar
  28. [28] Konig Alexandra, Satt Aharon, Sorin Alex, Hoory Ran, Derreumaux Alexandre, David Renaud, and Robert Phillippe H.. 2018. Use of speech analyses within a mobile application for the assessment of cognitive impairment in elderly people. Current Alzheimer Research 15, 2 (2018), 120129.Google ScholarGoogle Scholar
  29. [29] König Alexandra, Satt Aharon, Sorin Alexander, Hoory Ron, Toledo-Ronen Orith, Derreumaux Alexandre, Manera Valeria, Verhey Frans, Aalten Pauline, Robert Phillipe H., et al. 2015. Automatic speech analysis for the assessment of patients with predementia and Alzheimer’s disease. Alzheimer’s & Dementia: Diagnosis, Assessment & Disease Monitoring 1, 1 (2015), 112124.Google ScholarGoogle Scholar
  30. [30] Low Daniel M., Bentley Kate H., and Ghosh Satrajit S.. 2020. Automated assessment of psychiatric disorders using speech: A systematic review. Laryngoscope Investigative Otolaryngology 5, 1 (2020), 96116.Google ScholarGoogle ScholarCross RefCross Ref
  31. [31] Ma Xingchen, Yang Hongyu, Chen Qiang, Huang Di, and Wang Yunhong. 2016. DepAudioNet: An efficient deep model for audio based depression classification. In Proceedings of the 6th International Workshop on Audio/Visual Emotion Challenge. 3542. Google ScholarGoogle ScholarDigital LibraryDigital Library
  32. [32] Meilán Juan J. G., Martínez-Sánchez Francisco, Carro Juan, Sánchez José A., and Pérez Enrique. 2012. Acoustic markers associated with impairment in language processing in Alzheimer’s disease. The Spanish Journal of Psychology 15, 2 (2012), 487494.Google ScholarGoogle Scholar
  33. [33] Mitchell Alex J.. 2009. A meta-analysis of the accuracy of the mini-mental state examination in the detection of dementia and mild cognitive impairment. Journal of Psychiatric Research 43, 4 (2009), 411431.Google ScholarGoogle ScholarCross RefCross Ref
  34. [34] Organization World Health et al. 2017. Global action plan on the public health response to dementia 2017–2025. Retrieved on August 28, 2021 from https://www.who.int/publications/i/item/global-action-plan-on-the-public-health-response-to-dementia-2017---2025.Google ScholarGoogle Scholar
  35. [35] Park Daniel S., Chan William, Zhang Yu, Chiu Chung-Cheng, Zoph Barret, Cubuk Ekin D., and Le Quoc V.. 2019. SpecAugment: A simple data augmentation method for automatic speech recognition. arXiv:1904.08779. https://arxiv.org/abs/1904.08779Google ScholarGoogle Scholar
  36. [36] Petersen Ronald C.. 2011. Clinical practice. mild cognitive impairment.The New England Journal of Medicine 364, 23 (2011), 2227.Google ScholarGoogle Scholar
  37. [37] Raju Anirudh, Panchapagesan Sankaran, Liu Xing, Mandal Arindam, and Strom Nikko. 2018. Data augmentation for robust keyword spotting under playback interference. arXiv:1808.00563. https://arxiv.org/abs/1808.00563Google ScholarGoogle Scholar
  38. [38] Themistocleous Charalambos, Eckerström Marie, and Kokkinakis Dimitrios. 2018. Identification of mild cognitive impairment from speech in Swedish using deep sequential neural networks. Frontiers in Neurology 9 (2018), 975.Google ScholarGoogle Scholar
  39. [39] Themistocleous Charalambos, Kokkinakis Dimitrios, Eckerström Marie, Fraser Kathleen, and Fors Kristina Lundholm. [n.d.]. Effects of mild cognitive impairment on vowel duration. Retrieved on August 28, 2021 from https://gup.ub.gu.se/publication/270215?lang=en.Google ScholarGoogle Scholar
  40. [40] Tóth László, Hoffmann Ildikó, Gosztolya Gábor, Vincze Veronika, Szatlóczki Gréta, Bánréti Zoltán, Pákáski Magdolna, and Kálmán János. 2018. A speech recognition-based solution for the automatic detection of mild cognitive impairment from spontaneous speech. Current Alzheimer Research 15, 2 (2018), 130138.Google ScholarGoogle Scholar
  41. [41] Vincze Veronika, Gosztolya Gábor, Tóth László, Hoffmann Ildikó, and Szatlóczki Gréta. 2016. Detecting mild cognitive impairment by exploiting linguistic information from transcripts. In Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers), Berlin, Germany, August 2016. Association for Computational Linguistics, 181–187.Google ScholarGoogle Scholar
  42. [42] Wei Qiang, Franklin Amy, Cohen Trevor, and Xu Hua. 2018. Clinical text annotation-What factors are associated with the cost of time?. In AMIA Annual Symposium Proceedings, Vol. 2018. American Medical Informatics Association, 1552.Google ScholarGoogle Scholar
  43. [43] Wimo Anders, Guerchet Maëlenn, Ali Gemma-Claire, Wu Yu-Tzu, Prina A. Matthew, Winblad Bengt, Jönsson Linus, Liu Zhaorui, and Prince Martin. 2017. The worldwide costs of dementia 2015 and comparisons with 2010. Alzheimer’s & Dementia 13, 1 (2017), 17.Google ScholarGoogle Scholar
  44. [44] Yu Bea, Quatieri Thomas F., Williamson James R., and Mundt James C.. 2015. Cognitive impairment prediction in the elderly based on vocal biomarkers. In 16th Annual Conference of the International Speech Communication Association.Google ScholarGoogle Scholar

Index Terms

  1. Automatic Speech Classifier for Mild Cognitive Impairment and Early Dementia

      Recommendations

      Comments

      Login options

      Check if you have access through your login credentials or your institution to get full access on this article.

      Sign in

      Full Access

      PDF Format

      View or Download as a PDF file.

      PDF

      eReader

      View online with eReader.

      eReader

      Full Text

      View this article in Full Text.

      View Full Text

      HTML Format

      View this article in HTML Format .

      View HTML Format
      About Cookies On This Site

      We use cookies to ensure that we give you the best experience on our website.

      Learn more

      Got it!