skip to main content
research-article

Chronic Pain Protective Behavior Detection with Deep Learning

Published:15 July 2021Publication History
Skip Abstract Section

Abstract

In chronic pain rehabilitation, physiotherapists adapt physical activity to patients’ performance based on their expression of protective behavior, gradually exposing them to feared but harmless and essential everyday activities. As rehabilitation moves outside the clinic, technology should automatically detect such behavior to provide similar support. Previous works have shown the feasibility of automatic protective behavior detection (PBD) within a specific activity. In this article, we investigate the use of deep learning for PBD across activity types, using wearable motion capture and surface electromyography data collected from healthy participants and people with chronic pain. We approach the problem by continuously detecting protective behavior within an activity rather than estimating its overall presence. The best performance reaches mean F1 score of 0.82 with leave-one-subject-out cross validation. When protective behavior is modeled per activity type, performance achieves a mean F1 score of 0.77 for bend-down, 0.81 for one-leg-stand, 0.72 for sit-to-stand, 0.83 for stand-to-sit, and 0.67 for reach-forward. This performance reaches excellent level of agreement with the average experts’ rating performance suggesting potential for personalized chronic pain management at home. We analyze various parameters characterizing our approach to understand how the results could generalize to other PBD datasets and different levels of ground truth granularity.

References

  1. B. De Gelder. 2009. Why bodies? Twelve reasons for including bodily expressions in affective neuroscience. Philosophical Transactions of the Royal Society B: Biological Sciences 364, 1535 (2009), 3475–3484.Google ScholarGoogle ScholarCross RefCross Ref
  2. Andreas Bulling, Ulf Blanke, and Bernt Schiele. 2014. A tutorial on human activity recognition using body-worn inertial sensors. ACM Computing Surveys 46, 3 (2014), 1–33.Google ScholarGoogle ScholarDigital LibraryDigital Library
  3. D. K. Ahern, Michael J. Follick, James R. Council, Nancy Laser-Wolston, and Henry Litchman. 1988. Comparison of lumbar paravertebral EMG patterns in chronic low back pain patients and non-patient controls. Pain 34, 2 (1988), 153–160.Google ScholarGoogle ScholarCross RefCross Ref
  4. Fahd Albinali, Matthew S. Goodwin, and Stephen S. Intille. 2009. Recognizing stereotypical motor movements in the laboratory and classroom: A case study with children on the autism spectrum. In Proceedings of the 11th International Conference on Ubiquitous Computing (Ubicomp’09).Google ScholarGoogle Scholar
  5. M. S. H. Aung, Nadia Bianchi-Berthouze, Paul Watson, and AC de C. Williams. 2014. Automatic recognition of fear-avoidance behavior in chronic pain physical rehabilitation. In Proceedings of the 8th International Conference on Pervasive Computing Technologies for Healthcare (PervasiveHealth’14).Google ScholarGoogle ScholarDigital LibraryDigital Library
  6. M. S. H. Aung, Sebastian Kaltwang, Bernardino Romera-Paredes, Brais Martinez, Aneesha Singh, Matteo Cella, Michel Valstar, Hongying Meng, Andrew Kemp, and Moshen Shafizadeh et al., 2016. The automatic detection of chronic pain-related expression: Requirements, challenges and the multimodal EmoPain dataset. IEEE Transactions on Affective Computing 7, 4 (2016), 435–451.Google ScholarGoogle ScholarDigital LibraryDigital Library
  7. H. Breivik, Beverly Collett, Vittorio Ventafridda, Rob Cohen, and Derek Gallacher. 2006. Survey of chronic pain in Europe: Prevalence, impact on daily life, and treatment. European Journal of Pain 10, 4 (2006), 287–333.Google ScholarGoogle ScholarCross RefCross Ref
  8. Marc Bachlin, Daniel Roggen, Gerhard Troster, Meir Plotnik, Noit Inbar, Inbal Meidan, Talia Herman, Marina Brozgol, Eliya Shaviv, Nir Giladi, and others. 2009. Potentials of enhanced context awareness in wearable assistants for Parkinson’s disease patients with the freezing of gait syndrome. 13th International Symposium on Wearable Computers (ISWC) (2009).Google ScholarGoogle Scholar
  9. K. Cook, Francis Keefe, Mark P. Jensen, Toni S. Roddey, Leigh F. Callahan, Dennis Revicki, Alyssa M. Bamer, Jiseon Kim, Hyewon Chung, and Salem Rana et al. 2013. Development and validation of a new self-report measure of pain behaviors. Pain 154, 12 (2013), 2867–2876.Google ScholarGoogle ScholarCross RefCross Ref
  10. Ricardo Chavarriaga, Hesam Sagha, Alberto Calatroni, Sundara Tejaswi Digumarti, Gerhard Tröster, José del R. Millán, and Daniel Roggen. 2013. The opportunity challenge: A benchmark database for on-body sensor-based activity recognition. Pattern Recognition Letters 34, 15 (2013), 2033–2042.Google ScholarGoogle ScholarDigital LibraryDigital Library
  11. Chongyang Wang, Temitayo A. Olugbade, Akhil Mathur, Amanda C. De C. Williams, Nicholas D. Lane, and Nadia Bianchi-Berthouze. 2019. Recurrent network based automatic detection of chronic pain protective behavior using MoCap and sEMG data. In Proceedings of the 23rd ACM International Symposium on Wearable Computers (ISWC’19). 225–230.Google ScholarGoogle ScholarDigital LibraryDigital Library
  12. Cui Yin, Menglin Jia, Tsung-Yi Lin, Yang Song, and Serge Belongie. 2019. Class-balanced loss based on effective number of samples. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR’19). 9268–9277.Google ScholarGoogle Scholar
  13. J. P. Dickey, Michael R. Pierrynowski, Drew A. Bednar, and Simon X. Yang. 2013. Relationship between pain and vertebral motion in chronic low-back pain subjects. Clinical Biomechanics 7, 4 (2013), 412–418.Google ScholarGoogle Scholar
  14. Daniel Roggen, Daniel Roggen, Georg Ogris, Paul Lukowicz, and Gerhard Tröster. 2008. Wearable activity tracking in car manufacturing. IEEE Pervasive Computing 7, 2 (2008), 42–50.Google ScholarGoogle Scholar
  15. W. E. Fordyce, David Lansky, Donald A. Calsyn, John L. Shelton, Walter C. Stolov, and Daniel L. Rock. 1984. Pain measurement and pain behavior. Pain 18, 1 (1984), 53–69.Google ScholarGoogle Scholar
  16. H. Grip, Fredrik Ohberg, Urban Wiklund, Ylva Sterner, J. Stefan Karlsson, Björn Gerdle. 2013. Classification of neck movement patterns related to whiplash-associated disorders using neural networks. IEEE Transactions on Information Technology in Biomedicine 7, 4 (2013), 412–418.Google ScholarGoogle Scholar
  17. Greff Klaus, Rupesh K. Srivastava, Jan Koutník, Bas R. Steunebrink, and Jürgen Schmidhuber. 2017. LSTM: A search space odyssey. IEEE Transactions on Neural Networks and Learning Systems 28, 10 (2017), 2222–2232.Google ScholarGoogle Scholar
  18. Matthew S. Goodwin et al. 2014. Moving towards a real-time system for automatically recognizing stereotypical motor movements in individuals on the autism spectrum using wireless accelerometry. In Proceedings of the 16th International Joint Conference on Pervasive and Ubiquitous Computing (UbiComp’14).Google ScholarGoogle Scholar
  19. Guan Yu and Thomas Plötz. 2017. Ensembles of deep LSTM learners for activity recognition using wearables. In Proceedings of the ACM on Interactive, Mobile, Wearable, and Ubiquitous Technologies (IMWUT’17). Article 11.Google ScholarGoogle Scholar
  20. Nils Y. Hammerla, Shane Halloran, and Thomas Plötz. 2016. Deep, convolutional, and recurrent models for human activity recognition using wearables. arXiv:1604.08880Google ScholarGoogle Scholar
  21. Sepp Hochreiter and Jürgen Schmidhuber. 1997. Long short-term memory. Neural Computation 9, 8 (1997), 1735–1780.Google ScholarGoogle ScholarDigital LibraryDigital Library
  22. Sojeong Ha and Seungjin Choi. 2016. Convolutional neural networks for human activity recognition using multiple accelerometer and gyroscope sensors. In Proceedings of the International Joint Conference on Neural Networks (IJCNN’16). 381–388.Google ScholarGoogle ScholarCross RefCross Ref
  23. T. Huynh, Ulf Blanke and Bernt Schiele. 2007. Scalable recognition of daily activities with wearable sensors. In Location- and Context-Awareness. Lecture Notes in Computer Science, Vol. 4718. Springer, 50–67.Google ScholarGoogle Scholar
  24. Andrea Kleinsmith et al. 2011. Automatic recognition of non-acted affective postures. IEEE Transactions on Systems, Man, and Cybernetics, Part B (Cybernetics) 41, 4 (2011), 1027–1038.Google ScholarGoogle Scholar
  25. Andrea Kleinsmith and Nadia Bianchi-Berthouze. 2013. Affective body expression perception and recognition: A survey. IEEE Transactions on Affective Computing 4, 1 (2013), 15–33.Google ScholarGoogle ScholarDigital LibraryDigital Library
  26. The Pain Consortium. 2016. The pain consortium. UK Pain Messages: Pain News (2016), 21–22.Google ScholarGoogle Scholar
  27. Diederik P. Kingma and Jimmy Ba. 2014. Adam: A method for stochastic optimization. arXiv:1412.6980Google ScholarGoogle Scholar
  28. M. Karg, Nadia Bianchi-Berthouze, and Anthony Steed. 2013. Body movements for affective expression: A survey of automatic recognition and generation. IEEE Transactions on Affective Computing 4, 4 (2013), 341–359.Google ScholarGoogle Scholar
  29. Simon Kohl, Bernardino Romera-Paredes, Clemens Meyer, Jeffrey De Fauw, Joseph R. Ledsam, Klaus H. Maier-Hein, S. M. Eslami, Danilo Rezende, Jimenez, and Olaf Ronneberger. 2018. A probabilistic U-Net for segmentation of ambiguous images. Advances in Neural Information Processing Systems (NeurIPS’18), Vol. 31.6965–6975.Google ScholarGoogle Scholar
  30. Patrick Lucey, Jeffrey F. Cohn, Kenneth M. Prkachin, and Patricia E. Solomon, Iain Matthews. 2011. Painful data: The UNBC-McMaster shoulder pain expression archive database. In Proceedings of the IEEE International Conference on Automatic Face and Gesture Recognition and Workshops (FG’11).Google ScholarGoogle Scholar
  31. F. J. O. Morales and Daniel Roggen. 2008. Deep convolutional feature transfer across mobile activity recognition domains, sensor modalities and locations. In Proceedings of the 12th International Symposium on Wearable Computers (ISWC’08).Google ScholarGoogle Scholar
  32. K. O. McGraw and Seok P. Wong. 1996. Forming inferences about some intraclass correlation coefficients. Psychological Methods 1, 1 (1996), 30–46.Google ScholarGoogle ScholarCross RefCross Ref
  33. Riccardo Miotto, Fei Wang, Shuang Wang, Xiaoqian Jiang, and Joel T. Dudley. 2017. Deep learning for healthcare: Review, opportunities and challenges. Briefings in Bioinformatics 19, 6 (2017), 1236–1246.Google ScholarGoogle ScholarCross RefCross Ref
  34. T. A. Olugbade, Min S. H. Aung, Nadia Bianchi-Berthouze, Nicolai Marquardt, and Amanda C. De C. Williams. 2014. Bi-modal detection of painful reaching for chronic pain rehabilitation systems. In Proceedings of the 16th International Conference on Multimodal Interaction (ICMI’14). 455–458.Google ScholarGoogle Scholar
  35. T. A. Olugbade, Nadia Bianchi-Berthouze, Nicolai Marquardt, and Amanda C. De C. Williams. 2015. Pain level recognition using kinematics and muscle activity for physical rehabilitation in chronic pain. In Proceedings of the 6th Conference on Affective Computing and Intelligent Interaction (ACII’15). 243–249.Google ScholarGoogle Scholar
  36. T. A. Olugbade, Nadia Bianchi-Berthouze, Nicolai Marquardt, and Amanda C. De C. Williams. 2018. Human observer and automatic assessment of movement related self-efficacy in chronic pain: From movement to functional activity. IEEE Transactions on Affective Computing 11, 2 (2018), 214–229.Google ScholarGoogle ScholarCross RefCross Ref
  37. T. A. Olugbade, Aneesha Singh, Nadia Bianchi-Berthouze, Nicolai Marquardt, Min SH Aung, and Amanda C. De C. Williams. 2019. How can affect be detected and represented in technological support for physical rehabilitation?ACM Transactions on Computer-Human Interaction 26, 1 (2019), Article 1.Google ScholarGoogle Scholar
  38. Enrica Papi, Athina Belsi, and Alison H. McGregor. 2015. A knee monitoring device and the preferences of patients living with osteoarthritis: A qualitative study. BMJ Open 5, 9 (2015), 3007980.Google ScholarGoogle Scholar
  39. Enrica Papi, Ged M. Murtagh, and Alison H. McGregor. 2016. Wearable technologies in osteoarthritis: A qualitative study of clinicians’ preferences. BMJ Open 6 (2016), 2044–6055.Google ScholarGoogle Scholar
  40. Kenneth M. Prkachin et al. 2008. The structure, reliability and validity of pain expression: Evidence from patients with shoulder pain. Pain 139, 2 (2008), 267–274.Google ScholarGoogle ScholarCross RefCross Ref
  41. N. M. Rad and Cesare Furlanello. 2016. Applying deep learning to stereotypical motor movement detection in autism spectrum disorders. In Proceedings of the 16th International Conference on Data Mining Workshops (ICDMW’16).Google ScholarGoogle Scholar
  42. Attila Reiss and Didier Stricker. 2012. Introducing a new benchmarked dataset for activity monitoring. In Proceedings of the 16th International Symposium on Wearable Computers (ISWC’12).Google ScholarGoogle Scholar
  43. J. Rivas, Maria del Carmen Lara, Luis Castrejon, Jorge Hernandez-Franco, Felipe Orihuela-Espina, Lorena Palafox, Amanda Williams, Nadia Berthouze, and Enrique Sucar. 2021. Multi-label and multimodal classifier for affective states recognition in virtual rehabilitation. IEEE Transactions on Affective Computing. Early access, February 1, 2021.Google ScholarGoogle Scholar
  44. A. Singh, Annina Klapper, Jinni Jia, Antonio Fidalgo, Ana Tajadura-Jiménez, Natalie Kanakam, NadiaBianchi-Berthouze, and Amanda Williams. 2014. Motivating people with chronic pain to do physical activity: Opportunities for technology design. In Proceedings of the International Conference on Human Factors in Computing Systems (CHI’14). 2803–2812.Google ScholarGoogle Scholar
  45. A. Singh, Stefano Piana, Davide Pollarolo, Gualtiero Volpe, Giovanna Varni, Ana Tajadura-Jimenez, Amanda CdeC Williams, Antonio Camurri, and Nadia Bianchi-Berthouze. 2016. Go-with-the-flow: Tracking, analysis and sonification of movement and breathing to build confidence in activity despite chronic pain. Human–Computer Interaction 31, 3-4 (2016), 1–49.Google ScholarGoogle ScholarDigital LibraryDigital Library
  46. A. Singh, Nadia Bianchi-Berthouze, and Amanda C. de C. Williams. 2017. Supporting everyday function in chronic pain using wearable technology. In Proceedings of the International Conference on Human Factors in Computing Systems (CHI’17). 3903–3915.Google ScholarGoogle Scholar
  47. M. J. L. Sullivan, Pascal Thibault, André Savard, Richard Catchlove, John Kozey, and William D. Stanish. 2006. The influence of communication goals and physical demands on different dimensions of pain behavior. Pain 125, 3 (2006), 270–277.Google ScholarGoogle ScholarCross RefCross Ref
  48. T. T. Um, Franz M. J. Pfister, Daniel Pichler, Satoshi Endo, Muriel Lang, Sandra Hirche, Urban Fietzek, and Dana Kulić. 2017. Data augmentation of wearable sensor data for Parkinson’s disease monitoring using convolutional neural networks. In Proceedings of the 19th ACM International Conference on Multimodal Interaction (ICMI’17). 216–220.Google ScholarGoogle ScholarDigital LibraryDigital Library
  49. J. W. S. Vlaeyen, Stephen Morley, and Geert Crombez. 2016. The experimental analysis of the interruptive, interfering, and identity-distorting effects of chronic pain. Behaviour Research and Therapy 86 (2016), 23–34.Google ScholarGoogle ScholarCross RefCross Ref
  50. Limin Wang, Yu Qiao, and Xiaoou Tang. 2015. Action recognition with trajectory-pooled deep-convolutional descriptors. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR’15). 4305–4314.Google ScholarGoogle Scholar
  51. P. J. Watson, C. Kerry Booker, and Chris J. Main. 1997. Evidence for the role of psychological factors in abnormal paraspinal activity in patients with chronic low back pain. Journal of Musculoskeletal Pain 5, 4 (1997), 41–56.Google ScholarGoogle ScholarCross RefCross Ref
  52. D. V. Cicchetti. 1994. Guidelines, criteria, and rules of thumb for evaluating normed and standardized assessment instruments in psychology. Psychological Assess 6, 4 (1994), 284–290.Google ScholarGoogle ScholarCross RefCross Ref
  53. I. Tracey and M. C. Bushnell. 2009. How neuroimaging studies have challenged us to rethink: Is chronic pain a disease? Journal of Pain 10, 11 (2009), 1113–1120.Google ScholarGoogle ScholarCross RefCross Ref
  54. F. J. Keefe and A. R. Block 1982. Development of an observation method for assessing pain behavior in chronic low back pain patients. Behavior Therapy 13, 14 (1982), 363–375.Google ScholarGoogle ScholarCross RefCross Ref
  55. K. A. Hallgren. 2012. Computing inter-rater reliability for observational data: An overview and tutorial. Tutorials in Quantitative Methods for Psychology 8, 1 (2012), 23–24.Google ScholarGoogle ScholarCross RefCross Ref
  56. R. N. Jamison. 2016. Are we really ready for telehealth cognitive behavioral therapy for pain?Pain 158, 4 (2016), 538–540.Google ScholarGoogle Scholar
  57. J. W. S. Vlaeyen and S. J. Linton. 2000. Fear-avoidance and its consequences in chronic musculoskeletal pain: A state of the artPain 85, 3 (2000), 317–332.Google ScholarGoogle Scholar

Index Terms

  1. Chronic Pain Protective Behavior Detection with Deep Learning

      Recommendations

      Comments

      Login options

      Check if you have access through your login credentials or your institution to get full access on this article.

      Sign in

      Full Access

      PDF Format

      View or Download as a PDF file.

      PDF

      eReader

      View online with eReader.

      eReader

      HTML Format

      View this article in HTML Format .

      View HTML Format
      About Cookies On This Site

      We use cookies to ensure that we give you the best experience on our website.

      Learn more

      Got it!