skip to main content
research-article

Generalized and Efficient Skill Assessment from IMU Data with Applications in Gymnastics and Medical Training

Authors Info & Claims
Published:30 December 2020Publication History
Skip Abstract Section

Abstract

Human activity recognition is progressing from automatically determining what a person is doing and when, to additionally analyzing the quality of these activities—typically referred to as skill assessment. In this chapter, we propose a new framework for skill assessment that generalizes across application domains and can be deployed for near-real-time applications. It is based on the notion of repeatability of activities defining skill. The analysis is based on two subsequent classification steps that analyze (1) movements or activities and (2) their qualities, that is, the actual skills of a human performing them. The first classifier is trained in either a supervised or unsupervised manner and provides confidence scores, which are then used for assessing skills. We evaluate the proposed method in two scenarios: gymnastics and surgical skill training of medical students. We demonstrate both the overall effectiveness and efficiency of the generalized assessment method, especially compared to previous work.

References

  1. Ludovic Baudry, Ludovic Seifert, and David Leroy. 2008. Spatial consistency of circle on the pedagogic pommel horse: Influence of expertise.Journal of Strength and Conditioning Research 22, 2 (2008), 608--613.Google ScholarGoogle Scholar
  2. Karen T. Beatty, Andrew S. Mcintosh, and Bertrand O. Frechede. 2006. Method for analysing the risk of overuse injury in gymnastics. In Proc. Int. Symp. on Biomechanics in Sports. 1--4.Google ScholarGoogle Scholar
  3. Eugen Berlin and Kristof Van Laerhoven. 2012. Detecting leisure activities with dense motif discovery. In Proc. ACM Int. Joint Conf. Ubiquitous and Pervasive Comp. (UbiComp’12). ACM, New York, NY, 250--259. DOI:https://doi.org/10.1145/2370216.2370257Google ScholarGoogle Scholar
  4. Peter Blank, Benjamin H. Groh, and Bjoern M. Eskofier. 2017. Ball speed and spin estimation in table tennis using a racket-mounted inertial sensor. In Proc. Int. Symp. Wearable Computing (ISWC’17). ACM, New York, NY. http://doi.acm.org/10.1145/3123021.3123040.Google ScholarGoogle Scholar
  5. Aaron F. Bobick. 1997. Movement, activity and action: The role of knowledge in the perception of motion. Philosophical Transactions of the Royal Society of London. Series B, Biological sciences 352, 1358 (Aug. 1997), 1257--1265. DOI:https://doi.org/10.1098/rstb.1997.0108Google ScholarGoogle Scholar
  6. Elizabeth J. Bradshaw and Patria A. Hume. 2012. Biomechanical approaches to identify and quantify injury mechanisms and risk factors in women’s artistic gymnastics. Sports Biomechanics 11, 3 (2012), 324--341. DOI:https://doi.org/10.1080/14763141.2011.650186Google ScholarGoogle Scholar
  7. Leo Breiman, Jerome Friedman, Charles J. Stone, and Richard A. Olshen. 1984. Classification and Regression Trees. Chapman 8 Hall, New York, NY. 358 pages. http://www.crcpress.com/catalog/C4841.htm.Google ScholarGoogle Scholar
  8. Andreas Bulling, Ulf Blanke, and Bernt Schiele. 2014. A tutorial on human activity recognition using body-worn inertial sensors. Computing Surveys 46, 3 (2014), 1--33. DOI:https://doi.org/10.1145/2499621Google ScholarGoogle ScholarDigital LibraryDigital Library
  9. Lauren Burt, Geraldine Naughton, and Raul Landeo. 2007. Quantifying impacts during beam and floor training in pre-adolescent girls from two streams of artistic gymnastics. In Proc. Int. Symp. on Biomechanics in Sports. 354--357.Google ScholarGoogle Scholar
  10. Samprit Chatterjee and Ali S. Hadi. 2006. Regression Analysis by Example. John Wiley 8 Sons, Inc. DOI:https://doi.org/10.1002/0470055464Google ScholarGoogle Scholar
  11. Liming Chen, J. Hoey, C. D. Nugent, D. J. Cook, and Zhiwen Yu. 2012. Sensor-based activity recognition. IEEE Transactions on Systems, Man, and Cybernetics, Part C (Applications and Reviews) 42, 6 (Nov. 2012), 790--808. DOI:https://doi.org/10.1109/TSMCC.2012.2198883Google ScholarGoogle ScholarDigital LibraryDigital Library
  12. John W. Chow and Duane V. Knudson. 2011. Use of deterministic models in sports and exercise biomechanics research. Sports Biomechanics 10, 3 (Sept. 2011), 219--233. DOI:https://doi.org/10.1080/14763141.2011.592212Google ScholarGoogle Scholar
  13. Ruth H. Da-Silva, Frederike van Wijck, Lisa Shaw, Helen Rodgers, Madeline Balaam, Lianne Brkic, Thomas Ploetz, Dan Jackson, Karim Ladha, and Christopher I. Price. 2018. Prompting arm activity after stroke: A clinical proof of concept study of wrist-worn accelerometers with a vibrating alert function. Journal of Rehabilitation and Assistive Technologies Engineering 5 (May 2018), 1--8. DOI:https://doi.org/10.1177/2055668318761524Google ScholarGoogle Scholar
  14. Ruth H. Da-Silva, Frederike van Wijck, Lisa Shaw, Helen Rodgers, Madeline Balaam, Lianne Brkic, Thomas Ploetz, Dan Jackson, Karim Ladha, and Christopher I. Price. 2018. Prompting arm activity after stroke: A clinical proof of concept study of wrist-worn accelerometers with a vibrating alert function. Journal of Rehabilitation and Assistive Technologies Engineering 5 (2018), 2055668318761524. DOI:https://doi.org/10.1177/2055668318761524Google ScholarGoogle Scholar
  15. Vivek Datta, Simon Bann, Mirren Mandalia, and Ara Darzi. 2006. The surgical efficiency score: A feasible, reliable, and valid method of skills assessment. American Journal of Surgery 192, 3 (Sept. 2006), 372--378. DOI:https://doi.org/10.1016/j.amjsurg.2006.06.001Google ScholarGoogle Scholar
  16. Davide G. de Sousa, Lisa A. Harvey, Simone Dorsch, and Joanne V. Glinsky. 2018. Interventions involving repetitive practice improve strength after stroke: A systematic review. Journal of Physiotherapy 64, 4 (2018), 210--221. DOI:https://doi.org/10.1016/j.jphys.2018.08.004Google ScholarGoogle ScholarCross RefCross Ref
  17. Hazel Doughty, Dima Damen, and Walterio Mayol-Cuevas. 2017. Who’s better? Who’s best? Pairwise deep ranking for skill determination. In Proc. Conf. Computer Vision and Pattern Recognition (CVPR’17).Google ScholarGoogle Scholar
  18. Benjamin H. Groh, Martin Fleckenstein, Thomas Kautz, and Björn Eskofier. 2017. Classification and visualization of skateboard tricks using wearable sensors.Pervasive and Mobile Computing (PMC) 40, C (Sept. 2017), 42--55. DOI:https://doi.org/10.1016/j.pmcj.2017.05.007Google ScholarGoogle Scholar
  19. Nils Hammerla, Reuben Kirkham, Peter Andras, and Thomas Plötz. 2013. On preserving statistical characteristics of accelerometry data using their empirical cumulative distribution. In Proc. Int. Symp. Wearable Computing (ISWC). ACM, New York, NY, USA, 65--68. DOI:https://doi.org/10.1145/2493988.2494353Google ScholarGoogle Scholar
  20. Nils Y. Hammerla and Thomas Plötz. 2015. Let’s (not) stick together: Pairwise similarity biases cross-validation in activity recognition. In Proc. Ubicomp.Google ScholarGoogle Scholar
  21. Jesse Hoey, Thomas Plötz, Dan Jackson, Andrew Monk, Cuong Pham, and Patrick Olivier. 2011. Rapid specification and automated generation of prompting systems to assist people with dementia. Pervasive and Mobile Computing (PMC) 7, 3 (June 2011), 299--318. DOI:https://doi.org/10.1016/j.pmcj.2010.11.007Google ScholarGoogle Scholar
  22. Aftab Khan, Sebastian Mellor, Eugen Berlin, Robin Thompson, Roisin McNaney, Patrick Olivier, and Thomas Plötz. 2015. Beyond activity recognition: Skill assessment from accelerometer data. In Proc. ACM Int. Joint Conf. Ubiquitous and Pervasive Comp. (UbiComp’15). ACM, 1155--1166. DOI:https://doi.org/10.1145/2750858.2807534Google ScholarGoogle Scholar
  23. Aftab Khan, James Nicholson, and Thomas Plötz. 2017. Activity recognition for quality assessment of batting shots in cricket using a hierarchical representation. Proc. ACM Interactive, Mobile, Wearable and Ubiquitous Computing (IMWUT) 1, 3 (Sept. 2017), 62:1--62:31. DOI:https://doi.org/10.1145/3130927Google ScholarGoogle Scholar
  24. Aftab Khan, David Windridge, and Josef Kittler. 2014. Multilevel chinese takeaway process and label-based processes for rule induction in the context of automated sports video annotation. IEEE Transactions on Cybernetics 44, 10 (Oct. 2014), 1910--1923. DOI:https://doi.org/10.1109/tcyb.2014.2299955Google ScholarGoogle ScholarCross RefCross Ref
  25. Hyeokhyen Kwon, Gregory D. Abowd, and Thomas Plötz. 2018. Adding structural characteristics to distribution-based accelerometer representations for activity recognition using wearables. In Proc. Int. Symp. Wearable Computing (ISWC’18). 72--75.Google ScholarGoogle Scholar
  26. Cassim Ladha, Nils Y. Hammerla, Patrick Olivier, and Thomas Plötz. 2013. ClimbAX: Skill assessment for climbing enthusiasts. In Proc. ACM Int. Joint Conf. Ubiquitous and Pervasive Comp. (UbiComp’13). ACM, 235--244. DOI:https://doi.org/10.1145/2493432.2493492Google ScholarGoogle Scholar
  27. Claudine J. C. Lamoth, Rob C. van Lummel, and Peter J. Beek. 2009. Athletic skill level is reflected in body sway: A test case for accelometry in combination with stochastic dynamics. Gait 8 Posture 29, 4 (June 2009), 546--551. DOI:https://doi.org/10.1016/j.gaitpost.2008.12.006Google ScholarGoogle Scholar
  28. Oscar D. Lara and Miguel A. Labrador. 2013. A survey on human activity recognition using wearable sensors. IEEE Communications Surveys 8 Tutorials 15, 3 (2013), 1192--1209. DOI:https://doi.org/10.1109/surv.2012.110112.00192Google ScholarGoogle Scholar
  29. Jessica Lin, Eamonn Keogh, Li Wei, and Stefano Lonardi. 2007. Experiencing SAX: A novel symbolic representation of time series. Data Mining and Knowledge Discovery 15, 2 (April 2007), 107--144. DOI:https://doi.org/10.1007/s10618-007-0064-zGoogle ScholarGoogle Scholar
  30. Thomas B. Moeslund, Adrian Hilton, and Volker Krüger. 2006. A survey of advances in vision-based human motion capture and analysis. Computer Vision and Image Understanding 104, 2--3 (Nov. 2006), 90--126. DOI:https://doi.org/10.1016/j.cviu.2006.08.002Google ScholarGoogle ScholarDigital LibraryDigital Library
  31. Andreas Möller, Luis Roalter, Stefan Diewald, Matthias Kranz, Nils Hammerla, Patrick Olivier, and Thomas Plötz. 2012. GymSkill: A personal trainer for physical exercises. In Proc. IEEE Conf. Pervasive Comp. and Communication (PerCom’12). 213--220. DOI:https://doi.org/10.1109/PerCom.2012.6199869Google ScholarGoogle Scholar
  32. Paritosh Parmar and Brendan Tran Morris. 2017. Learning to score olympic events. In Proc. Conf. Computer Vision and Pattern Recognition (CVPR’17).Google ScholarGoogle Scholar
  33. John C. Platt. 1999. Probabilistic outputs for support vector machines and comparisons to regularized likelihood methods. In Advances In Large Margin Classifiers. MIT Press, 61--74.Google ScholarGoogle Scholar
  34. Thomas Plötz, Chen Chenn, Nils Y. Hammerla, and Gregory D. Abowd. 2012. Automatic synchronization of wearable sensors and video-cameras for ground truth annotation -- A practical approach. In Proc. Int. Symp. Wearable Computing (ISWC’12). 100--103. DOI:https://doi.org/10.1109/ISWC.2012.15Google ScholarGoogle Scholar
  35. Thomas Plötz, Nils Y. Hammerla, and Patrick Olivier. 2011. Feature learning for activity recognition in ubiquitous computing. In Proc. Int. Joint Conf. Artificial Intelligence (IJCAI’11). AAAI Press, 1729--1734. DOI:https://doi.org/10.5591/978-1-57735-516-8/IJCAI11-290Google ScholarGoogle Scholar
  36. Carl Edward Rasmussen and Christopher K. I. Williams. 2006. Gaussian Processes for Machine Learning. MIT Press.Google ScholarGoogle ScholarDigital LibraryDigital Library
  37. Martin Seiffert, Flavio Holstein, Rainer Schlosser, and Jochen Schiller. 2017. Next generation cooperative wearables: Generalized activity assessment computed fully distributed within a wireless body area network. IEEE Access 5 (Sept. 2017), 16793--16807. DOI:https://doi.org/10.1109/access.2017.2749005Google ScholarGoogle Scholar
  38. Yachna Sharma, Vinay Bettadapura, Thomas Plötz, Nils Hammerla, Sebastian Mellor, Roisin McNaney, Patrick Olivier, Sandeep Deshmukh, Andrew Mccaskie, and Irfan Essa. 2014. Video based assessment of OSATS using sequential motion textures. In Proc. 5th Workshop on Modeling and Monitoring of Computer Assisted Interventions (M2CAI’14).Google ScholarGoogle Scholar
  39. Yachna Sharma, Thomas Plötz, Nils Hammerla, Sebastian Mellor, Roisin McNaney, Patrick Olivier, Sandeep Deshmukh, Andrew Mccaskie, and Irfan Essa. 2014. Automated surgical OSATS prediction from videos. In Proc. IEEE Int. Symposium on Biomedical Imaging (ISBI’14). DOI:https://doi.org/10.1109/isbi.2014.6867908Google ScholarGoogle Scholar
  40. Chantal Simons and Elizabeth J. Bradshaw. 2016. Reliability of accelerometry to assess impact loads of jumping and landing tasks. Sports Biomechanics 15, 1 (Jan. 2016), 1--10. DOI:https://doi.org/10.1080/14763141.2015.1091032Google ScholarGoogle Scholar
  41. Alex J. Smola and Bernhard Schölkopf. 2004. A tutorial on support vector regression. Statistics and Computing 14, 3 (Aug 2004), 199--222. DOI:https://doi.org/10.1023/B:STCO.0000035301.49549.88Google ScholarGoogle ScholarDigital LibraryDigital Library
  42. Robin Thompson, Ilias Kyriazakis, Amey Holden, Patrick Olivier, and Thomas Plötz. 2015. Dancing with horses: Automated quality feedback for dressage riders. In Proc. ACM Int. Joint Conf. Ubiquitous and Pervasive Comp. (UbiComp’15). ACM, 325--336. DOI:https://doi.org/10.1145/2750858.2807536Google ScholarGoogle Scholar
  43. Eduardo Velloso, Andreas Bulling, Hans Gellersen, Wallace Ugulino, and Hugo Fuks. 2013. Qualitative activity recognition of weight lifting exercises. In Proc. Int. Conf. Augmented Human. ACM, New York, NY, 116--123. DOI:https://doi.org/10.1145/2459236.2459256Google ScholarGoogle Scholar
  44. Peter Wittenburg, Hennie Brugman, Albert Russel, Alex Klassmann, and Han Sloetjes. 2006. ELAN: A professional framework for multimodality research. In Proc. Int. Conf. Language Resources and Evaluation Conference (LREC’06).Google ScholarGoogle Scholar
  45. Bob Woolmer, Timothy Noakes, Helen Moffett, and Fay Lewis. 2008. Bob Woolmer’s art and Science of Cricket. New Holland London.Google ScholarGoogle Scholar
  46. Ting-Fan Wu, Chih-Jen Lin, and Ruby C. Weng. 2004. Probability estimates for multi-class classification by pairwise coupling. Journal of Machine Learning Research 5 (Dec. 2004), 975--1005. http://dl.acm.org/citation.cfm?id=1005332.1016791.Google ScholarGoogle Scholar
  47. Aneeq Zia, Yachna Sharma, Vinay Bettadapura, Eric L. Sarin, Thomas Plötz, Mark A. Clements, and Irfan Essa. 2016. Automated video-based assessment of surgical skills for training and evaluation in medical schools. International Journal of Computer Assisted Radiology and Surgery 11, 9 (Aug. 2016), 1623--1636. DOI:https://doi.org/10.1007/s11548-016-1468-2Google ScholarGoogle Scholar

Index Terms

  1. Generalized and Efficient Skill Assessment from IMU Data with Applications in Gymnastics and Medical Training

              Recommendations

              Comments

              Login options

              Check if you have access through your login credentials or your institution to get full access on this article.

              Sign in

              Full Access

              • Published in

                cover image ACM Transactions on Computing for Healthcare
                ACM Transactions on Computing for Healthcare  Volume 2, Issue 1
                Special Issue on Wearable Technologies for Smart Health: Part 2 and Regular Papers
                January 2021
                204 pages
                ISSN:2691-1957
                EISSN:2637-8051
                DOI:10.1145/3446563
                Issue’s Table of Contents

                Copyright © 2020 ACM

                Publisher

                Association for Computing Machinery

                New York, NY, United States

                Publication History

                • Published: 30 December 2020
                • Accepted: 1 August 2020
                • Revised: 1 March 2020
                • Received: 1 August 2019
                Published in health Volume 2, Issue 1

                Permissions

                Request permissions about this article.

                Request Permissions

                Check for updates

                Qualifiers

                • research-article
                • Research
                • Refereed

              PDF Format

              View or Download as a PDF file.

              PDF

              eReader

              View online with eReader.

              eReader

              HTML Format

              View this article in HTML Format .

              View HTML Format
              About Cookies On This Site

              We use cookies to ensure that we give you the best experience on our website.

              Learn more

              Got it!