Abstract
Human activity recognition is progressing from automatically determining what a person is doing and when, to additionally analyzing the quality of these activities—typically referred to as skill assessment. In this chapter, we propose a new framework for skill assessment that generalizes across application domains and can be deployed for near-real-time applications. It is based on the notion of repeatability of activities defining skill. The analysis is based on two subsequent classification steps that analyze (1) movements or activities and (2) their qualities, that is, the actual skills of a human performing them. The first classifier is trained in either a supervised or unsupervised manner and provides confidence scores, which are then used for assessing skills. We evaluate the proposed method in two scenarios: gymnastics and surgical skill training of medical students. We demonstrate both the overall effectiveness and efficiency of the generalized assessment method, especially compared to previous work.
- Ludovic Baudry, Ludovic Seifert, and David Leroy. 2008. Spatial consistency of circle on the pedagogic pommel horse: Influence of expertise.Journal of Strength and Conditioning Research 22, 2 (2008), 608--613.Google Scholar
- Karen T. Beatty, Andrew S. Mcintosh, and Bertrand O. Frechede. 2006. Method for analysing the risk of overuse injury in gymnastics. In Proc. Int. Symp. on Biomechanics in Sports. 1--4.Google Scholar
- Eugen Berlin and Kristof Van Laerhoven. 2012. Detecting leisure activities with dense motif discovery. In Proc. ACM Int. Joint Conf. Ubiquitous and Pervasive Comp. (UbiComp’12). ACM, New York, NY, 250--259. DOI:https://doi.org/10.1145/2370216.2370257Google Scholar
- Peter Blank, Benjamin H. Groh, and Bjoern M. Eskofier. 2017. Ball speed and spin estimation in table tennis using a racket-mounted inertial sensor. In Proc. Int. Symp. Wearable Computing (ISWC’17). ACM, New York, NY. http://doi.acm.org/10.1145/3123021.3123040.Google Scholar
- Aaron F. Bobick. 1997. Movement, activity and action: The role of knowledge in the perception of motion. Philosophical Transactions of the Royal Society of London. Series B, Biological sciences 352, 1358 (Aug. 1997), 1257--1265. DOI:https://doi.org/10.1098/rstb.1997.0108Google Scholar
- Elizabeth J. Bradshaw and Patria A. Hume. 2012. Biomechanical approaches to identify and quantify injury mechanisms and risk factors in women’s artistic gymnastics. Sports Biomechanics 11, 3 (2012), 324--341. DOI:https://doi.org/10.1080/14763141.2011.650186Google Scholar
- Leo Breiman, Jerome Friedman, Charles J. Stone, and Richard A. Olshen. 1984. Classification and Regression Trees. Chapman 8 Hall, New York, NY. 358 pages. http://www.crcpress.com/catalog/C4841.htm.Google Scholar
- Andreas Bulling, Ulf Blanke, and Bernt Schiele. 2014. A tutorial on human activity recognition using body-worn inertial sensors. Computing Surveys 46, 3 (2014), 1--33. DOI:https://doi.org/10.1145/2499621Google Scholar
Digital Library
- Lauren Burt, Geraldine Naughton, and Raul Landeo. 2007. Quantifying impacts during beam and floor training in pre-adolescent girls from two streams of artistic gymnastics. In Proc. Int. Symp. on Biomechanics in Sports. 354--357.Google Scholar
- Samprit Chatterjee and Ali S. Hadi. 2006. Regression Analysis by Example. John Wiley 8 Sons, Inc. DOI:https://doi.org/10.1002/0470055464Google Scholar
- Liming Chen, J. Hoey, C. D. Nugent, D. J. Cook, and Zhiwen Yu. 2012. Sensor-based activity recognition. IEEE Transactions on Systems, Man, and Cybernetics, Part C (Applications and Reviews) 42, 6 (Nov. 2012), 790--808. DOI:https://doi.org/10.1109/TSMCC.2012.2198883Google Scholar
Digital Library
- John W. Chow and Duane V. Knudson. 2011. Use of deterministic models in sports and exercise biomechanics research. Sports Biomechanics 10, 3 (Sept. 2011), 219--233. DOI:https://doi.org/10.1080/14763141.2011.592212Google Scholar
- Ruth H. Da-Silva, Frederike van Wijck, Lisa Shaw, Helen Rodgers, Madeline Balaam, Lianne Brkic, Thomas Ploetz, Dan Jackson, Karim Ladha, and Christopher I. Price. 2018. Prompting arm activity after stroke: A clinical proof of concept study of wrist-worn accelerometers with a vibrating alert function. Journal of Rehabilitation and Assistive Technologies Engineering 5 (May 2018), 1--8. DOI:https://doi.org/10.1177/2055668318761524Google Scholar
- Ruth H. Da-Silva, Frederike van Wijck, Lisa Shaw, Helen Rodgers, Madeline Balaam, Lianne Brkic, Thomas Ploetz, Dan Jackson, Karim Ladha, and Christopher I. Price. 2018. Prompting arm activity after stroke: A clinical proof of concept study of wrist-worn accelerometers with a vibrating alert function. Journal of Rehabilitation and Assistive Technologies Engineering 5 (2018), 2055668318761524. DOI:https://doi.org/10.1177/2055668318761524Google Scholar
- Vivek Datta, Simon Bann, Mirren Mandalia, and Ara Darzi. 2006. The surgical efficiency score: A feasible, reliable, and valid method of skills assessment. American Journal of Surgery 192, 3 (Sept. 2006), 372--378. DOI:https://doi.org/10.1016/j.amjsurg.2006.06.001Google Scholar
- Davide G. de Sousa, Lisa A. Harvey, Simone Dorsch, and Joanne V. Glinsky. 2018. Interventions involving repetitive practice improve strength after stroke: A systematic review. Journal of Physiotherapy 64, 4 (2018), 210--221. DOI:https://doi.org/10.1016/j.jphys.2018.08.004Google Scholar
Cross Ref
- Hazel Doughty, Dima Damen, and Walterio Mayol-Cuevas. 2017. Who’s better? Who’s best? Pairwise deep ranking for skill determination. In Proc. Conf. Computer Vision and Pattern Recognition (CVPR’17).Google Scholar
- Benjamin H. Groh, Martin Fleckenstein, Thomas Kautz, and Björn Eskofier. 2017. Classification and visualization of skateboard tricks using wearable sensors.Pervasive and Mobile Computing (PMC) 40, C (Sept. 2017), 42--55. DOI:https://doi.org/10.1016/j.pmcj.2017.05.007Google Scholar
- Nils Hammerla, Reuben Kirkham, Peter Andras, and Thomas Plötz. 2013. On preserving statistical characteristics of accelerometry data using their empirical cumulative distribution. In Proc. Int. Symp. Wearable Computing (ISWC). ACM, New York, NY, USA, 65--68. DOI:https://doi.org/10.1145/2493988.2494353Google Scholar
- Nils Y. Hammerla and Thomas Plötz. 2015. Let’s (not) stick together: Pairwise similarity biases cross-validation in activity recognition. In Proc. Ubicomp.Google Scholar
- Jesse Hoey, Thomas Plötz, Dan Jackson, Andrew Monk, Cuong Pham, and Patrick Olivier. 2011. Rapid specification and automated generation of prompting systems to assist people with dementia. Pervasive and Mobile Computing (PMC) 7, 3 (June 2011), 299--318. DOI:https://doi.org/10.1016/j.pmcj.2010.11.007Google Scholar
- Aftab Khan, Sebastian Mellor, Eugen Berlin, Robin Thompson, Roisin McNaney, Patrick Olivier, and Thomas Plötz. 2015. Beyond activity recognition: Skill assessment from accelerometer data. In Proc. ACM Int. Joint Conf. Ubiquitous and Pervasive Comp. (UbiComp’15). ACM, 1155--1166. DOI:https://doi.org/10.1145/2750858.2807534Google Scholar
- Aftab Khan, James Nicholson, and Thomas Plötz. 2017. Activity recognition for quality assessment of batting shots in cricket using a hierarchical representation. Proc. ACM Interactive, Mobile, Wearable and Ubiquitous Computing (IMWUT) 1, 3 (Sept. 2017), 62:1--62:31. DOI:https://doi.org/10.1145/3130927Google Scholar
- Aftab Khan, David Windridge, and Josef Kittler. 2014. Multilevel chinese takeaway process and label-based processes for rule induction in the context of automated sports video annotation. IEEE Transactions on Cybernetics 44, 10 (Oct. 2014), 1910--1923. DOI:https://doi.org/10.1109/tcyb.2014.2299955Google Scholar
Cross Ref
- Hyeokhyen Kwon, Gregory D. Abowd, and Thomas Plötz. 2018. Adding structural characteristics to distribution-based accelerometer representations for activity recognition using wearables. In Proc. Int. Symp. Wearable Computing (ISWC’18). 72--75.Google Scholar
- Cassim Ladha, Nils Y. Hammerla, Patrick Olivier, and Thomas Plötz. 2013. ClimbAX: Skill assessment for climbing enthusiasts. In Proc. ACM Int. Joint Conf. Ubiquitous and Pervasive Comp. (UbiComp’13). ACM, 235--244. DOI:https://doi.org/10.1145/2493432.2493492Google Scholar
- Claudine J. C. Lamoth, Rob C. van Lummel, and Peter J. Beek. 2009. Athletic skill level is reflected in body sway: A test case for accelometry in combination with stochastic dynamics. Gait 8 Posture 29, 4 (June 2009), 546--551. DOI:https://doi.org/10.1016/j.gaitpost.2008.12.006Google Scholar
- Oscar D. Lara and Miguel A. Labrador. 2013. A survey on human activity recognition using wearable sensors. IEEE Communications Surveys 8 Tutorials 15, 3 (2013), 1192--1209. DOI:https://doi.org/10.1109/surv.2012.110112.00192Google Scholar
- Jessica Lin, Eamonn Keogh, Li Wei, and Stefano Lonardi. 2007. Experiencing SAX: A novel symbolic representation of time series. Data Mining and Knowledge Discovery 15, 2 (April 2007), 107--144. DOI:https://doi.org/10.1007/s10618-007-0064-zGoogle Scholar
- Thomas B. Moeslund, Adrian Hilton, and Volker Krüger. 2006. A survey of advances in vision-based human motion capture and analysis. Computer Vision and Image Understanding 104, 2--3 (Nov. 2006), 90--126. DOI:https://doi.org/10.1016/j.cviu.2006.08.002Google Scholar
Digital Library
- Andreas Möller, Luis Roalter, Stefan Diewald, Matthias Kranz, Nils Hammerla, Patrick Olivier, and Thomas Plötz. 2012. GymSkill: A personal trainer for physical exercises. In Proc. IEEE Conf. Pervasive Comp. and Communication (PerCom’12). 213--220. DOI:https://doi.org/10.1109/PerCom.2012.6199869Google Scholar
- Paritosh Parmar and Brendan Tran Morris. 2017. Learning to score olympic events. In Proc. Conf. Computer Vision and Pattern Recognition (CVPR’17).Google Scholar
- John C. Platt. 1999. Probabilistic outputs for support vector machines and comparisons to regularized likelihood methods. In Advances In Large Margin Classifiers. MIT Press, 61--74.Google Scholar
- Thomas Plötz, Chen Chenn, Nils Y. Hammerla, and Gregory D. Abowd. 2012. Automatic synchronization of wearable sensors and video-cameras for ground truth annotation -- A practical approach. In Proc. Int. Symp. Wearable Computing (ISWC’12). 100--103. DOI:https://doi.org/10.1109/ISWC.2012.15Google Scholar
- Thomas Plötz, Nils Y. Hammerla, and Patrick Olivier. 2011. Feature learning for activity recognition in ubiquitous computing. In Proc. Int. Joint Conf. Artificial Intelligence (IJCAI’11). AAAI Press, 1729--1734. DOI:https://doi.org/10.5591/978-1-57735-516-8/IJCAI11-290Google Scholar
- Carl Edward Rasmussen and Christopher K. I. Williams. 2006. Gaussian Processes for Machine Learning. MIT Press.Google Scholar
Digital Library
- Martin Seiffert, Flavio Holstein, Rainer Schlosser, and Jochen Schiller. 2017. Next generation cooperative wearables: Generalized activity assessment computed fully distributed within a wireless body area network. IEEE Access 5 (Sept. 2017), 16793--16807. DOI:https://doi.org/10.1109/access.2017.2749005Google Scholar
- Yachna Sharma, Vinay Bettadapura, Thomas Plötz, Nils Hammerla, Sebastian Mellor, Roisin McNaney, Patrick Olivier, Sandeep Deshmukh, Andrew Mccaskie, and Irfan Essa. 2014. Video based assessment of OSATS using sequential motion textures. In Proc. 5th Workshop on Modeling and Monitoring of Computer Assisted Interventions (M2CAI’14).Google Scholar
- Yachna Sharma, Thomas Plötz, Nils Hammerla, Sebastian Mellor, Roisin McNaney, Patrick Olivier, Sandeep Deshmukh, Andrew Mccaskie, and Irfan Essa. 2014. Automated surgical OSATS prediction from videos. In Proc. IEEE Int. Symposium on Biomedical Imaging (ISBI’14). DOI:https://doi.org/10.1109/isbi.2014.6867908Google Scholar
- Chantal Simons and Elizabeth J. Bradshaw. 2016. Reliability of accelerometry to assess impact loads of jumping and landing tasks. Sports Biomechanics 15, 1 (Jan. 2016), 1--10. DOI:https://doi.org/10.1080/14763141.2015.1091032Google Scholar
- Alex J. Smola and Bernhard Schölkopf. 2004. A tutorial on support vector regression. Statistics and Computing 14, 3 (Aug 2004), 199--222. DOI:https://doi.org/10.1023/B:STCO.0000035301.49549.88Google Scholar
Digital Library
- Robin Thompson, Ilias Kyriazakis, Amey Holden, Patrick Olivier, and Thomas Plötz. 2015. Dancing with horses: Automated quality feedback for dressage riders. In Proc. ACM Int. Joint Conf. Ubiquitous and Pervasive Comp. (UbiComp’15). ACM, 325--336. DOI:https://doi.org/10.1145/2750858.2807536Google Scholar
- Eduardo Velloso, Andreas Bulling, Hans Gellersen, Wallace Ugulino, and Hugo Fuks. 2013. Qualitative activity recognition of weight lifting exercises. In Proc. Int. Conf. Augmented Human. ACM, New York, NY, 116--123. DOI:https://doi.org/10.1145/2459236.2459256Google Scholar
- Peter Wittenburg, Hennie Brugman, Albert Russel, Alex Klassmann, and Han Sloetjes. 2006. ELAN: A professional framework for multimodality research. In Proc. Int. Conf. Language Resources and Evaluation Conference (LREC’06).Google Scholar
- Bob Woolmer, Timothy Noakes, Helen Moffett, and Fay Lewis. 2008. Bob Woolmer’s art and Science of Cricket. New Holland London.Google Scholar
- Ting-Fan Wu, Chih-Jen Lin, and Ruby C. Weng. 2004. Probability estimates for multi-class classification by pairwise coupling. Journal of Machine Learning Research 5 (Dec. 2004), 975--1005. http://dl.acm.org/citation.cfm?id=1005332.1016791.Google Scholar
- Aneeq Zia, Yachna Sharma, Vinay Bettadapura, Eric L. Sarin, Thomas Plötz, Mark A. Clements, and Irfan Essa. 2016. Automated video-based assessment of surgical skills for training and evaluation in medical schools. International Journal of Computer Assisted Radiology and Surgery 11, 9 (Aug. 2016), 1623--1636. DOI:https://doi.org/10.1007/s11548-016-1468-2Google Scholar
Index Terms
Generalized and Efficient Skill Assessment from IMU Data with Applications in Gymnastics and Medical Training
Recommendations
Beyond activity recognition: skill assessment from accelerometer data
UbiComp '15: Proceedings of the 2015 ACM International Joint Conference on Pervasive and Ubiquitous ComputingThe next generation of human activity recognition applications in ubiquitous computing scenarios focuses on assessing the quality of activities, which goes beyond mere identification of activities of interest. Objective quality assessments are often ...
Providing Objective Feedback on Skill Assessment in a Dental Surgical Training Simulator
AIME '09: Proceedings of the 12th Conference on Artificial Intelligence in Medicine: Artificial Intelligence in MedicineDental students devote several years to the acquisition of sufficient psychomotor skills to prepare them for entry-level dental practice. Traditional methods of dental surgical skills training and assessment are being challenged by the complications ...
Recognizing new activities with limited training data
ISWC '15: Proceedings of the 2015 ACM International Symposium on Wearable ComputersActivity recognition (AR) systems are typically built to recognize a predefined set of common activities. However, these systems need to be able to learn new activities to adapt to a user's needs. Learning new activities is especially challenging in ...






Comments