Markerless Mocap for Active Shoulder Range of Motion Measurement in a Clinic Room: a Pilot Trial

This paper presents a pilot study that uses a markerless motion capture system to measure active shoulder joint range of motion in a clinic room. The measurements from this automated method are compared with manual measurements done by a clinician using a goniometer. All the parameters from both methods are positively correlated. The root-mean-square differences from the comparisons are in the range of 7.21° – 15.87° depending on the type of angles. With a careful protocol design and execution, the markerless motion capture system can potentially be used as an objective shoulder range of motion assessment tool in clinical practice.


INTRODUCTION
Our human shoulder is a complex joint responsible for a wide range of motions (ROM) needed for our activities of daily living (ADL) tasks.These huge degrees of mobility in our shoulder joint are achieved by the unit of the shoulder girdle, which consists of these primary bone structures: clavicle, scapular, humerus, joints: acromioclavicular (AC), sternoclavicular (SC), scapulothoracic (ST), and glenohumeral (GH) joint and various ligaments and muscles around the bony architecture [1].The common injuries on the shoulder, such as rotator cuff impingement, frozen shoulder, etc., result from overuse, extremes of motions and forces or various musculoskeletal conditions developed due to ageing [2,3].These shoulder disorders would result in limited movement for an individual, affecting their ADL needs.The ROM assessment of which patients can move their shoulder in various directions is crucial as it can provide valuable information about the patient's functional limit and the condition of their shoulder injuries.
The general shoulder ROM assessment will consist of flexionextension, abduction-adduction, and external-internal rotation [3].These shoulder ROM assessments are measured through either estimation ('eyeballing'), goniometer, or inclinometer.A survey conducted by Kraal et al. in Netherlands and Belgium identified that 90% of the clinician practice records shoulder ROM using the estimation method, and only 6% used a goniometer with a physical examination.This survey provided a clear gap to provide a more quantifiable assessment to improve the effectiveness of the treatment of shoulder conditions [4].
However, the current assessment of the shoulder ROM using the traditional goniometer method needs a better agreement between intra-and inter-tester reliability A study conducted by Mullaney et al. explored the intra-and inter-tester in two experienced clinicians to conduct a blinded measurement on 20 patients (9 males and 11 females) between the age of 18-79 years old with unilateral shoulder pathology using a goniometer and digital level.For the intratester, it has a difference of 3°to 9°of measurements made from the two systems in the 95% limit of agreement (LOA).For the intertester, the LOA ranges a lot higher, with a difference of 6o to 25o between the two systems [5].Another measurement reliability study using an inclinometer was conducted by Tozzo The intratester analysis has a difference of 4.1°to 10°and the intertester showed a lot higher with a difference of 9.5°to 23.4°between the two raters [6].These high differences in intra-and inter-tester have demonstrated repeatability issues to provide accurate shoulder ROM for the clinicians.These may affect the treatment in assessing the patient's shoulder condition and can affect the evaluation of the strategies that may alter shoulder function [7].
This study explored the new measurement approach using a markerless motion capture (mocap) system developed by the Rehabilitation Research Institute of Singapore (RRIS).The markerless mocap was deployed for a one-day pilot trial study at Outram Community Hospital (OCH) clinical room.The shoulder ROM data generated from the markerless mocap system was compared to measurements from a manual goniometer performed by a resident in OCH.

METHOD
In this experiment, the main objective was to see the level of agreement between the traditional clinical method using a goniometer and the new method using a markerless motion capture system.Both readings from the two separate measurement systems are compared and analyzed.

Experiment Protocol
The experiment protocol consists of 5 upper limb active ranges of motion (ROM) tasks to monitor and determine the maximum ROM of each participant.The list of ROM measurements was 1) maximum flexion and extension, 2) maximum abduction, 3) external lateral rotation, 4) internal rotation and 5) internal & external rotation with the elbow at 90°abduction.Only task numbers 1, 2, 3, and 5 are measured for the manual goniometer measurement.Refer to Figure 2 for the upper limb ROM tasks.
A total of 23 healthy subjects took part in this study, of which 9 males (height: 174.91±7.31cm,weight: 77.82±17.89Kg),14 females (height: 158.69±6.37m,weight: 64.46±18.82Kg).The order of measurement was randomly selected, where the subjects could either perform their maximum shoulder ROM measurement in the clinical room with the markerless mocap system first or the manual measurement by a resident of OCH first.Both measurements were first taken on the right side.For the measurement in the markerless system, the subjects were instructed to move their shoulders at a comfortable pace and the maximum range without feeling any discomfort.For the manual measurement, the subject was tasked to hold at the end position while the shoulder ROM was measured using the goniometer.

Markerless Mocap System Configuration
A multi-camera markerless motion capture system developed by RRIS is used in this experiment.The system consists of 5 cameras.Each camera has a resolution of 1920x1200 pixels and is configured to run at 25 frames per second.All cameras are synchronized at the individual frame level, and they are all calibrated intrinsically and extrinsically.The system uses machine learning to predict the location of virtual markers from each perspective and triangulate them to provide 3D anatomical marker positions as if the markers are placed on the subject.The placement of all the cameras is illustrated in Figures 1 and 3.The room size is 3.57 by 4.26 m with some existing furniture.Cameras are placed in a way that does not disturb the room's functionality.For example, no camera is placed beside the bed, in the corner with the door, or in the cluttered area around the lower right corner of Figure 3(b).These constraints cause a large gap between camera 1 and camera 5 which creates a blind spot in some tasks.Therefore, the subjects were always instructed to face a different direction in each action so that at least 2 cameras always see the moving forearm to allow uninterrupted 3D reconstruction of the limb.The visual instructions for all the tasks are placed on all sides for the subjects to see.
Each record's start and stop were operated by a remote control from outside the room to ensure that the operator did not block the view of any camera and to avoid the system's confusion between the target subject and the operator.
The post-processing to extract joint angles was done in all the recorded frames by the system from the 3D virtual marker positions.Then, the maximum values of the following joint angles are extracted to compare with values from manual measurements.Those angles include: A parameter from Task 4 (hand to back) was extracted and not compared against manual measurement because the goniometer cannot do it properly.

Results & Discussions
The measurement of all 23 subjects were executed within one day.The recording of 5 left tasks and 5 right tasks for one subject can take as fast as 3 minutes.The markerless motion capture system produces the range of motion values as expected without frame skipping.However, the values are not perfectly aligned with those collected from goniometer measurements.All the discrepancies are summarised in Table 1.The root-mean-square deviations range from 7.21°to 15.87°.The detailed comparisons of the value pairs from each individual measurement are illustrated in Figure 4.All the angles have positive correlation coefficients with the strongest correlation of 0.90 for maximum shoulder flexion from Task 1. and the weakest correlation of 0.35 for maximum shoulder internal rotation from Task 5.
For the maximum flexion angle from Task 1, one sample pairs with high differences (over 25°) are found in the blue oval in Figure 4(a).In this case, the mocap system provides an overshooting value.It appears in the videos that the subject's upper arm has made some large angle with the sagittal plane at the high flexion angle instead of staying parallel to the sagittal plane.This oversight allows subjects to gain more flexion angles than they should.This is considered a protocol-related issue during the motion capture session.The instruction should clearly state that the arm should always be parallel to the sagittal plane during the entire Task 1, and the operator should reconduct the task if the subject violates this rule.
A large error could also happen when the subject did not follow the instruction correctly.For example, the outlier point in the blue oval in Figure 4(b) occurs because the subject did not perform the extension part of Task 1.
For the internal rotation from Task 5, all 4 sample pairs with high differences (greater than 25°) tend to have the same pattern, which are large overshoots in goniometer measurement (samples in the blue oval in Figure 4(f)).This is likely due to the different frames of reference used during the measurements.The mocap system always measures relative to the torso, but the goniometer tends Figure 4: Scatter plots of values from manual measurements against the automated measurement from markerless motion capture system and their correlation coefficients (r).The value pairs from the right and the left shoulder are combined in each plot.The goniometer readings have roughly equal numbers of overshoots and undershoots from mocap readings except for the flexion (Task 1).For the extension plot, more negative means more extension.For the internal rotation plot, more negative means more internal rotation.The blue ovals circle around all the differences greater than 25°.
to measure relative to gravity and require the subject to set his posture upright.Suppose the user leans their upper torso forward to make the forearm point more downward perpendicular to the floor to gain more internal shoulder rotation.In that case, this act will not change the reading from the mocap system but will wrongly increase the reading of maximum internal rotation from the goniometer.
For the rest of the large discrepancies, two possible causes could be 1) the subject did not move to the same posture for both the goniometer and the mocap measurements and 2) human errors caused by the manual readings from the goniometer in terms of goniometer alignment inconsistency and how rigid can the subject hold the posture at the extreme joint angle during the measurement.These two issues are hard to confirm as there is no video record of the activity during the goniometer measurement to provide evidence.
In addition, we have found that it is not practical, and there is no standardised way to measure external/internal shoulder rotation angle in Task 4 (hand to back) using a goniometer.However, the markerless motion capture system can produce the value.
Another advantage of the markerless mocap over a goniometer is the ability to provide dynamic information of all the joint angles at any point in the movement, not just at a static posture.This feature allows more parameters to be analysed for better assessment.

Future Works & Conclusion
In the investigation effort to find the root causes of measurement discrepancy, the subsequent studies need a sharper protocol to constrain the subject movements and postures to minimise the measurement variance from the protocol issue.In addition, a video record during the manual measurement sessions should be added to help investigate the actual causes of errors and see whether the subject has complied with the protocol.
The current protocol only measures one sample for each combination of movement and the measurement method.With this limitation, the repeatability of each method cannot be analysed.The subsequent study should increase the number of samples to allow more in-depth comparison and analysis.
Lastly, this experiment is not designed to determine which measurement method is more accurate as there is no ground truth measurement such as a marker-based motion capture system running in parallel.Benchmarking both methods with a more accurate system will give better clues for the source of discrepancy in the measurements.
As the main purposes of this pilot study are to collect the lessons learned, to test the scalability of the markerless system in a clinical environment, and to pave the way towards a larger study with pathological subjects, this is considered a successful pilot study.

Figure 1 :
Figure 1: The five perspectives from the five cameras used by the markerless motion capture system.The photos also show the recording environment in the clinic room.The visual demonstrations of each movement are placed on the wall.

Figure 2 :
Figure 2: The upper limb ROM tasks: (a) Flexion-extension, (b) Abduction, (c) Internal-external, (d) Internal shoulder rotation: Hand to back, (e) Internal-external with the elbow at 90°abduction.(i) Shoulder start position of the subjects, (ii) Shoulder end position of the subjects.

Figure 3 :
Figure 3: (a) The top view plan of the room and the camera positions (the crosses).All the numbers are in cm.All five cameras are placed 170 cm from the floor.(b) The panoramic view of the actual setup of the clinic room.

Table 1 :
Root-mean-square (RMS) difference between manual (goniometer) and auto measurement (markerless mocap) for the angles that can be done by manual measurement.