A GIS-based serious game recommender for online physical therapy

ABSTRACT

As human-centered interactive technologies, serious games are getting popularity in a variety of fields such as training simulations, health, national defense, and education. To build the best learning experience when designing a serious game, a system requires the integration of accurate spatio-temporal information. Also, there is an increasing need for intelligent medical technologies, which enable patients to live independently at home. This paper introduces a novel e-Health framework that leverages GIS-based serious games for people with disabilities. This framework consists of a spatial map-browsing environment augmented with our newly introduced multi-sensory Natural User Interface. We propose a comprehensive architecture that includes a sensory data manager, a storage layer, an information processing and computational intelligence layer, and a user interface layer. Detailed mathematical modeling as well as mapping methodology to convert different therapy-based hand-gestures into navigational movements within the serious game environment are also presented. Moreover, an Intelligent Game Recommender has been developed for generating optimized navigational routes based on therapeutic gestures. Motion data is stored in a repository throughout the different sessions for offline replaying and advanced analysis; and different indicators are displayed in a live manner. This framework has been tested with Nokia, Google maps, ESRI map, and other maps whereby a subject can visualize and browse the 2D and 3D map of the world through therapy-based gestures. To the best of our knowledge, this is the first GIS-based game re-commender framework for online physical therapy. The prototype has been deployed to a disability center. The obtained results and feedback from therapists and patients are very encouraging.

References

1. D. Ayala, O. Wolfson, B. Xu, B. Dasgupta, and J. Lin. Parking slot assignment games. In Proceedings of the 19th ACM SIGSPATIAL International Conference on Advances in Geographic Information Systems, pages 299--308. ACM, 2011.
2. Y.-J. Chang, S.-F. Chen, and J.-D. Huang. A kinect-based system for physical rehabilitation: A pilot study for young adults with motor disabilities. Research in developmental disabilities, 32(6): 2566--2570, 2011.
3. Z. Cheng, F. Hao, Z. JianYou, and S. Yun. Research on design of serious game based on GIS. In Proceedings of the IEEE 11th International Conference on Computer-Aided Industrial Design & Conceptual Design (CAIDCD), volume 1, pages 231--233. IEEE, 2010.
4. R. Jacob, P. Mooney, P. Corcoran, and A. C. Winstanley. Haptic-GIS: Exploring the possibilities. SIGSPATIAL Special, 2(3): 13--18, 2010.
5. K. O. Jordan, I. Sheptykin, B. Grüter, and H.-R. Vatterrott. Identification of structural landmarks in a park using movement data collected in a location-based game. In Proceedings of The First ACM SIGSPATIAL International Workshop on Computational Models of Place, pages 1--8. ACM, 2013.
6. K. Khoshelham and S. O. Elberink. Accuracy and resolution of kinect depth data for indoor mapping applications. Sensors, 12(2): 1437--1454, 2012.
7. L. Omelina, B. Jansen, B. Bonnechere, S. Van Sint Jan, and J. Cornelis. Serious games for physical rehabilitation: designing highly configurable and adaptable games. In Proceedings of the 9th International Conference on Disability, Virutal Reality & Associated Technologies, pages 195--201, 2012.
8. R. M. A. Qamar, A. and S. Basalamah. Adding inverse kinematics for providing live feedback in a serious game-based rehabilitation system. In Proceedings of the 5th International Conference on Intelligent Systems, Modelling and Simulation (ISMS), 2014.
9. Q. A. H. D. Rahman, M. A. and S. Basalamah. Modeling therapy rehabilitation sessions using non-invasive serious games. In Proceedings of the IEEE International Symposium on Medical Measurements and Applications (MeMeA), 2014.
10. P. Rego, P. M. Moreira, and L. P. Reis. Serious games for rehabilitation: A survey and a classification towards a taxonomy. In Proceedings of the 2010 5th Iberian Conference on Information Systems and Technologies (CISTI), pages 1--6. IEEE, 2010.
11. E. E. Stone and M. Skubic. Passive in-home measurement of stride-to-stride gait variability comparing vision and kinect sensing. In Proceedings of Annual International Conference of the IEEE Engineering in Medicine and Biology Society, pages 6491--6494. IEEE, 2011.
12. C. Wang, K. S. Phua, K. K. Ang, C. Guan, H. Zhang, R. Lin, S. G. C., B. T. Ang, and C. W. K. Kuah. A feasibility study of non-invasive motor-imagery bci-based robotic rehabilitation for stroke patients. In Proceedings of the 4th International IEEE/EMBS Conference on Neural Engineering, pages 271--274. IEEE, 2009.
13. F. Weichert. et al. Analysis of the accuracy and robustness of the leap motion controller. Sensors, 13(5): 6380, 2013.
14. F. Weichert, D. Bachmann, B. Rudak, and D. Fisseler. Analysis of the accuracy and robustness of the leap motion controller. Sensors, 13(5): 6380--6393, 2013.
15. W. Zhao, J. Chai, and Y.-Q. Xu. Combining marker-based mo-cap and rgb-d camera for acquiring high-fidelity hand motion data. In Proceedings of the ACM SIGGRAPH/Eurographics Symposium on Computer Animation, pages 33--42. Eurographics Association, 2012.

Index Terms

• Human-centered computing

  • Visualization

    • Visualization application domains

      • Geographic visualization

• Information systems

  • Information systems applications

    • Multimedia information systems

    • Spatial-temporal systems