Abstract
In this article, we present a media adaptation framework for an immersive biofeedback system for stroke patient rehabilitation. In our biofeedback system, media adaptation refers to changes in audio/visual feedback as well as changes in physical environment. Effective media adaptation frameworks help patients recover generative plans for arm movement with potential for significantly shortened therapeutic time. The media adaptation problem has significant challenges—(a) high dimensionality of adaptation parameter space; (b) variability in the patient performance across and within sessions; (c) the actual rehabilitation plan is typically a non-first-order Markov process, making the learning task hard.
Our key insight is to understand media adaptation as a real-time feedback control problem. We use a mixture-of-experts based Dynamic Decision Network (DDN) for online media adaptation. We train DDN mixtures per patient, per session. The mixture models address two basic questions—(a) given a specific adaptation suggested by the domain experts, predict the patient performance, and (b) given the expected performance, determine the optimal adaptation decision. The questions are answered through an optimality criterion based search on DDN models trained in previous sessions. We have also developed new validation metrics and have very good results for both questions on actual stroke rehabilitation data.
- Abowd, G. D., Mynatt, E. D., and Rodden, T. 2002. The human experience {of ubiquitous computing}. IEEE Pervasive Comput. 1, 48--57. Google Scholar
Digital Library
- Brooks, R. A. 1991a. Intelligence Without Reason. In Proceedings of the International Joint Conference on Articial Intelligence. 569--595.Google Scholar
- Brooks, R. A. 1991b. Intelligence without representation. 47, 139--159. Google Scholar
Digital Library
- Brooks, R. A., Coen, M., Dang, D., Debonet, J., Kramer, J., Lozano-Perez, T., Mellor, J., Pook, P., Stauffer, C., Stein, L., Torrance, M., and Wessler, M. 1997. The intelligent room project. In Proceedings of the 2nd International Cognitive Technology Conference (CT'97). Google Scholar
Digital Library
- Bui, H. 2003. A general model for online probabilistic plan recognition. In Proceedings of the International Joint Conference on Artificial Intelligence. Google Scholar
Digital Library
- Burridge, R. R., Rizzi, A. A., and Koditschek, D. E. 1999. Sequential composition of dynamically dexterous robot behaviors. Int. J. Robot. Resear. 18, 6, 534--555.Google Scholar
Cross Ref
- Chen, Y., Huang, H., Xu, W., Wallis, R., Sundaram, H., Rikakis, T., He, J., Ingalls, T., and Olson, L. 2006. The design of a real-time, multimodal biofeedback system for stroke patient rehabilitation. SIG ACM Multimedia, Santa Barbara, CA. Google Scholar
Digital Library
- Chen, Y., Xu, W., Sundaram, H., Rikakis, T., and Liu, S.-M. 2007. Media adaptation framework in biofeedback system for stroke patient rehabilitation. SIG ACM Multimedia, Augsburg, Germany. Google Scholar
Digital Library
- Darwiche, A. 2001. Constant space reasoning in Dynamic Bayesian Networks. Intl. J. of Approximate Reasoning 26, 161--178.Google Scholar
Cross Ref
- Dempster, A., Laird, N., and Rubin, D. 1977. Maximum likelihood from incomplete data via the EM algorithm. J. Royal Statist. Soc. 39, 1, 1--38.Google Scholar
- Dourish, P. 2001. Where the Action Is: The Foundations of Embodied Interaction. MIT Press Cambridge, MA. Google Scholar
Digital Library
- Duda, R. O., Hart, P. E., and Stork, D. G. 2000. Pattern Classification. Wiley. Google Scholar
Digital Library
- Fox, M., Ghallab, M., Infantes, G., and Long, D. 2006. Robot introspection through learned hidden markov models. 170, 2, 59--113. Google Scholar
Digital Library
- Gallichio, J. and Kluding, P. 2004. Virtual reality in stroke rehabilitation: Review of the emerging research. Phys. Therapy Rev. 9, 4, 207--212.Google Scholar
Cross Ref
- Grout, D. J. and Palisca, C. V. 2001. A History of Western Music. Norton, New York.Google Scholar
- Grupen, R. A. and Coelho, J. J. A. 2002. Acquiring state from control dynamics to learn grasping policies for robot hands. Advanced Robotics 16, 5, 427--443.Google Scholar
Cross Ref
- He, X., Ma, W.-Y., King, O., Li, M., and Zhang, H. 2003. Learning and inferring a semantic space from user's relevance feedback for image retrieval. IEEE Trans. Circ. Syst. Video Tech.Google Scholar
- Holden, M. and Dyar, T. 2002. Virtual environment training: A new tool for neurorehabilitation. Neurology Rep. 26, 62--72.Google Scholar
Cross Ref
- Holden, M., Todorov, E., Callahan, J., and Bizzi, E. 1999. Virtual environment training imporves motor performance in two patients with stroke: Case report. Neurology Rep. 23, 57--67.Google Scholar
Cross Ref
- Hollan, J., Hutchins, E., Kirsh, D., and Sutcliffe, A. 2000. Distributed cognition: Toward a new foundation for human-computer interaction research on the effective use and reuse of HCI knowledge. ACM Trans. Comput.-Hum. Interact. ACM Press. 7, 174--196. Google Scholar
Digital Library
- Hsiao, K., Kaelbling, L. P., and Lozano-Perez, T. 2007. Grasping POMDPs. In Proceedings of the IEEE Conference on Robotics and Automation.Google Scholar
- Huber, P. J. 1981. Robust Statistics. Wiley.Google Scholar
- Hutchins, E. 1995. Cognition in the Wild. MIT Press, Cambridge, MA.Google Scholar
- Ishii, H. and Ullmer, B. 1997. Tangible bits: Towards seamless interfaces between people, bits and atoms. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems. ACM Press. 234--241. Google Scholar
Digital Library
- Ishii, H., Wisneski, C., Brave, S., Dahley, A., Gorbet, M., Ullmer, B., and Yarin, P. 1998. ambientROOM: Integrating ambient media with architectural space. CHI 98 Conference Summary on Human Factors in Computing Systems. ACM Press. 173--174. Google Scholar
Digital Library
- Jack, D., Boian, R., Merians, A. S., Tremaine, M., Burdea, G. C., Adamovich, S. V., Recce, M., and Poizner, H. 2001. Virtual reality-enhanced stroke rehabilitation. IEEE Trans. Neural Syst. Rehabi. Eng. 9, 308--318.Google Scholar
Cross Ref
- Kaelbling, L. P., Littman, M. L., and Cassandra, A. R. 1998. Planning and acting in partiallly observable stochastic domains. 101, 99--134. Google Scholar
Digital Library
- Kirsh, D. 1995. The intelligent use of space. 73, 31--68. Google Scholar
Digital Library
- Kjaerulff, U. 1995. dHugin: A computational system for dynamic time-sliced Bayesian networks. Intl. J. Forecasting 11, 89--111.Google Scholar
Cross Ref
- Latombe, J. C. 1991. Robot Motion Planning. Kluwer Academic Publishers, Norwell, MA. Google Scholar
Digital Library
- Lavalle, S. M. 2006. Planning Algorithms. Cambridge University Press. Google Scholar
Digital Library
- Liao, L., Fox, D., and Kautz, H. 2004. Learning and inferring transportation routines. In Proceedings of the National Conference on Artificial Intelligence. Google Scholar
Digital Library
- Ma, Y.-F. and Zhang, H.-J. 2003. Contrast-based image attention analysis by using fuzzy growing. In Proceedings of the 11th ACM International Conference on Multimedia. ACM Press. 374--381. Google Scholar
Digital Library
- Mazalek, A., Davenport, G., and Ishii, H. 2002. Tangible viewpoints: A physical approach to multimedia stories. In Proceedings of the 10th ACM International Conference on Multimedia. ACM Press. 153--160. Google Scholar
Digital Library
- Merians, A. S., Jack, D., Boian, R., Tremaine, M., Burdea, G. C., Adamovich, S. V., Recce, M., and Poizner, H. 2002. Virtual reality-augmented rehabilittion for patients following stroke. Phys. Ther. 82, 9, 898--915.Google Scholar
Cross Ref
- Murphy, K. 2002. Dynamic Bayesian networks: Representation, inference and learning. Ph.D. thesis. Dept. Computer Science, UC Berkeley. Google Scholar
Digital Library
- Murphy, K., Torralba, A., and Freeman, W. 2003. Using the forest to see the trees: A graphical model relating features, objects and scenes. In Proceedings of the NIPS'03 (Neural Info. Processing Systems).Google Scholar
- Pineau, J., Gordon, G., and Thrun, S. 2003. Point-based value iteration: An anytime algorithm for pomdps. International Joint Conference on Artificial Intelligence. Google Scholar
Digital Library
- Rabiner, L. R. 1989. A tutorial on hidden markov models and selected applications in speech recognition. In Proceedings of the IEEE 77, 2, 257--286.Google Scholar
Cross Ref
- Russell, S. and Norvig, P. 2003. Artificial Intelligence, A Modern Approach. Pearson Education, Inc. Google Scholar
Digital Library
- Simmons, R. and Koenig, S. 1995. Probabilistic robot navigation in partially observable environments. International Joint Conference on Artificial Intelligence. Google Scholar
Digital Library
- Smith, T. and Simmons, R. 2004. Heuristic search value iteration for pomdps. In Proceedings of the 20th Conference on Uncertainty in Artificial Intelligence. Google Scholar
Digital Library
- Spaan, M. T. J. and Vlassis, N. 2005. Perseus: Randomized point-based value iteration for pomdps. J. Artifi. Intell. Resear. 24, 195--220. Google Scholar
Digital Library
- Sun, Y., Zhang, H., Zhang, L., and Li, M. 2002. A system for home photo management and processing. In Proceedings of the 10th ACM International Conference on Multimedia. 81--82. Google Scholar
Digital Library
- Sundaram, H. and Chang, S.-F. 2000. Determining computable scenes in films and their structures using audio-visual memory models. In Proceedings of the ACM International Conference on Multimedia 2000. 95--104. Google Scholar
Digital Library
- Sundaram, H. and Rikakis, T. 2006. Experiential Media Systems. Encyclopedia of Multimedia. B. Furtht. Springer Verlag, NY.Google Scholar
- Theocharous, G. and Kaelbling, L. P. 2004. Approximate planning in POMDPs with macro-actions. Advances in Neural Information Prodessing Systems 16, Vancouver.Google Scholar
- Theocharous, G., Murphy, K., and Kaelbling, L. P. 2004. Representing hierarchical POMDPs as DBNs for multi-scale robot localization. In Proceedings of the International Conference on Robotics and Automation.Google Scholar
- Ullmer, B. and Ishii, H. 2000. Emerging frameworks for tangible user interfaces. IBM Syst. J. 39, 915--931. Google Scholar
Digital Library
- Weiser, M. 1993. Some computer science issues in ubiquitous computing. Comm. ACM 36, 75--84. Google Scholar
Digital Library
- White, D., Burdick, K., Fulk, G., Searleman, J. and Carroll, J. 2005. A virtual reality application for stroke patient rehabilitation. In Proceedings of the IEEE International Conference on Mechatronics & Automation Niagara Falls, Canada.Google Scholar
- Xie, L., Chang, S.-F., Divakaran, A., and Sun, H. 2003. Unsupervised discovery of multilevel statistical video structures using hierarchical hidden Markov models. In Proceedings of the IEEE Conference on Multimedia and Expo 2003. Baltimore MD. Google Scholar
Digital Library
- Xu, W., Chen, Y., Sundaram, H., and Rikakis, T. 2006. Multimodal archiving, real-time collaborative annotation and information visualization in a biofeedback system for stroke patient rehabilitation. In Proceedings of the 3rd Workshop on Capture Archival, Retrieval of Personal Experiences, in Conjunction with ACM MM 06, Santa Barbara, CA. Google Scholar
Digital Library
Index Terms
A dynamic decision network framework for online media adaptation in stroke rehabilitation
Recommendations
Media adaptation framework in biofeedback system for stroke patient rehabilitation
MM '07: Proceedings of the 15th ACM international conference on MultimediaIn this paper, we present a media adaptation framework for an immersive biofeedback system for stroke patient rehabilitation. In our biofeedback system, media adaptation refers to changes in audio/visual feedback as well as changes in physical ...
Evaluating rehabilitation exercise performance using a single inertial measurement unit
PervasiveHealth '13: Proceedings of the 7th International Conference on Pervasive Computing Technologies for HealthcareInertial measurement units (IMUs) may be used during exercise to provide feedback on exercise technique. However the number of IMUs that are required to deliver effective feedback during lower limb exercises has not yet been established. This ...
Development of a supine gait training device for stroke rehabilitation
i-CREATe '13: Proceedings of the 7th International Convention on Rehabilitation Engineering and Assistive TechnologyStroke is the leading cause of chronic physical disability in the developed world and can severely degrade walking function, leading to enormous negative ramifications on patients' participation in social, occupational, and recreational activities. ...






Comments