Toward Usable Evidence: Optimizing Knowledge Accumulation in HCI Research on Health Behavior Change

ABSTRACT

Over the last ten years, HCI researchers have introduced a range of novel ways to support health behavior change, from glanceable displays to sophisticated game dynamics. Yet, this research has not had as much impact as its originality warrants. A key reason for this is that common forms of evaluation used in HCI make it difficult to effectively accumulate-and use-knowledge across research projects. This paper proposes a strategy for HCI research on behavior change that retains the field's focus on novel technical contributions while enabling accumulation of evidence that can increase impact of individual research projects both in HCI and the broader behavior-change science. The core of this strategy is an emphasis on the discovery of causal effects of individual components of behavior-change technologies and the precise ways in which those effects vary with individual differences, design choices, and contexts in which those technologies are used.

References

1. Abraham, Charles and Michie, Susan.. 2008. A taxonomy of behavior change techniques used in interventions. Health Psychol 27, 3: 379--87. http://doi.org/10.1037/02786133.27.3.379
2. Collins, Francis S and Varmus, Harold.. 2015. A new initiative on precision medicine. New England Journal of Medicine 372, 9: 793--795.
3. Collins, Linda M, Chakraborty, Bibhas, Murphy, Susan A and Strecher, Victor.. 2009. Comparison of a phased experimental approach and a single randomized clinical trial for developing multicomponent behavioral interventions. Clinical Trials 6, 1: 5--15.
4. Collins, Linda M, Dziak, John J and Li, Runze.. 2009. Design of experiments with multiple independent variables: A resource management perspective on complete and reduced factorial designs. Psychological Methods 14, 3: 202224. http://doi.org/10.1037/a0015826
5. Collins, Linda M, Murphy, Susan A, Nair, Vijay N and Strecher, Victor J.. 2005. A strategy for optimizing and evaluating behavioral interventions. Annals of Behavioral Medicine 30, 1: 65--73.
6. Collins, Linda M, Murphy, Susan A and Strecher, Victor.. 2007. The multiphase optimization strategy (MOST) and the sequential multiple assignment randomized trial (SMART): new methods for more potent eHealth interventions. Am J Prev Med 32, 5 Suppl: S112--8. http://doi.org/10.1016/j.amepre.2007.01.022
7. Collins, Linda M., Dziak, John J., Kugler, Kari C. and Trail, Jessica B.. 2014. Factorial Experiments: Efficient Tools for Evaluation of Intervention Components. American Journal of Preventive Medicine 47, 4: 498--504. http://doi.org/10.1016/j.amepre.2014.06.021
8. Consolvo, Sunny, Bentley, Frank and Hekler, Eric B.. 2017. Mobile User Research: A Practical Guide. Morgan & Claypool Publisher, Williston, VT .
9. Consolvo, Sunny, Klasnja, Predrag, McDonald, David W, Avrahami, Daniel, Froehlich, Jon, LeGrand, Louis, Libby, Ryan, Mosher, Keith and Landay, James A.. 2008. Flowers or a robot army?: encouraging awareness & activity with personal, mobile displays. In Proceedings of the 10th international conference on Ubiquitous computing, 54--63.
10. Consolvo, Sunny, McDonald, David W, Toscos, Tammy, Chen, Mike Y, Froehlich, Jon, Harrison, Beverly, Klasnja, Predrag, LaMarca, Anthony, LeGrand, Louis, Libby, Ryan, Smith, Ian and Landay, James A.. 2008. Activity sensing in the wild: a field trial of UbiFit garden. In Proceeding of the twenty-sixth annual SIGCHI conference on Human factors in computing systems, 1797--1806.
11. Dallery, Jesse, Cassidy, Rachel N and Raiff, Bethany R.. 2013. Single-Case Experimental Designs to Evaluate Novel Technology-Based Health Interventions. Journal of Medical Internet Research 15, 2: e22. http://doi.org/10.2196/jmir.2227
12. Dallery, Jesse and Raiff, Bethany R.. 2014. Optimizing behavioral health interventions with single-case designs: from development to dissemination. Transl Behav Med 4, 3: 290--303. http://doi.org/10.1007/s13142-014-0258-z
13. Dow, Steven.. 2016. Probe to Learn, Probe to Design. interactions 23, 4: 22--23. http://doi.org/10.1145/2931079
14. Dow, Steven P, Glassco, Alana, Kass, Jonathan, Schwarz, Melissa, Schwartz, Daniel L and Klemmer, Scott R.. 2010. Parallel Prototyping Leads to Better Design Results, More Divergence, and Increased Self-efficacy. ACM Trans. Comput.-Hum. Interact 17, 4: 18:1--18:24. http://doi.org/10.1145/1879831.1879836
15. Froehlich, Jon, Dillahunt, Tawanna, Klasnja, Predrag, Mankoff, Jennifer, Consolvo, Sunny, Harrison, Beverly and Landay, James A.. 2009. UbiGreen: investigating a mobile tool for tracking and supporting green transportation habits. In Proceedings of the 27th international conference on Human factors in computing systems, 1043--1052.
16. Havassy, B E, Hall, S M and Wasserman, D A.. 1991. Social support and relapse: commonalities among alcoholics, opiate users, and cigarette smokers. Addict Behav 16, 5: 235--46.
17. Hekler, E B, Buman, M P, Poothakandiyil, N, Rivera, D E, Dzierzewski, J M, Aiken Morgan, A, McCrae, C S, Roberts, B L, Marsiske, M and Giacobbi, P R.. 2013. Exploring Behavioral Markers of Long-Term Physical Activity Maintenance: A Case Study of System Identification Modeling Within a Behavioral Intervention. Health Education & Behavior 40, 1 Suppl: 51S-62S. http://doi.org/10.1177/1090198113496787
18. Hekler, Eric B, Klasnja, Predrag, Froehlich, Jon E and Buman, Matthew P.. 2013. Mind the theoretical gap: interpreting, using, and developing behavioral theory in HCI research. In Proceedings of the 2013 ACM annual conference on Human factors in computing systems (CHI '13), 3307--3316. http://doi.org/10.1145/2466416.2466452
19. Hekler, Eric B, Klasnja, Predrag, Riley, William T, Buman, Matthew P, Huberty, Jennifer, Rivera, Daniel E and Martin, Cesar A.. 2016. Agile science: creating useful products for behavior change in the real world. Transl Behav Med 6, 2: 317--28. http://doi.org/10.1007/s13142-0160395--7
20. Hekler, Eric B, Michie, Susan, Pavel, Misha, Rivera, Daniel E, Collins, Linda M, Jimison, Holly B, Garnett, C, Parral, S and Spruijt-Metz, Donna.. Advancing models and theories for digital behavior change. American Journal of Preventive Medicine.
21. Hovsepian, Karen, al'Absi, Mustafa, Ertin, Emre, Kamarck, Thomas, Nakajima, Motohiro and Kumar, Santosh.. 2015. cStress: towards a gold standard for continuous stress assessment in the mobile environment. In Proceedings of the 2015 ACM International Joint Conference on Pervasive and Ubiquitous Computing, 493--504.
22. Kay, Matthew, Nelson, Gregory and Hekler, Eric.. 2016. Researcher-centered design of statistics: Why Bayesian statistics better fit the culture and incentives of HCI. In Proceedings of SIGCHI Conference on Human Factors in Computing Systems (CHI '16).
23. Kessler, Rodger and Glasgow, Russell E.. 2011. A proposal to speed translation of healthcare research into practice: dramatic change is needed. American journal of preventive medicine 40, 6: 637--644.
24. King, Abby C, Hekler, Eric B, Grieco, Lauren A, Winter, Sandra J, Sheats, Jylana L, Buman, Matthew P, Banerjee, Banny, Robinson, Thomas N and Cirimele, Jesse.. 2013. Harnessing Different Motivational Frames via Mobile Phones to Promote Daily Physical Activity and Reduce Sedentary Behavior in Aging Adults. PLoS ONE 8, 4: e62613. http://doi.org/10.1371/journal.pone.0062613
25. Klasnja, Predrag, Consolvo, Sunny and Pratt, Wanda.. 2011. How to evaluate technologies for health behavior change in HCI research. In CHI '11: Proceedings of the 2011 annual conference on Human factors in computing systems, 3063--3072. http://doi.org/10.1145/1978942.1979396
26. Klasnja, Predrag, Hekler, Eric B, Shiffman, Saul, Boruvka, Audrey, Almirall, Daniel, Tewari, Ambuj and Murphy, Susan A.. 2015. Microrandomized trials: An experimental design for developing just-in-time adaptive interventions. Health Psychol 34 Suppl: 1220--8. http://doi.org/10.1037/hea0000305
27. Kravitz, R L and Duan, N.. 2014. Design and Implementation of N-of-1 Trials: a users guide. Agency for Healthcare Research and Quality, Rockville, MD. from www.effectivehealthcare.ahrq.gov/N-1-Trials.cfm
28. Kumar, Santosh, Nilsen, Wendy J, Abernethy, Amy, Atienza, Audie, Patrick, Kevin, Pavel, Misha, Riley, William T, Shar, Albert, Spring, Bonnie, Spruijt-Metz, Donna, Hedeker, Donald, Honavar, Vasant, Kravitz, Richard, Craig Lefebvre, R, Mohr, David C, Murphy, Susan A, Quinn, Charlene, Shusterman, Vladimir and Swendeman, Dallas.. 2013. Mobile Health Technology Evaluation: The mHealth Evidence Workshop. Am J Prev Med 45, 2: 228--36. http://doi.org/10.1016/j.amepre.2013.03.017
29. Lane, Nicholas D, Mohammod, Mashfiqui, Lin, Mu, Yang, Xiaochao, Lu, Hong, Ali, Shahid, Doryab, Afsaneh, Berke, Ethan, Choudhury, Tanzeem and Campbell, Andrew.. 2011. BeWell: A smartphone application to monitor, model and promote wellbeing. In 5th International Conference on Pervasive Computing Technologies for Healthcare (PervasiveHealth2011).
30. Lei, H, Nahum-Shani, I, Lynch, K, Oslin, D and Murphy, S A.. 2012. A SMART design for building individualized treatment sequences. Annual Review of Clinical Psychology 8: 21--48.
31. Liao, Peng, Klasnja, Predrag, Tewari, Ambuj and Murphy, Susan A.. 2016. Sample size calculations for microrandomized trials in mHealth. Stat Med 35, 12: 1944--71. http://doi.org/10.1002/sim.6847
32. Lin, Mu, Lane, Nicholas D, Mohammod, Mashfiqui, Yang, Xiaochao, Lu, Hong, Cardone, Giuseppe, Ali, Shahid, Doryab, Afsaneh, Berke, Ethan and Campbell, Andrew T.. 2012. BeWell+: multi-dimensional wellbeing monitoring with community-guided user feedback and energy optimization. In Proceedings of the conference on Wireless Health, 10.
33. Michie, Susan, Ashford, Stefanie, Sniehotta, Falko F, Dombrowski, Stephan U, Bishop, Alex and French, David P.. 2011. A refined taxonomy of behaviour change techniques to help people change their physical activity and healthy eating behaviours: the CALO-RE taxonomy. Psychol Health 26, 11: 1479--98. http://doi.org/10.1080/08870446.2010.540664
34. Nahum-Shani, Inbal, Hekler, Eric B. and Spruijt-Metz, Donna.. 2015. Building health behavior models to guide the development of just-in-time adaptive interventions: A pragmatic framework. Health Psychology 34, Suppl: 12091219.
35. Nahum-Shani, Inbal, Qian, Min, Almirall, Daniel, Pelham, William E, Gnagy, Beth, Fabiano, Gregory A, Waxmonsky, James G, Yu, Jihnhee and Murphy, Susan A.. 2012. Experimental design and primary data analysis methods for comparing adaptive interventions. Psychological methods 17, 4: 457.
36. Pearl, Judea.. 2009. Causality. Cambridge university press.
37. Riley, William T, Rivera, Daniel E, Atienza, Audie A, Nilsen, Wendy, Allison, Susannah M and Mermelstein, Robin.. 2011. Health behavior models in the age of mobile interventions: are our theories up to the task? Transl Behav Med 1, 1: 53--71. http://doi.org/10.1007/s13142-011-0021--7
38. Riley, William T and Rivera, Daniel E.. 2014. Methodologies for optimizing behavioral interventions: introduction to special section. Translational behavioral medicine 4, 3: 234.
39. Spruijt-Metz, Donna and Nilsen, Wendy.. 2014. Dynamic Models of Behavior for Just-in-Time Adaptive Interventions. IEEE Pervasive Computing, 3: 13--17.

Index Terms

• Applied computing

  • Life and medical sciences

• Human-centered computing

  • Human computer interaction (HCI)