skip to main content
research-article

Using TEMPEST: End-User Programming of Web-Based Ecological Momentary Assessment Protocols

Published:19 June 2018Publication History
Skip Abstract Section

Abstract

Researchers who perform Ecological Momentary Assessment (EMA) studies tend to rely on informatics experts to set up and administer their data collection protocols with digital media. Contrary to standard surveys and questionnaires that are supported by widely available tools, setting up an EMA protocol is a substantial programming task. Apart from constructing the survey items themselves, researchers also need to design, implement, and test the timing and the contingencies by which these items are presented to respondents. Furthermore, given the wide availability of smartphones, it is becoming increasingly important to execute EMA studies on user-owned devices, which presents a number of software engineering challenges pertaining to connectivity, platform independence, persistent storage, and back-end control. We discuss TEMPEST, a web-based platform that is designed to support non-programmers in specifying and executing EMA studies. We discuss the conceptual model it presents to end-users, through an example of use, and its evaluation by 18 researchers who have put it to real-life use in 13 distinct research studies.

References

  1. N. Aharony, A. Gardner, C. Sumter, and A. Pentland. Funf: Open sensing framework, 2011.Google ScholarGoogle Scholar
  2. D. J. Barrett and D. Barrett. Esp, the experience sampling program. Retrieved March, 1:2007, 2005.Google ScholarGoogle Scholar
  3. N. Batalas and P. Markopoulos. Considerations for computerized in situ data collection platforms. In Proceedings of the 4th ACM SIGCHI symposium on Engineering interactive computing systems, pages 231--236. ACM, 2012. Google ScholarGoogle ScholarDigital LibraryDigital Library
  4. N. Batalas and P. Markopoulos. Introducing tempest, a modular platform for in situ data collection. In Proceedings of the 7th Nordic Conference on Human-Computer Interaction: Making Sense Through Design, pages 781--782. ACM, 2012. Google ScholarGoogle ScholarDigital LibraryDigital Library
  5. N. Batalas, J. Quevedo-Fernandez, J.-B. Martens, and P. Markopoulos. Meet your users in situ data collection from within apps in large-scale deployments. International Journal of Handheld Computing Research (IJHCR), 6(3):17--32, 2015. Google ScholarGoogle ScholarDigital LibraryDigital Library
  6. F. M. Bos, R. A. Schoevers, and M. aan het Rot. Experience sampling and ecological momentary assessment studies in psychopharmacology: A systematic review. European Neuropsychopharmacology, 25(11):1853--1864, 2015.Google ScholarGoogle ScholarCross RefCross Ref
  7. R. Bosman, C. Albers, N. Batalas, and M. aan het Rot. No cyclicity in mood and social functioning in women with self-reported pms. Manuscript submitted for publication.Google ScholarGoogle Scholar
  8. J. Brooke et al. Sus-a quick and dirty usability scale. Usability evaluation in industry, 189(194):4--7, 1996.Google ScholarGoogle Scholar
  9. M. Capatu, G. Regal, J. Schrammel, E. Mattheiss, M. Kramer, N. Batalas, and M. Tscheligi. Capturing mobile experiences: Context-and time-triggered in-situ questionnaires on a smartphone. Measuring Behavior 2014, 2014.Google ScholarGoogle Scholar
  10. S. Carter, J. Mankoff, S. R. Klemmer, and T. Matthews. Exiting the cleanroom: On ecological validity and ubiquitous computing. Human-Computer Interaction, 23(1):47--99, 2008.Google ScholarGoogle ScholarCross RefCross Ref
  11. T. C. Christensen, L. F. Barrett, E. Bliss-Moreau, K. Lebo, and C. Kaschub. A practical guide to experience-sampling procedures. Journal of Happiness Studies, 4(1):53--78, 2003.Google ScholarGoogle ScholarCross RefCross Ref
  12. T. S. Conner and M. R. Mehl. Ambulatory assessment: Methods for studying everyday life. Emerging Trends in the Social and Behavioral Sciences: An Interdisciplinary, Searchable, and Linkable Resource, 2015.Google ScholarGoogle Scholar
  13. S. Consolvo, B. Harrison, I. Smith, M. Y. Chen, K. Everitt, J. Froehlich, and J. A. Landay. Conducting in situ evaluations for and with ubiquitous computing technologies. International Journal of Human-Computer Interaction, 22(1--2):103--118, 2007.Google ScholarGoogle ScholarCross RefCross Ref
  14. P. Copley. Research and evaluation of marketing communications. In Marketing communications management: concepts and theories, cases and practices, chapter 18. Routledge, 2004.Google ScholarGoogle Scholar
  15. M. Csikszentmihalyi, R. Larson, and S. Prescott. The ecology of adolescent activity and experience. Journal of youth and adolescence, 6(3):281--294, 1977.Google ScholarGoogle Scholar
  16. D. de Feijter, V.-J. Khan, and M. van Gisbergen. Confessions of a'guilty'couch potato understanding and using context to optimize binge-watching behavior. In Proceedings of the ACM International Conference on Interactive Experiences for TV and Online Video, pages 59--67. ACM, 2016. Google ScholarGoogle ScholarDigital LibraryDigital Library
  17. D. Estrin, M. Carroll, and N. Lakin. Researchstack. http://researchstack.org. Accessed: 2017-01-02.Google ScholarGoogle Scholar
  18. B. EVANS. Paco-applying computational methods to scale qualitative methods. In Ethnographic Praxis in Industry Conference Proceedings, volume 2016, pages 348--368. Wiley Online Library, 2016.Google ScholarGoogle Scholar
  19. D. Ferreira, V. Kostakos, and A. K. Dey. Aware: mobile context instrumentation framework. Frontiers in ICT, 2:6, 2015.Google ScholarGoogle ScholarCross RefCross Ref
  20. J. Firth and J. Torous. Smartphone apps for schizophrenia: a systematic review. JMIR mHealth and uHealth, 3(4):e102, 2015.Google ScholarGoogle Scholar
  21. J. E. Fischer. Experience-sampling tools: a critical review. Mobile living labs, 9:1--3, 2009.Google ScholarGoogle Scholar
  22. R. C. Fraley and N. W. Hudson. Review of intensive longitudinal methods: An introduction to diary and experience sampling research, 2014.Google ScholarGoogle Scholar
  23. J. Froehlich, M. Y. Chen, S. Consolvo, B. Harrison, and J. A. Landay. Myexperience: a system for in situ tracing and capturing of user feedback on mobile phones. In Proceedings of the 5th international conference on Mobile systems, applications and services, pages 57--70. ACM, 2007. Google ScholarGoogle ScholarDigital LibraryDigital Library
  24. M. Goelema, M. Regis, R. Haakma, E. van den Heuvel, P. Markopoulos, and S. Overeem. Determinants of perceived sleep quality in normal sleepers (in press). Behavioural Sleep Medicine, 2017.Google ScholarGoogle Scholar
  25. A. Gordon. Surveymonkey. com-web-based survey and evaluation system: http://www. surveymonkey. com, 2002.Google ScholarGoogle Scholar
  26. J. M. Hektner, J. A. Schmidt, and M. Csikszentmihalyi. Experience sampling method: Measuring the quality of everyday life. Sage, 2007.Google ScholarGoogle Scholar
  27. T. Hendela. Apple introduces researchkit, giving medical researchers the tools to revolutionize medical studies. URL: http://www.apple.com/ca/pr/library/2015/03/, March 2015. Accessed: 2016--12--29.Google ScholarGoogle Scholar
  28. J. R. Hinrichs. Communications activity of industrial research personnel. Personnel Psychology, 17(2):193--206, 1964.Google ScholarGoogle ScholarCross RefCross Ref
  29. J. B. Hoeksma, S. M. Sep, F. C. Vester, P. F. Groot, R. Sijmons, and J. De Vries. The electronic mood device: Design, construction, and application. Behavior Research Methods, Instruments, & Computers, 32(2):322--326, 2000.Google ScholarGoogle Scholar
  30. A. E. Hühn, V.-J. Khan, P. Ketelaar, J. van't Riet, R. Konig, E. Rozendaal, N. Batalas, and P. Markopoulos. Does location congruence matter? a field study on the effects of location-based advertising on perceived ad intrusiveness, relevance & value. Computers in Human Behavior, 2017. Google ScholarGoogle ScholarDigital LibraryDigital Library
  31. S. S. Intille, J. Rondoni, C. Kukla, I. Ancona, and L. Bao. A context-aware experience sampling tool. In CHI'03 extended abstracts on Human factors in computing systems, pages 972--973. ACM, 2003. Google ScholarGoogle ScholarDigital LibraryDigital Library
  32. V. javed Khan, P. Markopoulos, D. Dolech, A. Eindhoven, B. Eggen, and D. Dolech. Features for the future experience sampling tool, 2009.Google ScholarGoogle Scholar
  33. S. Kieffer, N. Batalas, and P. Markopoulos. Towards task analysis tool support. In Proceedings of the 26th Australian Computer-Human Interaction Conference on Designing Futures: the Future of Design, pages 59--68. ACM, 2014. Google ScholarGoogle ScholarDigital LibraryDigital Library
  34. J. Maloney, M. Resnick, N. Rusk, B. Silverman, and E. Eastmond. The scratch programming language and environment. ACM Transactions on Computing Education (TOCE), 10(4):16, 2010. Google ScholarGoogle ScholarDigital LibraryDigital Library
  35. P. Markopoulos, N. Batalas, and A. Timmermans. On the use of personalization to enhance compliance in experience sampling. In Proceedings of the European Conference on Cognitive Ergonomics 2015, page 15. ACM, 2015. Google ScholarGoogle ScholarDigital LibraryDigital Library
  36. G. Metaxas, P. Markopoulos, and E. H. Aarts. Modelling social translucency in mediated environments. Universal Access in the Information Society, 11(3):311--321, 2012. Google ScholarGoogle ScholarDigital LibraryDigital Library
  37. J.-K. Min, A. Doryab, J. Wiese, S. Amini, J. Zimmerman, and J. I. Hong. Toss'n'turn: smartphone as sleep and sleep quality detector. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, pages 477--486. ACM, 2014. Google ScholarGoogle ScholarDigital LibraryDigital Library
  38. D. S. Moskowitz and S. N. Young. Ecological momentary assessment: what it is and why it is a method of the future in clinical psychopharmacology. Journal of psychiatry & neuroscience: JPN, 31(1):13, 2006.Google ScholarGoogle Scholar
  39. I. Myin-Germeys, M. Birchwood, and T. Kwapil. From environment to therapy in psychosis: a real-world momentary assessment approach. Schizophrenia bulletin, 37(2):244--247, 2011.Google ScholarGoogle ScholarCross RefCross Ref
  40. S. Offermans. Interacting with Light. Eindhoven University of Technology, April 2016.Google ScholarGoogle Scholar
  41. V. O'Reilly-Shah and S. Mackey. Survalytics: An open-source cloud-integrated experience sampling, survey, and analytics and metadata collection module for android operating system apps. JMIR mHealth and uHealth, 4(2):e46, 2016.Google ScholarGoogle Scholar
  42. F. Paternò. End user development: Survey of an emerging field for empowering people. ISRN Software Engineering, 2013, 2013.Google ScholarGoogle Scholar
  43. J. Quevedo-Fernández and J.-B. Martens. idanimate--supporting conceptual design with animation-sketching. In Collaboration in Creative Design, pages 251--270. Springer, 2016.Google ScholarGoogle Scholar
  44. N. Ramanathan, F. Alquaddoomi, H. Falaki, D. George, C.-K. Hsieh, J. Jenkins, C. Ketcham, B. Longstaff, J. Ooms, J. Selsky, et al. Ohmage: an open mobile system for activity and experience sampling. In 2012 6th International Conference on Pervasive Computing Technologies for Healthcare (PervasiveHealth) and Workshops, pages 203--204. IEEE, 2012.Google ScholarGoogle ScholarCross RefCross Ref
  45. M. ranzen, G. Sadikaj, D. Moskowitz, B. Ostafin, and M. aan het Rot. Intra- and interindividual variability in the behavioural, affective, and perceptual effects of alcohol consumption in a social context. Manuscript submitted for publication.Google ScholarGoogle Scholar
  46. C. Reynoldson, C. Stones, M. Allsop, P. Gardner, M. I. Bennett, S. J. Closs, R. Jones, and P. Knapp. Assessing the quality and usability of smartphone apps for pain self-management. Pain Medicine, 15(6):898--909, 2014.Google ScholarGoogle ScholarCross RefCross Ref
  47. D. Rough and A. Quigley. Jeeves-a visual programming environment for mobile experience sampling. In Visual Languages and Human-Centric Computing (VL/HCC), 2015 IEEE Symposium on, pages 121--129. IEEE, 2015.Google ScholarGoogle ScholarCross RefCross Ref
  48. J. D. Runyan, T. A. Steenbergh, C. Bainbridge, D. A. Daugherty, L. Oke, and B. N. Fry. A smartphone ecological momentary assessment/intervention "app" for collecting real-time data and promoting self-awareness. PLoS One, 8(8):e71325, 2013.Google ScholarGoogle ScholarCross RefCross Ref
  49. C. Schmitz et al. Limesurvey: An open source survey tool. LimeSurvey Project Hamburg, Germany. URL http://www. limesurvey. org, 2012.Google ScholarGoogle Scholar
  50. S. M. Schueller, M. Begale, F. J. Penedo, and D. C. Mohr. Purple: a modular system for developing and deploying behavioral intervention technologies. Journal of medical Internet research, 16(7):e181, 2014.Google ScholarGoogle Scholar
  51. C. Slofstra, M. Nauta, E. Holmes, E. Bos, M. Wichers, N. Batalas, N. Klein, and C. Bockting. Exploring the relation between visual mental imagery and affect in the daily life of previously depressed and never depressed individuals. Cognition and Emotion, 2017. In press.Google ScholarGoogle Scholar
  52. A. A. Stone, C. A. Bachrach, J. B. Jobe, H. S. Kurtzman, and V. S. Cain. The science of self-report: Implications for research and practice. Psychology Press, 1999.Google ScholarGoogle ScholarCross RefCross Ref
  53. A. A. Stone and S. Shiffman. Ecological momentary assessment (ema) in behavorial medicine. Annals of Behavioral Medicine, 1994.Google ScholarGoogle Scholar
  54. P. Totterdell and S. Folkard. In situ repeated measures of affect and cognitive performance facilitated by use of a hand-held computer. Behavior Research Methods, Instruments, & Computers, 24(4):545--553, 1992.Google ScholarGoogle Scholar
  55. D. Trossen and D. Pavel. Airs: A mobile sensing platform for lifestyle management research and applications. In International Conference on Mobile Wireless Middleware, Operating Systems, and Applications, pages 1--15. Springer, 2012.Google ScholarGoogle Scholar
  56. V. Venkatesh, M. G. Morris, G. B. Davis, and F. D. Davis. User acceptance of information technology: Toward a unified view. MIS quarterly, pages 425--478, 2003. Google ScholarGoogle ScholarDigital LibraryDigital Library
  57. P. Wilhelm and M. Perrez. A history of research conducted in daily life (scientific report nr. 170). 2013.Google ScholarGoogle Scholar
  58. M. Wolf-Meyer. Therapy, remedy, cure: disorder and the spatiotemporality of medicine and everyday life. Medical anthropology, 33(2):144--159, 2014.Google ScholarGoogle ScholarCross RefCross Ref
  59. H. Xiong, Y. Huang, L. E. Barnes, and M. S. Gerber. Sensus: a cross-platform, general-purpose system for mobile crowdsensing in human-subject studies. In Proceedings of the 2016 ACM International Joint Conference on Pervasive and Ubiquitous Computing, pages 415--426. ACM, 2016. Google ScholarGoogle ScholarDigital LibraryDigital Library

Index Terms

  1. Using TEMPEST: End-User Programming of Web-Based Ecological Momentary Assessment Protocols

      Recommendations

      Comments

      Login options

      Check if you have access through your login credentials or your institution to get full access on this article.

      Sign in

      Full Access

      PDF Format

      View or Download as a PDF file.

      PDF

      eReader

      View online with eReader.

      eReader
      About Cookies On This Site

      We use cookies to ensure that we give you the best experience on our website.

      Learn more

      Got it!