Abstract
Researchers who perform Ecological Momentary Assessment (EMA) studies tend to rely on informatics experts to set up and administer their data collection protocols with digital media. Contrary to standard surveys and questionnaires that are supported by widely available tools, setting up an EMA protocol is a substantial programming task. Apart from constructing the survey items themselves, researchers also need to design, implement, and test the timing and the contingencies by which these items are presented to respondents. Furthermore, given the wide availability of smartphones, it is becoming increasingly important to execute EMA studies on user-owned devices, which presents a number of software engineering challenges pertaining to connectivity, platform independence, persistent storage, and back-end control. We discuss TEMPEST, a web-based platform that is designed to support non-programmers in specifying and executing EMA studies. We discuss the conceptual model it presents to end-users, through an example of use, and its evaluation by 18 researchers who have put it to real-life use in 13 distinct research studies.
- N. Aharony, A. Gardner, C. Sumter, and A. Pentland. Funf: Open sensing framework, 2011.Google Scholar
- D. J. Barrett and D. Barrett. Esp, the experience sampling program. Retrieved March, 1:2007, 2005.Google Scholar
- N. Batalas and P. Markopoulos. Considerations for computerized in situ data collection platforms. In Proceedings of the 4th ACM SIGCHI symposium on Engineering interactive computing systems, pages 231--236. ACM, 2012. Google Scholar
Digital Library
- N. Batalas and P. Markopoulos. Introducing tempest, a modular platform for in situ data collection. In Proceedings of the 7th Nordic Conference on Human-Computer Interaction: Making Sense Through Design, pages 781--782. ACM, 2012. Google Scholar
Digital Library
- N. Batalas, J. Quevedo-Fernandez, J.-B. Martens, and P. Markopoulos. Meet your users in situ data collection from within apps in large-scale deployments. International Journal of Handheld Computing Research (IJHCR), 6(3):17--32, 2015. Google Scholar
Digital Library
- F. M. Bos, R. A. Schoevers, and M. aan het Rot. Experience sampling and ecological momentary assessment studies in psychopharmacology: A systematic review. European Neuropsychopharmacology, 25(11):1853--1864, 2015.Google Scholar
Cross Ref
- R. Bosman, C. Albers, N. Batalas, and M. aan het Rot. No cyclicity in mood and social functioning in women with self-reported pms. Manuscript submitted for publication.Google Scholar
- J. Brooke et al. Sus-a quick and dirty usability scale. Usability evaluation in industry, 189(194):4--7, 1996.Google Scholar
- M. Capatu, G. Regal, J. Schrammel, E. Mattheiss, M. Kramer, N. Batalas, and M. Tscheligi. Capturing mobile experiences: Context-and time-triggered in-situ questionnaires on a smartphone. Measuring Behavior 2014, 2014.Google Scholar
- S. Carter, J. Mankoff, S. R. Klemmer, and T. Matthews. Exiting the cleanroom: On ecological validity and ubiquitous computing. Human-Computer Interaction, 23(1):47--99, 2008.Google Scholar
Cross Ref
- T. C. Christensen, L. F. Barrett, E. Bliss-Moreau, K. Lebo, and C. Kaschub. A practical guide to experience-sampling procedures. Journal of Happiness Studies, 4(1):53--78, 2003.Google Scholar
Cross Ref
- T. S. Conner and M. R. Mehl. Ambulatory assessment: Methods for studying everyday life. Emerging Trends in the Social and Behavioral Sciences: An Interdisciplinary, Searchable, and Linkable Resource, 2015.Google Scholar
- S. Consolvo, B. Harrison, I. Smith, M. Y. Chen, K. Everitt, J. Froehlich, and J. A. Landay. Conducting in situ evaluations for and with ubiquitous computing technologies. International Journal of Human-Computer Interaction, 22(1--2):103--118, 2007.Google Scholar
Cross Ref
- P. Copley. Research and evaluation of marketing communications. In Marketing communications management: concepts and theories, cases and practices, chapter 18. Routledge, 2004.Google Scholar
- M. Csikszentmihalyi, R. Larson, and S. Prescott. The ecology of adolescent activity and experience. Journal of youth and adolescence, 6(3):281--294, 1977.Google Scholar
- D. de Feijter, V.-J. Khan, and M. van Gisbergen. Confessions of a'guilty'couch potato understanding and using context to optimize binge-watching behavior. In Proceedings of the ACM International Conference on Interactive Experiences for TV and Online Video, pages 59--67. ACM, 2016. Google Scholar
Digital Library
- D. Estrin, M. Carroll, and N. Lakin. Researchstack. http://researchstack.org. Accessed: 2017-01-02.Google Scholar
- B. EVANS. Paco-applying computational methods to scale qualitative methods. In Ethnographic Praxis in Industry Conference Proceedings, volume 2016, pages 348--368. Wiley Online Library, 2016.Google Scholar
- D. Ferreira, V. Kostakos, and A. K. Dey. Aware: mobile context instrumentation framework. Frontiers in ICT, 2:6, 2015.Google Scholar
Cross Ref
- J. Firth and J. Torous. Smartphone apps for schizophrenia: a systematic review. JMIR mHealth and uHealth, 3(4):e102, 2015.Google Scholar
- J. E. Fischer. Experience-sampling tools: a critical review. Mobile living labs, 9:1--3, 2009.Google Scholar
- R. C. Fraley and N. W. Hudson. Review of intensive longitudinal methods: An introduction to diary and experience sampling research, 2014.Google Scholar
- J. Froehlich, M. Y. Chen, S. Consolvo, B. Harrison, and J. A. Landay. Myexperience: a system for in situ tracing and capturing of user feedback on mobile phones. In Proceedings of the 5th international conference on Mobile systems, applications and services, pages 57--70. ACM, 2007. Google Scholar
Digital Library
- M. Goelema, M. Regis, R. Haakma, E. van den Heuvel, P. Markopoulos, and S. Overeem. Determinants of perceived sleep quality in normal sleepers (in press). Behavioural Sleep Medicine, 2017.Google Scholar
- A. Gordon. Surveymonkey. com-web-based survey and evaluation system: http://www. surveymonkey. com, 2002.Google Scholar
- J. M. Hektner, J. A. Schmidt, and M. Csikszentmihalyi. Experience sampling method: Measuring the quality of everyday life. Sage, 2007.Google Scholar
- T. Hendela. Apple introduces researchkit, giving medical researchers the tools to revolutionize medical studies. URL: http://www.apple.com/ca/pr/library/2015/03/, March 2015. Accessed: 2016--12--29.Google Scholar
- J. R. Hinrichs. Communications activity of industrial research personnel. Personnel Psychology, 17(2):193--206, 1964.Google Scholar
Cross Ref
- J. B. Hoeksma, S. M. Sep, F. C. Vester, P. F. Groot, R. Sijmons, and J. De Vries. The electronic mood device: Design, construction, and application. Behavior Research Methods, Instruments, & Computers, 32(2):322--326, 2000.Google Scholar
- A. E. Hühn, V.-J. Khan, P. Ketelaar, J. van't Riet, R. Konig, E. Rozendaal, N. Batalas, and P. Markopoulos. Does location congruence matter? a field study on the effects of location-based advertising on perceived ad intrusiveness, relevance & value. Computers in Human Behavior, 2017. Google Scholar
Digital Library
- S. S. Intille, J. Rondoni, C. Kukla, I. Ancona, and L. Bao. A context-aware experience sampling tool. In CHI'03 extended abstracts on Human factors in computing systems, pages 972--973. ACM, 2003. Google Scholar
Digital Library
- V. javed Khan, P. Markopoulos, D. Dolech, A. Eindhoven, B. Eggen, and D. Dolech. Features for the future experience sampling tool, 2009.Google Scholar
- S. Kieffer, N. Batalas, and P. Markopoulos. Towards task analysis tool support. In Proceedings of the 26th Australian Computer-Human Interaction Conference on Designing Futures: the Future of Design, pages 59--68. ACM, 2014. Google Scholar
Digital Library
- J. Maloney, M. Resnick, N. Rusk, B. Silverman, and E. Eastmond. The scratch programming language and environment. ACM Transactions on Computing Education (TOCE), 10(4):16, 2010. Google Scholar
Digital Library
- P. Markopoulos, N. Batalas, and A. Timmermans. On the use of personalization to enhance compliance in experience sampling. In Proceedings of the European Conference on Cognitive Ergonomics 2015, page 15. ACM, 2015. Google Scholar
Digital Library
- G. Metaxas, P. Markopoulos, and E. H. Aarts. Modelling social translucency in mediated environments. Universal Access in the Information Society, 11(3):311--321, 2012. Google Scholar
Digital Library
- J.-K. Min, A. Doryab, J. Wiese, S. Amini, J. Zimmerman, and J. I. Hong. Toss'n'turn: smartphone as sleep and sleep quality detector. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, pages 477--486. ACM, 2014. Google Scholar
Digital Library
- D. S. Moskowitz and S. N. Young. Ecological momentary assessment: what it is and why it is a method of the future in clinical psychopharmacology. Journal of psychiatry & neuroscience: JPN, 31(1):13, 2006.Google Scholar
- I. Myin-Germeys, M. Birchwood, and T. Kwapil. From environment to therapy in psychosis: a real-world momentary assessment approach. Schizophrenia bulletin, 37(2):244--247, 2011.Google Scholar
Cross Ref
- S. Offermans. Interacting with Light. Eindhoven University of Technology, April 2016.Google Scholar
- V. O'Reilly-Shah and S. Mackey. Survalytics: An open-source cloud-integrated experience sampling, survey, and analytics and metadata collection module for android operating system apps. JMIR mHealth and uHealth, 4(2):e46, 2016.Google Scholar
- F. Paternò. End user development: Survey of an emerging field for empowering people. ISRN Software Engineering, 2013, 2013.Google Scholar
- J. Quevedo-Fernández and J.-B. Martens. idanimate--supporting conceptual design with animation-sketching. In Collaboration in Creative Design, pages 251--270. Springer, 2016.Google Scholar
- N. Ramanathan, F. Alquaddoomi, H. Falaki, D. George, C.-K. Hsieh, J. Jenkins, C. Ketcham, B. Longstaff, J. Ooms, J. Selsky, et al. Ohmage: an open mobile system for activity and experience sampling. In 2012 6th International Conference on Pervasive Computing Technologies for Healthcare (PervasiveHealth) and Workshops, pages 203--204. IEEE, 2012.Google Scholar
Cross Ref
- M. ranzen, G. Sadikaj, D. Moskowitz, B. Ostafin, and M. aan het Rot. Intra- and interindividual variability in the behavioural, affective, and perceptual effects of alcohol consumption in a social context. Manuscript submitted for publication.Google Scholar
- C. Reynoldson, C. Stones, M. Allsop, P. Gardner, M. I. Bennett, S. J. Closs, R. Jones, and P. Knapp. Assessing the quality and usability of smartphone apps for pain self-management. Pain Medicine, 15(6):898--909, 2014.Google Scholar
Cross Ref
- D. Rough and A. Quigley. Jeeves-a visual programming environment for mobile experience sampling. In Visual Languages and Human-Centric Computing (VL/HCC), 2015 IEEE Symposium on, pages 121--129. IEEE, 2015.Google Scholar
Cross Ref
- J. D. Runyan, T. A. Steenbergh, C. Bainbridge, D. A. Daugherty, L. Oke, and B. N. Fry. A smartphone ecological momentary assessment/intervention "app" for collecting real-time data and promoting self-awareness. PLoS One, 8(8):e71325, 2013.Google Scholar
Cross Ref
- C. Schmitz et al. Limesurvey: An open source survey tool. LimeSurvey Project Hamburg, Germany. URL http://www. limesurvey. org, 2012.Google Scholar
- S. M. Schueller, M. Begale, F. J. Penedo, and D. C. Mohr. Purple: a modular system for developing and deploying behavioral intervention technologies. Journal of medical Internet research, 16(7):e181, 2014.Google Scholar
- C. Slofstra, M. Nauta, E. Holmes, E. Bos, M. Wichers, N. Batalas, N. Klein, and C. Bockting. Exploring the relation between visual mental imagery and affect in the daily life of previously depressed and never depressed individuals. Cognition and Emotion, 2017. In press.Google Scholar
- A. A. Stone, C. A. Bachrach, J. B. Jobe, H. S. Kurtzman, and V. S. Cain. The science of self-report: Implications for research and practice. Psychology Press, 1999.Google Scholar
Cross Ref
- A. A. Stone and S. Shiffman. Ecological momentary assessment (ema) in behavorial medicine. Annals of Behavioral Medicine, 1994.Google Scholar
- P. Totterdell and S. Folkard. In situ repeated measures of affect and cognitive performance facilitated by use of a hand-held computer. Behavior Research Methods, Instruments, & Computers, 24(4):545--553, 1992.Google Scholar
- D. Trossen and D. Pavel. Airs: A mobile sensing platform for lifestyle management research and applications. In International Conference on Mobile Wireless Middleware, Operating Systems, and Applications, pages 1--15. Springer, 2012.Google Scholar
- V. Venkatesh, M. G. Morris, G. B. Davis, and F. D. Davis. User acceptance of information technology: Toward a unified view. MIS quarterly, pages 425--478, 2003. Google Scholar
Digital Library
- P. Wilhelm and M. Perrez. A history of research conducted in daily life (scientific report nr. 170). 2013.Google Scholar
- M. Wolf-Meyer. Therapy, remedy, cure: disorder and the spatiotemporality of medicine and everyday life. Medical anthropology, 33(2):144--159, 2014.Google Scholar
Cross Ref
- H. Xiong, Y. Huang, L. E. Barnes, and M. S. Gerber. Sensus: a cross-platform, general-purpose system for mobile crowdsensing in human-subject studies. In Proceedings of the 2016 ACM International Joint Conference on Pervasive and Ubiquitous Computing, pages 415--426. ACM, 2016. Google Scholar
Digital Library
Index Terms
Using TEMPEST: End-User Programming of Web-Based Ecological Momentary Assessment Protocols
Recommendations
Students' Experiences with Ecological Momentary Assessment Tools to Report on Emotional Well-being
Ecological Momentary Assessment (EMA) methods have emerged as an approach that enhances the ecological validity of data collected for the study of human behavior and experience. In particular, EMA methods are used to capture individuals' experiences (...
Internet use and non-use: views of older users
This paper reports the results of two connected surveys of computer and Internet use among the older population in the UK. One hundred and twenty questionnaires and interviews were completed with participants aged over 55 in Derbyshire and 353 ...
Willingness to Participate in Smartphone-Based Mobile Data Collection Studies
Human Aspects of IT for the Aged Population. Design, Interaction and Technology AcceptanceAbstractToday, digital and mobile forms of data collection are increasingly being used to capture information on the real lives of older adults. One method used is smartphone-based mobile data collection, which uses a standard smartphone to collect ...






Comments