10.1145/2858036.2858047acmconferencesArticle/Chapter ViewAbstractPublication PageschiConference Proceedings
note

Utilizing Employees as Usability Participants: Exploring When and When Not to Leverage Your Coworkers

ABSTRACT

Usability testing is an everyday practice for usability professionals in corporations. But, as in all experimental situations, who you study can be as important as what you study. In this Note we explore a common practice in the corporation: experimenting on the company's employees. While fellow employees can be convenient and avoid issues such as confidentiality, we use two usability studies of mobile and web applications to show that employees spend less time-on-task on competitor websites than non-employees. Non-employees reliably rate competitor websites and apps higher than employees on both usability (on the 10-question SUS scale) and ease of use (on the 1-question SEQ scale). We conclude with recommendations for best practices for usability testing in the corporation.

References

  1. William Albert, Thomas Tullis, and Donna Tedesco. 2009. Beyond the Usability Lab: Conducting LargeScale Online User Experience Studies. Morgan Kaufmann. Google ScholarGoogle ScholarDigital LibraryDigital Library
  2. Sarah Anderson, Sarah Wandersee, Ariana Arcenas, and Lynn Baumgartner. 2013. "Craigslist samples of convenience: recruiting hard-to-reach populations." Retrieved August 13, 2015 from: http://fiesta.bren.ucsb.edu/~sanderson/CraigslistSurvey.Google ScholarGoogle Scholar
  3. Aaron Bangor, Philip T. Kortum, and James T. Miller. 2008. An empirical evaluation of the system usability scale. Intl. J. of Human-Computer Interaction 24, 6: 574--594.Google ScholarGoogle ScholarCross RefCross Ref
  4. Carol M. Barnum. 2010. Usability Testing Essentials: Ready, Set...Test! Elsevier. Google ScholarGoogle ScholarDigital LibraryDigital Library
  5. John Brooke. 1996. SUS - A quick and dirty usability scale. In Usability Evaluation in Industry, Patrick W. Jordan et al. (eds.). Taylor & Francis, 189--194.Google ScholarGoogle Scholar
  6. Mohamed A. Bujang, Puzziawati A. Ghani, Nur A. Zolkep, Shamini Selvarajah, and Jamaiyah Haniff. 2012. A comparison between convenience sampling versus systematic sampling in getting the true parameter in a population: Explore from a clinical database: the Audit Diabetes Control Management (ADCM) registry in 2009. In Statistics in Science, Business, and Engineering (ICSSBE), International Conference (IEEE, '12).Google ScholarGoogle Scholar
  7. Joseph S. Dumas and Janice Redish. 1999. A Practical Guide to Usability Testing. Intellect Books. Google ScholarGoogle ScholarDigital LibraryDigital Library
  8. Eugene H. Fram and Michael S. McCarthy. 2003. From employee to brand champion. Marketing Management 12, 1: 24--30.Google ScholarGoogle Scholar
  9. Wintre M. Gallander, C. North, and L. A. Sugar. 2001. Psychologists' response to criticisms about research based on undergraduate participants: A developmental perspective. Canadian Psychology 42, 3: 216.Google ScholarGoogle ScholarCross RefCross Ref
  10. David Gefen and Detmar W. Straub. 1997. Gender differences in the perception and use of e-mail: An extension to the technology acceptance model. MIS Quarterly, 21, 4: 389--400. Google ScholarGoogle ScholarDigital LibraryDigital Library
  11. Fredrick J. Gravetter and Lori-Ann B. Forzano. 2012. Research Methods for the Behavioral Sciences (4th ed.). Wadsworth Cengage Learning.Google ScholarGoogle Scholar
  12. Haredo Sahai and Mohammed I. Ageel. 2000. The Analysis of Variance: Fixed, Random and Mixed Models. Springer Science.Google ScholarGoogle Scholar
  13. Joseph Henrich, Steven J. Heine, and Ara Norenzayan, A. 2010. The weirdest people in the world? Behavioral & Brain Sciences 33, 2--3: 61--83.Google ScholarGoogle ScholarCross RefCross Ref
  14. Sari Kujala and Marjo Kauppinen. 2004. Identifying and selecting users for user-centered design. In Proceedings of the third Nordic Conference on Human-Computer Interaction (ACM, '04), 297--303. Google ScholarGoogle ScholarDigital LibraryDigital Library
  15. Mike Kuniavsky. 2003. Observing the User Experience: A Practitioner's Guide to User Research. Morgan Kaufmann. Google ScholarGoogle ScholarDigital LibraryDigital Library
  16. Jung-Joo Lee and Kun-Pyo Lee. 2007. Cultural differences and design methods for user experience research: Dutch and Korean participants compared. In Proceedings of the 2007 Conference on Designing Pleasurable Products and Interfaces (ACM, '07). Google ScholarGoogle ScholarDigital LibraryDigital Library
  17. James R. Lewis. 1986. Power switches: Some user expectations and preferences. In Proceedings of the Human Factors and Ergonomics Society Annual Meeting 30, 9: 895--899.Google ScholarGoogle ScholarCross RefCross Ref
  18. Michael G. Morris and Viswanath Venkatesh. 2000. Age differences in technology adoption decisions: Implications for a changing workforce. Personnel Psychology 53, 2: 375--403.Google ScholarGoogle ScholarCross RefCross Ref
  19. Jakob Nielsen. 1994. Usability Engineering. Elsevier.Google ScholarGoogle Scholar
  20. Lene Nielsen and Sabine Madsen. 2012. The usability expert's fear of agility - An empirical study of global trends and emerging practices. In Proceedings of the 7th Nordic Conference on Human-Computer Interaction: Making Sense Through Design (ACM, '12). Google ScholarGoogle ScholarDigital LibraryDigital Library
  21. Robert A. Peterson. 2001. On the use of college students in social science research: Insights from a second-order meta-analysis. Journal of Consumer Research 28:450 - 461.Google ScholarGoogle ScholarCross RefCross Ref
  22. Khanyapuss Punjaisri and Alan Wilson. 2007. The role of internal branding in the delivery of employee brand promise. Journal of Brand Management 15, 1: 57--70.Google ScholarGoogle ScholarCross RefCross Ref
  23. Khanyapuss Punjaisri, Evanschitzky Heiner, and Alan Wilson.2009. Internal branding: An enabler of employee brand-supporting behaviors. Journal of Service Management 20, 2: 209 - 226.Google ScholarGoogle ScholarCross RefCross Ref
  24. Stephanie Rosenbaum, Janice A. Rohn, and Judee Humburg Rosenbaum. 2000. A toolkit for strategic usability: Results from workshops, panels, and surveys. In Proceedings of the SIGCHI conference on Human Factors in Computing Systems (ACM, '00), 327--344. Google ScholarGoogle ScholarDigital LibraryDigital Library
  25. Allen Rubin and Earl R. Babbie. 2010. Essential Research Methods for Social Work (2nd ed.).Wadsworth Cengage Learning.Google ScholarGoogle Scholar
  26. Jeff Sauro. 2014. Measuring Usability with the System Usability Scale. Retrieved on August 6, 2014 from https://www.measuringusability.com/sus.phpGoogle ScholarGoogle Scholar
  27. Jeff Sauro. 2010. Do you need a random sample for your usability test? Retrieved August 15, 2015 from http://www.measuringu.com/blog/random-sample.phpGoogle ScholarGoogle Scholar
  28. Jeff Sauro and Joseph S. Dumas. 2009. Comparison of three one-question, post-task usability questionnaires. In Proceedings of the SIGCHI Conference on Human Factors in Computing (CHI'09), 1599--1608... Google ScholarGoogle ScholarDigital LibraryDigital Library
  29. Jeff Sauro and James R. Lewis. 2010. Average task times in usability tests: What to report. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI, '10). Google ScholarGoogle ScholarDigital LibraryDigital Library
  30. Andrew Sears and Julie A. Jacko (Eds.). 2009. Human-Computer Interaction: Development Process. CRC Press. Google ScholarGoogle ScholarDigital LibraryDigital Library
  31. Osama Sohaib and Khan Khalid. 2010. Integrating usability engineering and agile software development: A literature review. In Computer Design and Applications (ICCDA), (IEEE, '10), V2--33.Google ScholarGoogle Scholar
  32. Anne S. Tui, Jone L. Pearce, Lyman W. Porter and Angela M. Tripoli. 1997. Alternative approaches to the Employee-Organization Relationship: Does Investment in Employees Pay Off? Academic Management Journal, 40, 5, 1089--1121.Google ScholarGoogle ScholarCross RefCross Ref
  33. Tom Tullis and Bill Albert. 2013. Measuring the User Experience: Collecting, Analyzing, and Presenting Usability Metrics (2nd Ed.). Newnes. Google ScholarGoogle ScholarDigital LibraryDigital Library
  34. Brady T. West, Kathleen B. Welch, and Andrez T. Galecki. 2007. Linear Mixed Models: A Practical Guide Using Statistical Software. Taylor & Francis Group.Google ScholarGoogle Scholar

Index Terms

  1. Utilizing Employees as Usability Participants

    Comments

    Login options

    Check if you have access through your login credentials or your institution to get full access on this article.

    Sign in

    PDF Format

    View or Download as a PDF file.

    PDF

    eReader

    View online with eReader.

    eReader
    About Cookies On This Site

    We use cookies to ensure that we give you the best experience on our website.

    Learn more

    Got it!