skip to main content
research-article

Using Interaction Data to Explain Difficulty Navigating Online

Published:06 November 2014Publication History
Skip Abstract Section

Abstract

A user's behaviour when browsing a Web site contains clues to that user's experience. It is possible to record some of these behaviours automatically, and extract signals that indicate a user is having trouble finding information. This allows for Web site analytics based on user experiences, not just page impressions.

A series of experiments identified user browsing behaviours—such as time taken and amount of scrolling up a page—which predict navigation difficulty and which can be recorded with minimal or no changes to existing sites or browsers. In turn, patterns of page views correlate with these signals and these patterns can help Web authors understand where and why their sites are hard to navigate. A new software tool, “LATTE,” automates this analysis and makes it available to Web authors in the context of the site itself.

References

  1. Eugene Agichtein, Eric Brill, and Susan Dumais. 2006. Improving web search ranking by incorporating user behaviour information. In Proceedings of the International ACM SIGIR Conference on Research and Development in Information Retrieval. ACM Press, 19--26. Google ScholarGoogle ScholarDigital LibraryDigital Library
  2. Jaspreet S. Ahuja and Jane Webster. 2001. Perceived disorientation: An examination of a new meaure to assess web design effectiveness. Interact. with Computers 14, 15--29.Google ScholarGoogle ScholarCross RefCross Ref
  3. Hirotugu Akaike. 1974. A new look at the statistical model identification. IEEE Trans. Autom. Control 19, 6, 716--723.Google ScholarGoogle ScholarCross RefCross Ref
  4. Muhammed Sakaib Akram and Aneela Malik. 2012. Evaluating citizens' readiness to embrace e-government services. In Proceedings of the 13th International Conference on Digital Government Research. ACM Press, 58--67. Google ScholarGoogle ScholarDigital LibraryDigital Library
  5. Richard Atterer, Monika Wnuk, and Albrecht Schmidt. 2006. Knowing the user's every move—User activity tracking for website usability evaluation and implicit interaction. In Proceedings of the 15th International World Wide Web Conference. ACM Press, 203--212. Google ScholarGoogle ScholarDigital LibraryDigital Library
  6. Anne Aula, Rehan M. Khan, and Zhiwei Guan. 2010. How does search behaviour change as search becomes more difficult? In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems. ACM Press, 35--44. Google ScholarGoogle ScholarDigital LibraryDigital Library
  7. José Borges and Mark Levene. 1999. Data mining of user navigation patterns. In Proceedings of the WEBKDD'99 Workshop on Web Usage Analysis and User Profiling, Lecture Notes in Artificial Intelligence, Vol. 1836, Springer, 92--112. Google ScholarGoogle ScholarDigital LibraryDigital Library
  8. Rodrigo A. Botafogo, Ehud Rivlin, and Ben Shneiderman. 1992. Structural analysis of hypertexts: Identifying hierarchies and useful metrics. ACM Trans. Inf. Syst. 10, 2, 142--180. Google ScholarGoogle ScholarDigital LibraryDigital Library
  9. Kenneth P. Burnham and David R. Anderson. 2005. Model Selection and Multimodel Inference: A Practical Information-Theoretic Approach 2nd Ed. Springer.Google ScholarGoogle Scholar
  10. Cabinet Office. 2012. Digital efficiency report. http://publications.cabinetoffice.gov.uk/digital/efficiency/digital-efficiency-report.pdf.Google ScholarGoogle Scholar
  11. Igor Cadez, David Heckerman, Christopher Meek, Padhraic Smith, and Steven White. 2000. Visualization of navigation patters on a web site using model-based clustering. In Proceedings of the Conference on Knowledge Discovery and Data Mining. 280--284. Google ScholarGoogle ScholarDigital LibraryDigital Library
  12. Igor Cadez, David Heckerman, Christopher Meek, Padhraic Smith, and Steven White. 2003. Model-based clustering and visualization of navigation patterns on a web site. Data Mining Knowl. Discov. 7, 399--424. Google ScholarGoogle ScholarDigital LibraryDigital Library
  13. Ed H. Chi, Peter Pirolli, Kim Chen, and James Pitkow. 2001. Using information scent to model user information needs and actions and the web. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems. ACM Press, 490--497. Google ScholarGoogle ScholarDigital LibraryDigital Library
  14. Ed H. Chi, Adam Rosien, Gesara Supattanasiri, Amanda Williams, Christiaan Royer, Celia Chow, Erica Robles, Brinda Dalal, Julie Chen, and Steve Cousins. 2003. The Bloodhound project: Automating discovery of web usability issues using the InfoScent™ simulator. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems. ACM Press, 505--512. Google ScholarGoogle ScholarDigital LibraryDigital Library
  15. Timo Christophersen and Udo Konradt. 2011. Reliability, validity, and sensitivity of a single-item measure of online store usability. Int. J. Human-Computer Studies 69, 269--280. Google ScholarGoogle ScholarDigital LibraryDigital Library
  16. Mark Claypool, Phong Le, Makoto Waseda, and David Brown. 2001. Implicit interest indicators. In Proceedings of the Conference on Intelligent User Interfaces. 33--40. Google ScholarGoogle ScholarDigital LibraryDigital Library
  17. Robert Cooley. 2003. The use of web structure and content to identify subjectively interesting web usage patterns. ACM Trans. Internet Technol. 3, 2, 93--116. Google ScholarGoogle ScholarDigital LibraryDigital Library
  18. Fred D. Davis. 1989. Perceived usefulness, preceived ease of use, and user acceptance of information technology. MIS Quart. 13, 3, 319--340. Google ScholarGoogle ScholarDigital LibraryDigital Library
  19. Maureen Dostert. 2011. Does domain knowledge influence search stopping behaviour?. In Proceedings of the Annual Meeting of the Association for Information Science and Technology. Association for Information Science and Technology.Google ScholarGoogle Scholar
  20. Tom Fawcett. 2006. An introduction to ROC analysis. Pattern Recog. Lett. 27, 861--874. Google ScholarGoogle ScholarDigital LibraryDigital Library
  21. Henry Feild, James Allan, and Rosie Jones. 2010. Predicting searcher frustration. In Proceedings of the International ACM SIGIR Conference on Research and Development in Information Retrieval. ACM Press, 34--41. Google ScholarGoogle ScholarDigital LibraryDigital Library
  22. Adrian Fernandez, Emilio Insfran, and Silvia Abrah ao. 2011. Usability evaluation methods for the web: A systematicmapping study. Inf. Software Tech. 53, 789--817. Google ScholarGoogle ScholarDigital LibraryDigital Library
  23. R. Fielding, J. Gettys, J. Mogul, H. Frystyk, L. Masinter, P. Leach, and T. Berners-Lee. 1999. Hypertext transfer protocol—HTTP/1.1. RFC 2616. Google ScholarGoogle ScholarDigital LibraryDigital Library
  24. Carolyn L. Foss. 1989. Detecting lost users: Emprical studies of browsing hypertext. Rapport de Recherche 972, INRIA.Google ScholarGoogle Scholar
  25. Steve Fox, Kuldeep Karnawat, Mark Mydland, Susan Dumais, and Thomas White. 2005. Evaluating implicit measures to improve web search. ACM Trans. Inf. Syst. 23, 2, 147--168. Google ScholarGoogle ScholarDigital LibraryDigital Library
  26. Wai-Tat Fu and Peter Pirolli. 2007. SNIF-ACT: A cognitive model of user navigation on the world wide web. Human-Computer Interact. 22, 4, 355--412. Google ScholarGoogle ScholarDigital LibraryDigital Library
  27. David Gilbert, Pierre Balestrini, and Darren Littleboy. 2004. Barriers and benefits in the adoption of e-government. Int. J. Public Sector Management 17, 4, 286--301.Google ScholarGoogle ScholarCross RefCross Ref
  28. Qi Guo, Haojian Jin, Dmitry Lagu, Shuai Yuan, and Eugene Agichtein. 2013. Mining touch interaction data on mobile devices to predict web search result relevance. In Proceedings of the International ACM SIGIR Conference on Research and Development in Information Retrieval. ACM Press, 153--162. Google ScholarGoogle ScholarDigital LibraryDigital Library
  29. Jacek Gwizdka and Ian Spence. 2006. What can searching behaviour tell us about the difficulty of information tasks? A study of web navigation. In Proceedings of the 69th Annual Meeting of the American Society for Information Science and Technology. 1--7.Google ScholarGoogle Scholar
  30. Jacek Gwizdka and Ian Spence. 2007. Implicit measures of lostness and success in web navigation. Interact. with Computers 19, 357--369. Google ScholarGoogle ScholarDigital LibraryDigital Library
  31. Daqing He and Ayşe Göker. 2000. Detecting session boundaries from web user logs. In Proceedings of the BCS-IRSG 22nd Annual Colloquium on Information Retrieval Research. 57--66.Google ScholarGoogle Scholar
  32. Eelco Herder. 2003. Revisitation patterns and disorientation. In Proceedings of the German Workshop on Adaptivity and User Modeling in Interactive Systems. 291--294.Google ScholarGoogle Scholar
  33. Melody Y. Ivory. 2003. Automated Web Site Evaluation: Researchers' and Practitioners' Perspectives. Springer. Google ScholarGoogle ScholarDigital LibraryDigital Library
  34. Bernard J. Jansen. 2009. Understanding User-Web Interactions via Web Analytics. Synthesis Lectures on Information Concepts, Retrieval, and Services, No. 6, Morgan & Claypool. Google ScholarGoogle ScholarDigital LibraryDigital Library
  35. Thorsten Joachims, Laura Granka, Bing Pan, Helene Hembrooke, and Geri Gay. 2005. Accurately interpreting clickthrough data as implicit feedback. In Proceedings of the International ACM SIGIR Conference on Research and Development in Information Retrieval. ACM Press, 154--161. Google ScholarGoogle ScholarDigital LibraryDigital Library
  36. Thorsten Joachims, Laura Granka, Bing Pan, Helene Hembrooke, Filip Radlinksi, and Geri Gay. 2007. Evaluating the accuracy of implicit feedback from clicks and query reformulations in web search. ACM Trans. Inf. Syst. 25, 2, Article 7. Google ScholarGoogle ScholarDigital LibraryDigital Library
  37. Rosie Jones and Kristina Lisa Klinkner. 2008. Beyond the session timeout: Automatic hierarchical segmentation of search topics in query logs. In Proceedings of the ACM International Conference on Information and Knowledge Management. ACM Press. Google ScholarGoogle ScholarDigital LibraryDigital Library
  38. Diane Kelly and Nicholas J. Belkin. 2004. Display time as implicit feedback: Understanding task effects. In Proceedings of the International ACM SIGIR Conference on Research and Development in Information Retrieval. ACM Press. Google ScholarGoogle ScholarDigital LibraryDigital Library
  39. Janette Lehmann, Mounia Lalmas, Elad Yom-Tov, and Georges Dupret. 2012. Models of user engagement. In Proceedings of the International Conference on User Modeling, Adaptation, and Personalization, Lecture Notes in Computer Science, vol. 7379, Springer, 164--175. Google ScholarGoogle ScholarDigital LibraryDigital Library
  40. Chang Liu, Jacek Gwizdka, and Jingjing Liu. 2010. Helping identify when users find useful documents: Examination of query reformulation intervals. In Proceedings of the Conference on Information Interaction in Context. ACM Press, 215--224. Google ScholarGoogle ScholarDigital LibraryDigital Library
  41. Jingjing Liu, Chang Liu, Michael Cole, Nicholas J. Belkin, and Xiangmin Zhang. 2012. Exploring and predicting search task difficulty. In Proceedings of the 21st ACM International Conference on Information and Knowledge Management. ACM Press, 1313--1322. Google ScholarGoogle ScholarDigital LibraryDigital Library
  42. Jingjing Liu, Chang Liu, Jacek Gwizdka, and Nicholas J. Belkin. 2010. Can search systems detect users' task difficulty? Some behavioural signals. In Proceedings of the International ACM SIGIR Conference on Research and Development in Information Retrieval. ACM Press, 845--846. Google ScholarGoogle ScholarDigital LibraryDigital Library
  43. Claudio Lucchese, Salvatore Orlando, Raffaele Perego, Fabrizio Silverstri, and Gabriele Tolomei. 2013. Discovering tasks from search engine query logs. ACM Trans. Inf. Syst. 31, 3, 14:1--14:43. Google ScholarGoogle ScholarDigital LibraryDigital Library
  44. Nizar R. Mabroukeh and C. I. Ezeife. 2010. A taxonomy of sequential pattern mining algorithms. ACM Comput. Surv. 43, 1, 3:1--3:41. Google ScholarGoogle ScholarDigital LibraryDigital Library
  45. F. Masseglia, P. Poncelet, and M. Teisseire. 1999. Using data mining techniques on web access logs to dynamically improve hypertext structure. ACM SIGWEB Newsl. 8, 3, 13--19. Google ScholarGoogle ScholarDigital LibraryDigital Library
  46. Takehiro Nakayama, Hiroki Kato, and Yohei Yamane. 2000. Discovering the gap between web site designers' expectations and users' behaviour. Computer Netw. 33, 811--822. Google ScholarGoogle ScholarDigital LibraryDigital Library
  47. Peter Peduzzi, John Concato, Elizabeth Kemper, Theodore R. Holford, and Alvan R. Feinstein. 1996. A simulation study of the number of events per variable in logistic regression analysis. J. Clinical Epidemiology 49, 12, 1373--1379.Google ScholarGoogle ScholarCross RefCross Ref
  48. R Core Team. 2012. R: A language and environment for statistical computing. R Foundation for Statistical Computing, Vienna, Austria. http://www.R-project.org/.Google ScholarGoogle Scholar
  49. Louis Rosenfeld. 2011. Search Analytics for Your Site: Conversations with Your Customers. Rosenfeld Media, Brooklyn. Google ScholarGoogle ScholarDigital LibraryDigital Library
  50. Fabrizio Silvestri. 2010. Mining query logs: Turning search usage data into knowledge. Found. Trends Inf. Retrieval 4, 1--2. Google ScholarGoogle ScholarDigital LibraryDigital Library
  51. Katherine L. Smith and Paul B. Kantor. 2008. User adaptation: Good results from poor systems. In Proceedings of the International ACM SIGIR Conf. on Research and Development in Information Retrieval. ACM Press, 147--154. Google ScholarGoogle ScholarDigital LibraryDigital Library
  52. Pauline A. Smith. 1996. Towards a practical measure of hypertext usability. Interacting with Computers 8, 4, 365--381. Google ScholarGoogle ScholarDigital LibraryDigital Library
  53. Myra Spiliopoulou, Carsten Pohle, and Lukas C. Faulstich. 1999. Improving the effectiveness of a web site with web usage mining. In Proceedings of the International Workshop on Web Usage Analysis and User Profiling, Lecture Notes in Artificial Intelligence. Springer, 142--162. Google ScholarGoogle ScholarDigital LibraryDigital Library
  54. Leong-Hwee Teo, Bonnie E. John, and Marilyn Hughes Blackmon. 2012. CogTool-Explorer: A model of goal-directed user exploration that considers information layout. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems 2479--2488. Google ScholarGoogle ScholarDigital LibraryDigital Library
  55. Paul Thomas. 2012. Explaining difficulty navigating a website using page view data. In Proceedings of the Australasian Document Computing Symposium. ACM Press, 31--38. Google ScholarGoogle ScholarDigital LibraryDigital Library
  56. Paul Thomas, Alex O'Neill, and Cécile Paris. 2010. Interaction differences in web search and browse logs. In Proceedings of the Australasian Document Computing Symposium. 52--59.Google ScholarGoogle Scholar
  57. Paul Thomas, Falk Scholer, and Alistair Moffat. 2013. Fading away: Dilution and user behaviour. In Proceedings of the European Workshop on Human-Computer Interaction and Information Retrieval. 3--6.Google ScholarGoogle Scholar
  58. Wan-Ching Wu, Diane Kelly, Ashlee Edwards, and Jaime Arguello. 2012. Grannies, tanning beds, tattoos and NASCAR: Evaluation of search tasks with varying levels of cognitive complexity. In Proceedings of the Conference on Information Interaction in Context. 254--257. Google ScholarGoogle ScholarDigital LibraryDigital Library
  59. Yuming Zhou, Hareton Leung, and Pinata Winoto. 2007. MNav: A Markov model-based web site navigability measure. IEEE Trans. Softw. Eng. 33, 12, 869--890. Google ScholarGoogle ScholarDigital LibraryDigital Library

Index Terms

  1. Using Interaction Data to Explain Difficulty Navigating Online

          Recommendations

          Comments

          Login options

          Check if you have access through your login credentials or your institution to get full access on this article.

          Sign in

          Full Access

          • Published in

            cover image ACM Transactions on the Web
            ACM Transactions on the Web  Volume 8, Issue 4
            October 2014
            178 pages
            ISSN:1559-1131
            EISSN:1559-114X
            DOI:10.1145/2686863
            Issue’s Table of Contents

            Copyright © 2014 ACM

            Publisher

            Association for Computing Machinery

            New York, NY, United States

            Publication History

            • Published: 6 November 2014
            • Revised: 1 July 2014
            • Accepted: 1 July 2014
            • Received: 1 January 2014
            Published in tweb Volume 8, Issue 4

            Permissions

            Request permissions about this article.

            Request Permissions

            Check for updates

            Qualifiers

            • research-article
            • Research
            • Refereed

          PDF Format

          View or Download as a PDF file.

          PDF

          eReader

          View online with eReader.

          eReader
          About Cookies On This Site

          We use cookies to ensure that we give you the best experience on our website.

          Learn more

          Got it!