How does search behavior change as search becomes more difficult?

ABSTRACT

Search engines make it easy to check facts online, but finding some specific kinds of information sometimes proves to be difficult. We studied the behavioral signals that suggest that a user is having trouble in a search task. First, we ran a lab study with 23 users to gain a preliminary understanding on how users' behavior changes when they struggle finding the information they're looking for. The observations were then tested with 179 participants who all completed an average of 22.3 tasks from a pool of 100 tasks. The large-scale study provided quantitative support for our qualitative observations from the lab study. When having difficulty in finding information, users start to formulate more diverse queries, they use advanced operators more, and they spend a longer time on the search result page as compared to the successful tasks. The results complement the existing body of research focusing on successful search strategies.

References

1. Aula, A. (2003) Query Formulation in Web Information Search. Proc. IADIS WWW/Internet 2003, 403--410.
2. Aula, A., Majaranta, P., and Räihä, K.-J. (2005) Eye-tracking reveals the personal styles for search result evaluation. Proceedings of Human-Computer Interaction -- INTERACT 2005, 1058--1061.
3. Aula, A. & Nordhausen, K. (2006) Modeling successful performance in web searching. Journal of the American Society for Information Science and Technology, 57(12), 1678--1693.
4. Aula, A. and Siirtola, H. (2005) Hundreds of folders or one ugly pile -- strategies for information search and re-access. Proc. INTERACT 2005, 954--957.
5. Brand-Gruwel, S., Wopereis, I. and Vermetten, Y. (2005) Information problem solving by experts and novices: analysis of a complex cognitive skill. Computers in Human Behavior, 21, 487--508.
6. Downey, D., Dumais, S., Liebling, D., and Horvitz, E. (2008) Understanding the relationship between searchers' queries and information goals. CIKM'08, 449--458.
7. Eastman, C.M. and Jansen, B.J. (2003) Coverage, relevance, and ranking: the impact of query operators on web search engine results. ACM Transactions on Information Systems, 21(4), 383--411.
8. Fields, B., Keith, S. and Blandford, A. (2004) Designing for expert information finding strategies. Proc. of HCI 2004, 89--102.
9. Granka, L.A., Joachims, T. Gay, G. (2004) Eye-tracking analysis of user behavior in WWW search. Proc. SIGIR'04, 478--479.
10. Huntington, P., Nicholas, D., and Jamali, H.R. (2007) Employing log metrics to evaluate search behavior and success: case study BBC search engine. Journal of Information Science, 33(5), 584--597.
11. Hölscher, C. and Strube, G. (2000) Web search behavior of internet experts and newbies. Proc. WWW 2000, 337--346.
12. Jansen, B. and Spink, A. (2006) How are we searching the world wide web? A comparison of nine search engine transaction logs. Information Processing and Management, 42, 248--263.
13. Jansen, B., Spink, A. and Saracevic, T. (1998) Real life information retrieval: a study of user queries on the web. SIGIR Forum, 32(1), 5--17.
14. Jansen, B., Spink, A. and Saracevic, T. (2000) Real life, real users, and real needs: A study and analysis of user queries on the web. Information Processing and Management, 36(2), 207--227.
15. Jenkins, C., Corritore, C.L. and Wiedenbeck, S. (2003) Patterns of information seeking on the web: a qualitative study of domain expertise and web expertise. IT & Society, 1(3), 64--89.
16. Kamvar, M., Kellar, M., Patel, R. and Xu, Y. (2009) Computers and iPhones and mobile phones, oh my! A logs-based comparison of search users on different devices. In Proc. WWW 2009, 801--810.
17. Khan, K. and Locatis, C. (1998) Searching through the cyberspace: the effects of link display and link density on information retrieval from hypertext on the World Wide Web. Journal of the American Society for Information Science, 49(2), 176--182.
18. Lazonder, A.W., Biemans, H.J.A. and Worpeis, I.G.J.H. (2000) Differences between novice and experienced users in searching information on the World Wide Web. Journal of American Society for Information Science, 51(6), 576--581.
19. Navarro-Prieto, R., Scaife, M. and Rogers, Y. (1999) Cognitive strategies in web searching. Proc. 5th Conference on Human Factors and the Web. Retrieved September 15, 2009, http://zing.ncsl.nist.gov/hfweb/proceedings/navarro-prieto/
20. Rose, D.E. and Levinson, D. (2004) Understanding user goals in Web search. In Proc. WWW 2004, 13--19.
21. Silverstein, C., Henzinger, M., Marais, H. and Moricz, M. (1999) Analysis of a very large web search engine query log. SIGIR Forum, 33(1), 6--12.
22. Saito, H. and Miwa, K. (2002) A cognitive study of information seeking processes in the WWW: the effects of searcher's knowledge and experience. Proc. WISE'01, 321--333.
23. Spink, A., Wolfram, D., Jansen, B.J. and Saracevic, T. (2002) From e-sex to e-commerce: web search changes. IEEE Computer, 35(3), 107--109.
24. Teevan, J. Alvarado, C., Ackerman, M.S. and Karger, D.R. (2004) The perfect search engine is not enough: a study of orienteering behavior in directed search. Proc. CHI'2004, 415--422.
25. White, R.W. and Drucker, S.M. (2007) Investigating behavioral variability in web search. Proc. WWW 2007, 21--30.
26. White, R.W. and Morris, D. (2007) Investigating the querying and browsing behavior of advanced search engine users. Proc. SIGIR 2007, 255--262.

Index Terms

• Human-centered computing