skip to main content
research-article

Captions and biases in diagnostic search

Published:01 November 2013Publication History
Skip Abstract Section

Abstract

People frequently turn to the Web with the goal of diagnosing medical symptoms. Studies have shown that diagnostic search can often lead to anxiety about the possibility that symptoms are explained by the presence of rare, serious medical disorders, rather than far more common benign syndromes. We study the influence of the appearance of potentially-alarming content, such as severe illnesses or serious treatment options associated with the queried for symptoms, in captions comprising titles, snippets, and URLs. We explore whether users are drawn to results with potentially-alarming caption content, and if so, the implications of such attraction for the design of search engines. We specifically study the influence of the content of search result captions shown in response to symptom searches on search-result click-through behavior. We show that users are significantly more likely to examine and click on captions containing potentially-alarming medical terminology such as “heart attack” or “medical emergency” independent of result rank position and well-known positional biases in users' search examination behaviors. The findings provide insights about the possible effects of displaying implicit correlates of searchers' goals in search-result captions, such as unexpressed concerns and fears. As an illustration of the potential utility of these results, we developed and evaluated an enhanced click prediction model that incorporates potentially-alarming caption features and show that it significantly outperforms models that ignore caption content. Beyond providing additional understanding of the effects of Web content on medical concerns, the methods and findings have implications for search engine design. As part of our discussion on the implications of this research, we propose procedures for generating more representative captions that may be less likely to cause alarm, as well as methods for learning to more appropriately rank search results from logged search behavior, for examples, by also considering the presence of potentially-alarming content in the captions that motivate observed clicks and down-weighting clicks seemingly driven by searchers' health anxieties.

References

  1. Agichtein, E., Brill, E., and Dumais, S. 2006. Improving Web search ranking by incorporating user behavior. In Proceedings of the Annual International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR). 19--26. Google ScholarGoogle ScholarDigital LibraryDigital Library
  2. Agichtein, E., Brill, E., Dumais, S., and Ragno, R. 2006. Learning user interaction models for predicting web search result preferences. In Proceedings of the Annual International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR). 3--10. Google ScholarGoogle ScholarDigital LibraryDigital Library
  3. Anderson, G. and Hussey, P. S. 2001. Comparing health system performance in OECD countries. Health Affairs 20, 3, 219--232.Google ScholarGoogle ScholarCross RefCross Ref
  4. Asmundson, G. J. C., Taylor, S., and Cox, B. J. 2001. Health Anxiety: Clinical and Research Perspectives on Hypochondriasis and Related Conditions. Wiley.Google ScholarGoogle Scholar
  5. Ayers, S. and Kronenfeld. J. 2007. Chronic illness and health-seeking information on the Internet. Health, 11, 3.Google ScholarGoogle Scholar
  6. Beeferman, D. and Berger, A. 2000. Agglomerative clustering of a search engine query log. In Proceedings of the International ACM SIGKDD Conference on Knowledge Discovery and Data Mining (KDD). 407--416. Google ScholarGoogle ScholarDigital LibraryDigital Library
  7. Bhavnani, S. K., Jacob, R. T., Nardine, J., and Peck, F. A. 2003. Exploring the distribution of online healthcare information. In Proceedings of the Extended Abstracts on Human Factors in Computing Systems (SIGCHI). 816--817. Google ScholarGoogle ScholarDigital LibraryDigital Library
  8. Brin, S. and Page, L. 1998. Anatomy of a large-scale hypertextual Web search engine. In Proceedings of the WWW. 107--117. Google ScholarGoogle ScholarDigital LibraryDigital Library
  9. Buscher, G., White, R. W., Dumais, S. T., and Huang, J. 2012. Large-scale analysis of individual and task differences in search result page examination strategies. In Proceedings of the International ACM Conference on Web Search and Data Mining (WSDM). 373--382. Google ScholarGoogle ScholarDigital LibraryDigital Library
  10. Buscher, G., Dumais, S. T., and Cutrell, E. 2010. The good, the bad, and the random: An eye-tracking study of ad quality in web search. In Proceedings of the Annual International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR). 42--49. Google ScholarGoogle ScholarDigital LibraryDigital Library
  11. Cartright, M., White, R. W., and Horvitz, E. 2011. Intentions and attention in exploratory health search. In Proceedings of the Annual International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR). 65--74 Google ScholarGoogle ScholarDigital LibraryDigital Library
  12. Chapelle, O. and Zhang, Y. 2009. A dynamic Bayesian network click model for Web search ranking. In Proceedings of the WWW Conference. 1--10. Google ScholarGoogle ScholarDigital LibraryDigital Library
  13. Cho, J. and Roy, S. 2004. Impact of search engines on page popularity. In Proceedings of the WWW Conference. 20--29. Google ScholarGoogle ScholarDigital LibraryDigital Library
  14. Clarke, C., Agichtein, E., Dumais, S. T., and White, R. W. 2007. The influence of caption features on click-through patterns in Web search. In Proceedings of the Annual International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR). 135--142. Google ScholarGoogle ScholarDigital LibraryDigital Library
  15. Craswell, N., Zoeter, O., Taylor, M., and Ramsey, B. 2008. An experimental comparison of click position-bias models. In Proceedings of the International ACM Conference on Web Search and Data Mining (WSDM). 87--94. Google ScholarGoogle ScholarDigital LibraryDigital Library
  16. Cutrell, E. and Guan, Z. 2007. What are you looking for? An eye-tracking study of information usage in web search. In Proceedings of the ACM SIGCHI Extended Abstracts on Human Factors in Computing Systems (SIGCHI). 407--416. Google ScholarGoogle ScholarDigital LibraryDigital Library
  17. Diriye, A., White, R. W., Buscher, G., and Dumais, S. T. 2012. Leaving so soon? Understanding and predicting Web search abandonment. In Proceedings of the International Conference on Information and Knowledge Management (CIKM). 1025--1034. Google ScholarGoogle ScholarDigital LibraryDigital Library
  18. Dupret, G. E. and Liao, C. 2010. A model to estimate intrinsic document relevance from the click-through logs of a Web search engine. In Proceedings of the International ACM Conference on Web Search and Data Mining (WSDM). 181--190. Google ScholarGoogle ScholarDigital LibraryDigital Library
  19. Dupret, G. E. and Piwowarski, B. 2008. A user browsing model to predict search engine click data from past observations. In Proceedings of the Annual International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR). 331--338. Google ScholarGoogle ScholarDigital LibraryDigital Library
  20. Eysenbach, G. and Kohler, K. 2002. How do consumers search for and appraise health information on the World Wide Web? Bri. Med. J. 324, 573--577.Google ScholarGoogle ScholarCross RefCross Ref
  21. Fortunato, S., Flammini, A., Menczer, F., and Vespignani, A. 2006. Topical interests and the mitigation of search engine bias. Proc. Nat. Acad. Sci. U. S. A. 103, 34, 12684--12689.Google ScholarGoogle ScholarCross RefCross Ref
  22. Fox, S. 2011. Health topics. Pew Internet Amer. Life Project. http://pewinternet.org/Reports/2011/Health-online.aspx.Google ScholarGoogle Scholar
  23. Fox, S. and Duggan, M. 2013. Health topics. Pew Internet Amer. Life Project. http://pewinternet.org/Reports/2013/Health-online.aspx.Google ScholarGoogle Scholar
  24. Fox, S., Karnawat, K., Mydland, M., Dumais, S., and White, T. 2005. Evaluating implicit measures to improve the search experience. ACM Trans. Inf. Syst. 23, 2, 147--168. Google ScholarGoogle ScholarDigital LibraryDigital Library
  25. Gerhart, S. 2004. Do Web search engines suppress controversy? First Monday, 9, 1--5.Google ScholarGoogle Scholar
  26. Goldman, E. 2006. Search engine bias and the demise of search utopianism. Yale J. Law Technol. 188.Google ScholarGoogle Scholar
  27. Guo, F., Liu, C., Kannan. A., Minka, T., Taylor, M. J., Wang, Y. M., and Faloutsos, C. 2009. Click chain model in Web search. In Proceedings of the WWW Conference. 11--20. Google ScholarGoogle ScholarDigital LibraryDigital Library
  28. Guo, F., Liu, C., and Wang, Y. M. 2009. Efficient multiple-click models in web search. In Proceedings of the International ACM Conference on Web Search and Data Mining (WSDM). 124--131. Google ScholarGoogle ScholarDigital LibraryDigital Library
  29. Guo, Q. and Agichtein, E. 2010. Ready to buy or just browsing? Detecting Web searcher goals from interaction data. In Proceedings of the Annual International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR). 130--137. Google ScholarGoogle ScholarDigital LibraryDigital Library
  30. Guo, Q., White, R. W., Zhang, Q., Anderson, B., and Dumais, S. 2011. Why searchers switch: Understanding and predicting engine switching rationales. In Proceedings of the Annual International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR). 335--344. Google ScholarGoogle ScholarDigital LibraryDigital Library
  31. Huang, J., White, R. W., and Dumais, S. T. 2011. No clicks, no problem: Using cursor movements to understand and improve search. In Proceedings of the ACM SIGCHI Extended Abstracts on Human Factors in Computing Systems (SIGCHI). 1225--1234. Google ScholarGoogle ScholarDigital LibraryDigital Library
  32. Ieong, S., Mishra, N., Sadikov, E., and Zhang, L. 2012. Domain bias in Web search. In Proceedings of the International ACM Conference on Web Search and Data Mining (WSDM). 413--422. Google ScholarGoogle ScholarDigital LibraryDigital Library
  33. Joachims, T. 2002. Optimizing search engines using click-through data. In Proceedings of the International ACM SIGKDD Conference on Knowledge Discovery and Data Mining (SIGKDD). 132--142. Google ScholarGoogle ScholarDigital LibraryDigital Library
  34. Joachims, T., Granka, L. A., Pan, B., Hembrooke, H., Radlinski, F., and Gay, G. 2007. Evaluating the accuracy of implicit feedback from clicks and query reformulations in Web search. ACM Trans. Inf. Syst. 25, 2. Google ScholarGoogle ScholarDigital LibraryDigital Library
  35. Kang, C., Lin, X., Wang, X., Chang, Y., and Tseng, B. 2011. Modeling perceived relevance for tail queries without click-through data. CoRR abs/1110.1112.Google ScholarGoogle Scholar
  36. Kring, A. M., Johnson, S., Davison, G. C., and Neale, J. M. 2007. Abnormal Psychology 10th Ed. Wiley.Google ScholarGoogle Scholar
  37. Lauckner, C. and Hsieh, G. 2013. The presentation of health-related search results and its impact on negative emotional outcomes. In Proceedings of the Extended Abstracts on Human Factors in Computing Systems (SIGCHI). In press. Google ScholarGoogle ScholarDigital LibraryDigital Library
  38. Mowshowitz, A. and Kawaguchi, A. 2002a. Assessing bias in search engines. Inf. Process. Manage. 38, 1, 141--156. Google ScholarGoogle ScholarDigital LibraryDigital Library
  39. Mowshowitz, A. and Kawaguchi, A. 2002b. Bias on the Web. Commun. ACM 45, 9, 56--60. Google ScholarGoogle ScholarDigital LibraryDigital Library
  40. Murata, M., Toda, H., Marsuura, Y., and Kataoka, R. 2009. Query-page intention matching using clicked titles and snippets to boost search rankings. In Proceedings of the 9th ACM/IEEE-CS Joint Conference on Digital Libraries (JCDL), 105--114. Google ScholarGoogle ScholarDigital LibraryDigital Library
  41. Radlinski, F. and Joachims, T. 2006. Minimally invasive randomization for collecting unbiased preferences from click-through logs. In Proceedings of the AAAI Conference on Artificial Intelligence (AAAI). Google ScholarGoogle ScholarDigital LibraryDigital Library
  42. Rodden, K., Fu, X., Aula, A., and Spiro, I. 2008. Eye-mouse coordination patterns on Web search results pages. In Proceedings of the ACM SIGCHI Extended Abstracts on Human Factors in Computing Systems. 2997--3002. Google ScholarGoogle ScholarDigital LibraryDigital Library
  43. Schwarz, J. and Morris, M. R. 2011. Augmenting Web pages and search results to help people find trustworthy information online. In Proceedings of the ACM SIGCHI Extended Abstracts on Human Factors in Computing Systems (SIGCHI). 1245--1254. Google ScholarGoogle ScholarDigital LibraryDigital Library
  44. Shaparenko, B., Cetin, O., and Iyer, R. 2009. Data-driven text features for sponsored search click prediction. In Proceedings of the 3rd International Workshop on Data Mining and Audience Intelligence for Advertising (ADKDD). Google ScholarGoogle ScholarDigital LibraryDigital Library
  45. Sillence, E., Briggs, P., Harris, P. R., and Fishwick, L. 2007. How do patients evaluate and make use of online health information? Social Sci. Med. 64, 9, 1853--1862.Google ScholarGoogle ScholarCross RefCross Ref
  46. Spink, A., Yang, Y., Jansen, J., Nykanen, P., Lorence, D. P., Ozmutlu, S., and Ozmutlu, H. C. 2004. A study of medical and health queries to Web search engines. Health Info. Lib. J. 21, 44--51.Google ScholarGoogle Scholar
  47. Taylor, S. and Asmundson, G. J. C. 2004. Treating Health Anxiety: A Cognitive-Behavioral Approach. Guilford Press.Google ScholarGoogle Scholar
  48. Tombros, A. and Sanderson, M. 1998. Advantages of query biased summaries in information retrieval. In Proceedings of the Annual International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR). 2--10. Google ScholarGoogle ScholarDigital LibraryDigital Library
  49. Tversky, A. and Kahneman, D. 1974. Judgment under uncertainty: Heuristics and biases. Science, 185, 4157.Google ScholarGoogle Scholar
  50. Vaughn, L. and Thelwall, M. 2004. Search engine coverage bias: Evidence and possible causes. Info. Process. Manage. 40, 4, 693--707. Google ScholarGoogle ScholarDigital LibraryDigital Library
  51. Wang, K., Gloy, N., and Li, X. 2010. Inferring search behaviors using partially observable Markov (POM) model. In Proceedings of the International ACM Conference on Web Search and Data Mining (WSDM). 211--220. Google ScholarGoogle ScholarDigital LibraryDigital Library
  52. Weber, I. and Castillo, C. 2010. The demographics of Web search. In Proceedings of the Annual International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR). 523--530. Google ScholarGoogle ScholarDigital LibraryDigital Library
  53. White, R. W. and Buscher, G. 2012. Text selections as implicit relevance feedback. In Proceedings of the Annual International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR). 1151--1152. Google ScholarGoogle ScholarDigital LibraryDigital Library
  54. White, R. W., Dumais, S. T., and Teevan, J. 2009. Characterizing the influence of domain expertise on Web search behavior. In Proceedings of the International ACM Conference on Web Search and Data Mining (WSDM). 132--141. Google ScholarGoogle ScholarDigital LibraryDigital Library
  55. White, R. W. and Horvitz, E. 2009a. Cyberchondria: Studies of the escalation of medical concerns in web search. ACM Trans. Inf. Syst. 27, 4, 23. Google ScholarGoogle ScholarDigital LibraryDigital Library
  56. White, R. W. and Horvitz, E. 2009b. Experiences with Web search on medical concerns and self-diagnosis. In Proceedings of the AMIA Annual Symposium (AMIA). 696--700.Google ScholarGoogle Scholar
  57. White, R. W. and Horvitz, E. 2012. Studies on the onset and persistence of medical concerns in search logs. In Proceedings of the Annual International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR). 265--274. Google ScholarGoogle ScholarDigital LibraryDigital Library
  58. White, R. W. and Horvitz, E. 2010. Predicting escalations of medical queries based on web page structure and content. In Proceedings of the Annual International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR). 769--770. Google ScholarGoogle ScholarDigital LibraryDigital Library
  59. White, R. W., Ruthven, I., and Jose, J. M. 2002. Finding relevant documents using top ranking sentences: An evaluation of two alternative schemes. In Proceedings of the Annual International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR). 57--64. Google ScholarGoogle ScholarDigital LibraryDigital Library
  60. Yue, Y., Patel, R., and Roehrig, H. 2010. Beyond position bias: Examining result attractiveness as a source of presentation bias in click-through data. In Proceedings of the WWW Conference, 1011--1018. Google ScholarGoogle ScholarDigital LibraryDigital Library
  61. Zhang, Y., Wang, D., Wang, G., Chen, W., Zhang, Z., Hu, B., and Zhang, L. 2010. Learning click models via probit Bayesian inference. In Proceedings of the International ACM Conference on Information and Knowledge Management (CIKM). 439--448. Google ScholarGoogle ScholarDigital LibraryDigital Library

Index Terms

  1. Captions and biases in diagnostic search

    Recommendations

    Comments

    Login options

    Check if you have access through your login credentials or your institution to get full access on this article.

    Sign in

    Full Access

    • Published in

      cover image ACM Transactions on the Web
      ACM Transactions on the Web  Volume 7, Issue 4
      October 2013
      220 pages
      ISSN:1559-1131
      EISSN:1559-114X
      DOI:10.1145/2540635
      Issue’s Table of Contents

      Copyright © 2013 ACM

      Publisher

      Association for Computing Machinery

      New York, NY, United States

      Publication History

      • Published: 1 November 2013
      • Accepted: 1 May 2013
      • Revised: 1 March 2013
      • Received: 1 November 2012
      Published in tweb Volume 7, Issue 4

      Permissions

      Request permissions about this article.

      Request Permissions

      Check for updates

      Qualifiers

      • research-article
      • Research
      • Refereed

    PDF Format

    View or Download as a PDF file.

    PDF

    eReader

    View online with eReader.

    eReader
    About Cookies On This Site

    We use cookies to ensure that we give you the best experience on our website.

    Learn more

    Got it!