skip to main content
research-article

Content Bias in Online Health Search

Published:06 November 2014Publication History
Skip Abstract Section

Abstract

Search engines help people answer consequential questions. Biases in retrieved and indexed content (e.g., skew toward erroneous outcomes that represent deviations from reality), coupled with searchers' biases in how they examine and interpret search results, can lead people to incorrect answers. In this article, we seek to better understand biases in search and retrieval, and in particular those affecting the accuracy of content in search results, including the search engine index, features used for ranking, and the formulation of search queries. Focusing on the important domain of online health search, this research broadens previous work on biases in search to examine the role of search systems in contributing to biases. To assess bias, we focus on questions about medical interventions and employ reliable ground truth data from authoritative medical sources. In the course of our study, we utilize large-scale log analysis using data from a popular Web search engine, deep probes of result lists on that search engine, and crowdsourced human judgments of search result captions and landing pages. Our findings reveal bias in results, amplifying searchers' existing biases that appear evident in their search activity. We also highlight significant bias in indexed content and show that specific ranking signals and specific query terms support bias. Both of these can degrade result accuracy and increase skewness in search results. Our analysis has implications for bias mitigation strategies in online search systems, and we offer recommendations for search providers based on our findings.

References

  1. Steven Abney, Michael Collins, and Amit Singhal. 2000. Answer extraction. In Proceedings of the Conference on Applied Natural Language. 296--301. Google ScholarGoogle ScholarDigital LibraryDigital Library
  2. Eugene Agichtein, Eric Brill, and Susan Dumais. 2006. Improving web search ranking by incorporating user behavior information. In Proceedings of the Annual International ACM SIGIR Conference on Research and Development in Information Retrieval. 19--26. Google ScholarGoogle ScholarDigital LibraryDigital Library
  3. Yin Aphinyanaphongs and Constantin Aliferis. 2007. Text categorization models for identifying unproven cancer treatments on the web. In Proceedings of the 12th World Congress on Health (Medical) Informatics. 968.Google ScholarGoogle Scholar
  4. Rawia Awadallah, Maya Ramanath, and Gerhard Weikum. 2012. Harmony and dissonance: organizing the people's voices on political controversies. In Proceedings of the ACM Conference on Web Search and Data Mining. 523--532. Google ScholarGoogle ScholarDigital LibraryDigital Library
  5. Stephanie L. Ayers and Jennie J. Kronenfeld. 2007. Chronic illness and health-seeking information on the Internet. Health 11, 3, 327--347.Google ScholarGoogle Scholar
  6. Leif Azzopardi and Ciaran Owens. 2009. Search engine predilection toward news media providers. In Proceedings of the Annual International ACM SIGIR Conference on Research and Development in Information Retrieval. 774--775. Google ScholarGoogle ScholarDigital LibraryDigital Library
  7. Leif Azzopardi and Vishwa Vinay. 2008. Retrievability: An evaluation measure for higher order information access tasks. In Proceedings of the International Conference on Information and Knowledge Management. 561--570. Google ScholarGoogle ScholarDigital LibraryDigital Library
  8. Doug Beeferman and Adam Berger. 2000. Agglomerative clustering of a search engine query log. In Proceedings of the ACM SIGKDD International Conference on Knowledge Discovery and Data Mining. 407--416. Google ScholarGoogle ScholarDigital LibraryDigital Library
  9. Nicholas J. Belkin, Colleen Cool, W. Bruce Croft, and Jamie P. Callan. 1993. The effect of multiple query representations on information retrieval system performance. In Proceedings of the Annual International ACM SIGIR Conference on Research and Development in Information Retrieval. 339--346. Google ScholarGoogle ScholarDigital LibraryDigital Library
  10. Mike Bengeri and Pierre Pluye. 2003. Shortcomings of health-related information on the internet. Health Promotion Int. 18, 4, 381--387.Google ScholarGoogle ScholarCross RefCross Ref
  11. Michael Bernstein, Jaime Teevan, Susan Dumais, Daniel Liebling, and Eric Horvitz. 2012. Direct answers for search queries in the long tail. In Proceedings of the ACM Special Interest Group on Computer-Human Interaction. 237--246. Google ScholarGoogle ScholarDigital LibraryDigital Library
  12. Krishna Bharat and Andrei Broder. 1998. A technique for measuring the relative size and overlap of public web search engines. Computer Netw. ISDN Syst. 30, 1, 379--388. Google ScholarGoogle ScholarDigital LibraryDigital Library
  13. Suresh K. Bhavnani. 2002. Domain-specific search strategies for the effective retrieval of healthcare and shopping information. In Proceedings of the ACM Special Interest Group on Computer-Human Interaction. 610--611. Google ScholarGoogle ScholarDigital LibraryDigital Library
  14. Suresh K. Bhavnani, Renju T. Jacob, Jennifer Nardine, and Frederick A. Peck. 2003. Exploring the distribution of online healthcare information. In Proceedings of the ACM Special Interest Group on Computer-Human Interaction. 816--817. Google ScholarGoogle ScholarDigital LibraryDigital Library
  15. Mikhail Bilenko and Ryen W. White. 2008. Mining the search trails of the surfing crowds: Identifying relevant websites from user activity. In Proceedings of the International World Wide Web Conference. 51--60. Google ScholarGoogle ScholarDigital LibraryDigital Library
  16. Oliver Bodenreider. 2004. The Unified Medical Language System (UMLS). Oxford University Press.Google ScholarGoogle Scholar
  17. Robert L. Brennan and Dale J. Prediger. 1981. Coefficient kappa: Some uses, misuses, and alternatives. Educ. Psych. Measurement 41, 3, 687--699.Google ScholarGoogle ScholarCross RefCross Ref
  18. Sergei Brin and Larry Page. 1998. The anatomy of a large-scale hypertextual search engine. In Proceedings of the International World Wide Web Conference. Google ScholarGoogle ScholarDigital LibraryDigital Library
  19. Olivier Chapelle, Thorsten Joachims, Filip Radlinski, and Yisong Yue. 2012. Large scale validation and analysis of interleaved search evaluation. ACM Trans. Inf. Syst. 30, 1. Google ScholarGoogle ScholarDigital LibraryDigital Library
  20. Lydia Chilton and Jaime Teevan. 2011. Addressing people's information needs directly in a web search result page. In Proceedings of the International World Wide Web Conference. 27--36. Google ScholarGoogle ScholarDigital LibraryDigital Library
  21. Junghoo Cho and Sourashis Roy. 2004. Impact of search engines on page popularity. In Proceedings of the International World Wide Web Conference. 20--29. Google ScholarGoogle ScholarDigital LibraryDigital Library
  22. Charles Clarke, Eugene Agichtein, Susan Dumais, and Ryen W. White. 2007. The influence of caption features on clickthrough patterns in Web search. In Proceedings of the Annual International ACM SIGIR Conference on Research and Development in Information Retrieval, 135--142. Google ScholarGoogle ScholarDigital LibraryDigital Library
  23. Rebecca J. W. Cline and Katie M. Haynes. 2001. Consumer health information seeking on the internet: The state of the art. Health Educ. Res. 16, 6, 671--692.Google ScholarGoogle ScholarCross RefCross Ref
  24. James Cogley, Nicola Stokes, and Joe Carthy. 2013. Exploring the effectiveness of medical entity recognition for clinical information retrieval. In Proceedings of the 7th International Workshop on Data and Text Mining in Biomedical Informatics, 3--4. Google ScholarGoogle ScholarDigital LibraryDigital Library
  25. Kevyn Collins-Thompson, Jamie Callan, Egidio Terra, and Charles L. A. Clarke. 2004. The effect of document retrieval quality on factoid question answering performance. In Proceedings of the Annual International ACM SIGIR Conference on Research and Development in Information Retrieval, 574--575. Google ScholarGoogle ScholarDigital LibraryDigital Library
  26. Nick Craswell, David Hawking, Ross Wilkinson, and Mingfang Wu. 2003. Overview of the TREC 2003 Web Track. In Proceedings of the Text Retrieval Conference.Google ScholarGoogle Scholar
  27. Nick Craswell, Onno Zoeter, Michael Taylor, and Bill Ramsey. 2008. An experimental comparison of click position-bias models. In Proceedings of the ACM Conference on Web Search and Data Mining. 87--94. Google ScholarGoogle ScholarDigital LibraryDigital Library
  28. Anirban Dasgupta, Arpita Gosh, Ravi Kumar, Christopher Olston, Sandeep Pandey, and Andrew Tomkins. 2007. The discoverability of the web. In Proceedings of the International World Wide Web Conference. 421--430. Google ScholarGoogle ScholarDigital LibraryDigital Library
  29. Kay Dickersin. 1990. The existence of publication bias and risk factors for its occurrence. J. Amer. Medical Assoc. 263, 10, 1385--1389.Google ScholarGoogle ScholarCross RefCross Ref
  30. Shiri Dori-Hacohen and James Allan. 2013. Detecting controversy on the web. In Proceedings of the International Conference on Information and Knowledge Management. 1845--1848. Google ScholarGoogle ScholarDigital LibraryDigital Library
  31. Susan Dumais, Michele Banko, Eric Brill, Jimmy Lin, and Andrew Ng. 2002. Web question answering: Is more always better? In Proceedings of the Annual International ACM SIGIR Conference on Research and Development in Information Retrieval. 291--298. Google ScholarGoogle ScholarDigital LibraryDigital Library
  32. Phillipa J. Easterbrook, R. Gopalan, J. A. Berlin, and David R. Matthews. 1991. Publication bias in clinical research. Lancet 337, 8746, 867--872.Google ScholarGoogle Scholar
  33. Matthias Egger, George Davey Smith, Martin Schneider, and Christoph Minder. 1997. Bias in meta-analysis detected by a simple, graphical test. British Med. J. 315, 7109, 629--634.Google ScholarGoogle ScholarCross RefCross Ref
  34. Rob Ennals, Beth Trushkowsky, and John M. Agosta. 2010. Highlighting disputed claims on the web. In Proceedings of the International World Wide Web Conference. 341--350. Google ScholarGoogle ScholarDigital LibraryDigital Library
  35. Gunther Eysenbach and Christian Köhler. 2002. How do consumers search for and appraise health information on the World Wide Web? Qualitative studies using focus groups, usability test, and in-depth interviews. British Med. J. 324, 573--577Google ScholarGoogle ScholarCross RefCross Ref
  36. Daniele Fanelli. 2012. Negative results are disappearing from most disciplines and countries. Scientometrics 90, 3, 891--901. Google ScholarGoogle ScholarDigital LibraryDigital Library
  37. Joseph L. Fleiss. 1971. Measuring nominal scale agreement among many raters. Psych. Bull. 76, 5, 378--382.Google ScholarGoogle ScholarCross RefCross Ref
  38. Santo Fortunato, Alessandro Flammini, Fillipo Menczer, and Alessandro Vespignani. 2006. Topical interests and the mitigation of search engine bias. Proc. Nat. Acad. Sci. U.S.A. 103, 34, 12684--12689.Google ScholarGoogle ScholarCross RefCross Ref
  39. Susannah Fox. 2006. Online Health Search 2006. Pew Internet and American Life Project.Google ScholarGoogle Scholar
  40. Susannah Fox and Maeve Duggan. 2013. Health Topics. Pew Internet and American Life Project.Google ScholarGoogle Scholar
  41. Steve Fox, Kuldeep Karnawat, Mark Mydland, Susan Dumais, and Thomas White. 2005. Evaluating implicit measures to improve Web search. ACM Trans. Inf. Syst. 23, 2, 147--168. Google ScholarGoogle ScholarDigital LibraryDigital Library
  42. Arnaud Gaudinat, Patrick Ruch, Michel Joubert, Phillipe Uziel, P., Anne Strauss, Michèle Thonnet, Robert Baud, Stéphane Spahni, Patrick Weber, Juan Bonai, Celia Boyer, Marius Fieschi, and Antoine Geissbuhler. 2006. Health search engine with e-document analysis for reliable search results. Int. J. Med. 75, 1, 73--85.Google ScholarGoogle Scholar
  43. Susan Gerhart. 2004. Do Web search engines suppress controversy? First Monday 9, 1--5.Google ScholarGoogle Scholar
  44. Jeremy Ginsberg, Matthew H. Mohebbi, Rajan S. Patel, Lynnette Brammer, Mark S. Smolinksi, and Larry Brilliant. 2008. Detecting influenza epidemics using search engine query data. Nature 457, 7232, 1012--1014.Google ScholarGoogle Scholar
  45. Eric Goldman. 2006. Search engine bias and the demise of search utopianism. Yale J. Law Technol.188.Google ScholarGoogle Scholar
  46. Qi Guo and Eugene Agichtein. 2012. Beyond dwell time: Estimating document relevance from cursor movements and other post-click searcher behavior. In Proceedings of the International World Wide Web Conference. 569--578. Google ScholarGoogle ScholarDigital LibraryDigital Library
  47. Donna Harman and Chris Buckley. 2009. Overview of the reliable information access workshop. Inf. Retrieval 12, 614--641. Google ScholarGoogle ScholarDigital LibraryDigital Library
  48. William Hart, Dolores Albarracín, Alice H. Eagly, Inge Brechan, Matthew J. Lindberg, and Lisa Merrill. 2009. Feeling validated versus being correct: A meta-analysis of selective exposure to information. Psych. Bull. 135, 4, 555--588.Google ScholarGoogle ScholarCross RefCross Ref
  49. William Hersh. 2008. Information Retrieval: A Health and Biomedical Perspective. Springer. William R. Hersh and David H. Hickam. 1998. How well do physicians use electronic information retrieval systems? A frame work for investigation and systematic review. J. Amer. Medical Informatics Assoc. 280, 1347.Google ScholarGoogle ScholarCross RefCross Ref
  50. William R. Hersh, Katherine Crabtree, David H. Hickam, Lynetta Sacherek, Charles P. Friedman, Patricia Tidmarsh, Craig Mosbaek, and Dale Kraemer. 2002. Factors associated with success in searching MEDLINE and applying evidence to answer clinical questions. J. Amer. Medical Informatics Assoc. 9, 283--293.Google ScholarGoogle ScholarCross RefCross Ref
  51. Julian P. T. Higgins (Ed.). 2008. Cochrane Handbook for Systematic Reviews of Interventions (Vol. 5). Wiley-Blackwell, Chichester, UK.Google ScholarGoogle Scholar
  52. Samuel Ieong, Nina Mishra, Eldar Sadikov, and Li Zhang. 2012. Domain bias in Web search. In Proceedings of the ACM Conference on Web Search and Data Mining. 413--422. Google ScholarGoogle ScholarDigital LibraryDigital Library
  53. Peter Ingwersen. 1994. Polyrepresentation of information needs and semantic entities: Elements of a cognitive theory of information retrieval interaction. In Proceedings of the Annual International ACM SIGIR Conference on Research and Development in Information Retrieval. 101--110. Google ScholarGoogle ScholarDigital LibraryDigital Library
  54. Charles B. Inlander. 1993. Good Operations, Bad Operations. The People's Medical Society's Guide to Surgery. Viking Adult.Google ScholarGoogle Scholar
  55. Alejandro R. Jadad, Deborah J. Cook, Alison Jones, Terry P. Klassen, Peter Tugwell, Michael Moher, and David Moher. 1998. Methodology and reports of systematic reviews and meta-analyses: a comparison of Cochrane reviews with articles published in paper-based journals. J. Amer. Medical Assoc. 280, 3, 278--280.Google ScholarGoogle ScholarCross RefCross Ref
  56. Thorsten Joachims. 2002. Optimizing search engines using click-through data. In Proceedings of the ACM SIGKDD International Conference on Knowledge Discovery and Data Mining. 132--142. Google ScholarGoogle ScholarDigital LibraryDigital Library
  57. Thorsten Joachims, Laura Granka, Bing Pan, Helene Hembrooke, Filip Radlinski, and Geri Gay. 2007. Evaluating the accuracy of implicit feedback from clicks and query reformulations in web search. ACM Trans. Inf. Syst. 25, 2. Google ScholarGoogle ScholarDigital LibraryDigital Library
  58. Anders W. Jørgensen, Jørgen Hilden, and Peter C. Gøtzsche. 2006. Cochrane reviews compared with industry supported meta-analyses and other meta-analyses of the same drugs: systematic review. British Med. J. 333, 7572, 782.Google ScholarGoogle Scholar
  59. Mouna Kacimi and Johann Gamper. 2011. Diversifying search results of controversial queries. In Proceedings of the International Conference on Information and Knowledge Management. 93--98. Google ScholarGoogle ScholarDigital LibraryDigital Library
  60. Mouna Kacimi and Johann Gamper. 2012. MOUNA: mining opinions to unveil neglected arguments. In Proceedings of the International Conference on Information and Knowledge Management. 2722--2724. Google ScholarGoogle ScholarDigital LibraryDigital Library
  61. Diane Kelly, Xiao-jun Yuan, Nicholas J. Belkin, Vanessa Murdock, and W. Bruce Croft. 2002. Features of documents relevant to task- and fact-oriented questions. In Proceedings of the International Conference on Information and Knowledge Management. 645--647. Google ScholarGoogle ScholarDigital LibraryDigital Library
  62. Jon Kleinberg. 1999. Authoritative sources in a hyperlinked environment. J. ACM 46, 5, 604--632. Google ScholarGoogle ScholarDigital LibraryDigital Library
  63. Bevan Koopman, Guido Zuccon, Peter Bruza, Laurianne Sitbon, and Michael Lawley. 2012. An evaluation of corpus-driven measures of medical concept similarity for information retrieval. In Proceedings of the International Conference on Information and Knowledge Management. 2439--2442. Google ScholarGoogle ScholarDigital LibraryDigital Library
  64. Carolyn Lauckner and Gary Hsieh. 2013. The presentation of health-related search results and its impact on negative emotional outcomes. In Proceedings of the ACM Special Interest Group on Computer-Human Interaction. 333--342. Google ScholarGoogle ScholarDigital LibraryDigital Library
  65. Hady W. Lauw, Ee-Peng Lim, and Ke Wang. 2006. Bias and controversy: Beyond the statistical deviation. In Proceedings of the ACM SIGKDD International Conference on Knowledge Discovery and Data Mining. 625--630. Google ScholarGoogle ScholarDigital LibraryDigital Library
  66. Steve Lawrence and C. Lee Giles. 1999. Accessibility of information on the web. Nature 400, 6740, 107.Google ScholarGoogle ScholarCross RefCross Ref
  67. Nut Limsopatham, Craig Macdonald, Richard McCreadie, and Iadh Ounis. 2012. Exploiting term dependence while handling negation in medical search. In Proceedings of the Annual International ACM SIGIR Conference on Research and Development in Information Retrieval. 1065--1066. Google ScholarGoogle ScholarDigital LibraryDigital Library
  68. Abbe Mowshowitz and Akira Kawaguchi. 2002a. Assessing bias in search engines. Inf. Process. Manage. 38, 1, 141--156. Google ScholarGoogle ScholarDigital LibraryDigital Library
  69. Abbe Mowshowitz and Akira Kawaguchi. 2002b. Bias on the Web. Commun. ACM 45, 9, 56--60. Google ScholarGoogle ScholarDigital LibraryDigital Library
  70. Yishai Ofran, Ora Paltiel, Dan Pelleg, Jacob M. Rowe, and Elad Yom-Tov. 2012. Patterns of information-seeking for cancer on the internet: An analysis of real world data. PloS One 7, 9, e45921.Google ScholarGoogle ScholarCross RefCross Ref
  71. Bo Pang and Ravi Kumar. 2011. Search in the lost sense of “query”: Question formulation in Web search queries and its temporal changes. In Proceedings of the Annual Meeting of the Association for Computational Linguistics. 135--140. Google ScholarGoogle ScholarDigital LibraryDigital Library
  72. Sandeep Pandey, Kedar Dhamdhere, and Christopher Olston. 2004. Wic: A general-purpose algorithm for monitoring web information sources. In Proceedings of the International Conference on Very Large Databases. 360--371. Google ScholarGoogle ScholarDigital LibraryDigital Library
  73. Eli Pariser. 2011. The Filter Bubble: What is the Internet Hiding from You? Penguin Press. Google ScholarGoogle ScholarDigital LibraryDigital Library
  74. Mark Petticrew, Paul Wilson, Kath Wright, and Fujian Song. 2002. Quality of Cochrane reviews: Quality of Cochrane reviews is better than that of non-Cochrane reviews. British Med. J. 324, 7336, 545.Google ScholarGoogle ScholarCross RefCross Ref
  75. Mary C. Politi, Paul K. J. Han, and Nananda F. Col. 2007. Communicating the uncertainty of harms and benefits of medical interventions. Medical Decision Making 27, 5, 681--695.Google ScholarGoogle ScholarCross RefCross Ref
  76. Filip Radlinski and Thorsten Joachims. 2006. Minimally invasive randomization for collecting unbiased preferences from click-through logs. In Proceedings of the National Conference on Artificial Intelligence. Google ScholarGoogle ScholarDigital LibraryDigital Library
  77. Matthew Richardson, Amit Prakash, and Eric Brill. 2006. Beyond PageRank: Machine learning for static ranking. In Proceedings of the International World Wide Web Conference. 707--715. Google ScholarGoogle ScholarDigital LibraryDigital Library
  78. Bonnie Rochman. 2011. Jenny McCarthy, vaccine expert? A quarter of parents trust celebrities. Time (26 April 2011).Google ScholarGoogle Scholar
  79. Ian Ruthven. 2003. Re-examining the potential effectiveness of interactive query expansion. In Proceedings of the Annual International ACM SIGIR Conference on Research and Development in Information Retrieval. 213--220. Google ScholarGoogle ScholarDigital LibraryDigital Library
  80. David L. Sackett, William Rosenberg, J. A. Gray, R. Brian Haynes, and W. Scott Richardson. 1996. Evidence based medicine: What it is and what it isn't. British Med. J. 312, 7023: 71.Google ScholarGoogle ScholarCross RefCross Ref
  81. Julia Schwarz and Meredith Ringel Morris. 2011. Augmenting web pages and search results to help people find trustworthy information online. In Proceedings of the ACM Special Interest Group on Computer-Human Interaction. 1245--1254. Google ScholarGoogle ScholarDigital LibraryDigital Library
  82. Elizabeth Sillence, Pam Briggs, Lesley Fishwick, and Peter Harris. 2004. Trust and mistrust of online health sites. In Proceedings of the ACM Special Interest Group on Computer-Human Interaction. 663--670. Google ScholarGoogle ScholarDigital LibraryDigital Library
  83. Herbert A. Simon. 1991. Bounded rationality and organizational learning. Organization Sci. 2, 1, 125--134. Google ScholarGoogle ScholarDigital LibraryDigital Library
  84. Amit Singhal, Chris Buckley, and Mandar Mitra. 2006. Pivoted document length normalization. In Proceedings of the Annual International ACM SIGIR Conference on Research and Development in Information Retrieval. 21--29. Google ScholarGoogle ScholarDigital LibraryDigital Library
  85. Karen Spärck-Jones, Steve Walker, and Stephen E. Robertson. 2000. A probabilistic model of information retrieval: development and comparative experiments. Inf. Process. Manage. 36, 6, 779--840. Google ScholarGoogle ScholarDigital LibraryDigital Library
  86. Amanda Spink, Yin Yang, Jim Jansen, Pirrko Nykanen, Daniel P. Lorence, Seda Ozmutlu, and H. Cenk Ozmutlu. 2004. A study of medical and health queries to Web search engines. Health Inf, Libraries J. 21, 44--51.Google ScholarGoogle ScholarCross RefCross Ref
  87. Thanh Tin Tang, David Hawking, Nick Craswell, and Kathy Griffiths. 2005. Focused crawling for both topical relevance and quality of medical information. In Proceedings of the International Conference on Information and Knowledge Management. 147--154. Google ScholarGoogle ScholarDigital LibraryDigital Library
  88. Robert S. Taylor. 1968. Question-negotiation and information seeking in libraries. College Research Libraries 29, 178--194.Google ScholarGoogle ScholarCross RefCross Ref
  89. Amos Tversky and Daniel Kahneman. 1973. Availability: A heuristic for judging frequency and probability. Cognitive Psych. 5, 1, 207--233.Google ScholarGoogle ScholarCross RefCross Ref
  90. Tristan Upstill, Nick Craswell, and David Hawking. 2002. Buying bestsellers online: A case study in search and searchability. In Proceedings of the 7th Australasian Document Computing Symposium.Google ScholarGoogle Scholar
  91. Liwen Vaughn and Mike Thelwall. 2004. Search engine coverage bias: Evidence and possible causes. Inf. Process. Manage. 40, 4, 693--707. Google ScholarGoogle ScholarDigital LibraryDigital Library
  92. V. G. Vydiswaran, Cheng Xiang Zhai, Dan Roth, and Peter Pirolli. 2012. BiasTrust: Teaching biased users about controversial topics. In Proceedings of the International Conference on Information and Knowledge Management. 1905--1909. Google ScholarGoogle ScholarDigital LibraryDigital Library
  93. Peter C. Wason. 1960. On the failure to eliminate hypotheses in a conceptual task. Quart. J. Exper. Psych. 12, 129--140.Google ScholarGoogle ScholarCross RefCross Ref
  94. Robert West, Ryen W. White, and Eric Horvitz. 2013. From cookies to cooks: Insights on dietary patterns via analysis of web usage logs. In Proceedings of the International World Wide Web Conference. 1399--1410. Google ScholarGoogle ScholarDigital LibraryDigital Library
  95. Ryen W. White. 2013. Beliefs and biases in Web search. In Proceedings of the Annual International ACM SIGIR Conference on Research and Development in Information Retrieval. 3--10. Google ScholarGoogle ScholarDigital LibraryDigital Library
  96. Ryen W. White, Rave Harpaz, Nigam H. Shah, William DuMouchel, and Eric Horvitz. 2014. Toward enhanced pharmacovigilance using patient-generated data on the Internet. In Nature Clinical Pharmacology and Therapeutics. (To appear)Google ScholarGoogle Scholar
  97. Ryen W. White and Eric Horvitz. 2009a. Cyberchondria: Studies of the escalation of medical concerns in web search. ACM Trans. Inf. Syst. 27, 4, 23. Google ScholarGoogle ScholarDigital LibraryDigital Library
  98. Ryen W. White and Eric Horvitz. 2009b. Experiences with web search on medical concerns and self-diagnosis. In Proceedings of the American Medical Information Association Annual Symposium. 696--700.Google ScholarGoogle Scholar
  99. Ryen W. White and Eric Horvitz. 2010a. Web to World: Predicting transitions from self-diagnosis to the pursuit of local medical assistance in web search. In Proceedings of the American Medical Information Association Annual Symposium. 882--886.Google ScholarGoogle Scholar
  100. Ryen W. White and Eric Horvitz. 2010b. Predicting escalations of medical queries based on web page structure and content. In Proceedings of the Annual International ACM SIGIR Conference on Research and Development in Information Retrieval. 769--770. Google ScholarGoogle ScholarDigital LibraryDigital Library
  101. Ryen W. White and Eric Horvitz. 2013 Captions and biases in diagnostic search. ACM Trans. Web 7, 4, 23. Google ScholarGoogle ScholarDigital LibraryDigital Library
  102. Ryen W. White, Nicholas P. Tatonetti, Nigam H. Shah, Russ B. Altman, and Eric Horvitz. 2013. Web-scale pharmacovigilance: Listening to signals from the crowd. J. Amer. Med. Informatics Assoc. 20, 3, 404--408.Google ScholarGoogle ScholarCross RefCross Ref
  103. Barbara M. Wildemuth. 2004. The effects of domain knowledge on search tactic information. J. Assoc. Inf. Sci. Tech. 55, 3, 246--258. Google ScholarGoogle ScholarDigital LibraryDigital Library
  104. Colin Wilkie and Leif Azzopardi. 2014. Best and fairest: An empirical analysis of retrieval system bias. In Proceedings of the European Conference on Information Retrieval, 13--25.Google ScholarGoogle ScholarCross RefCross Ref
  105. Janice C. Wright and Milton C. Weinstein. 1998. Gains in life expectancy from medical interventions: Standardizing data on outcomes. New England J. Med. 339, 6, 380--386.Google ScholarGoogle ScholarCross RefCross Ref
  106. Elad Yom-Tov, Susan Dumais, and Qi Guo. 2014. Promoting civil discourse through search engine diversity. Social Science Computer Rev. 32, 2, 145--154. Google ScholarGoogle ScholarDigital LibraryDigital Library
  107. Elad Yom-Tov and Evgeniy Gabrilovich. 2013. Post-market drug surveillance without trial costs: Discovery of adverse drug reactions through large-scale analysis of web search queries. J. Med. Internet Res. 15, 6.Google ScholarGoogle ScholarCross RefCross Ref
  108. Yisong Yue, Rajan Patel, and Hein Roehrig. 2010. Beyond position bias: Examining result attractiveness as a source of presentation bias in clickthrough data. In Proceedings of the International World Wide Web Conference. 1011--1018. Google ScholarGoogle ScholarDigital LibraryDigital Library
  109. Cheng Xiang Zhai, William W. Cohen, and John Lafferty. 2003. Beyond independent relevance: Methods and evaluation metrics for subtopic retrieval. In Proceedings of the Annual International ACM SIGIR Conference on Research and Development in Information Retrieval. 10--17. Google ScholarGoogle ScholarDigital LibraryDigital Library

Index Terms

  1. Content Bias in Online Health Search

        Recommendations

        Comments

        Login options

        Check if you have access through your login credentials or your institution to get full access on this article.

        Sign in

        Full Access

        • Published in

          cover image ACM Transactions on the Web
          ACM Transactions on the Web  Volume 8, Issue 4
          October 2014
          178 pages
          ISSN:1559-1131
          EISSN:1559-114X
          DOI:10.1145/2686863
          Issue’s Table of Contents

          Copyright © 2014 ACM

          Publisher

          Association for Computing Machinery

          New York, NY, United States

          Publication History

          • Published: 6 November 2014
          • Revised: 1 August 2014
          • Accepted: 1 August 2014
          • Received: 1 April 2014
          Published in tweb Volume 8, Issue 4

          Permissions

          Request permissions about this article.

          Request Permissions

          Check for updates

          Qualifiers

          • research-article
          • Research
          • Refereed

        PDF Format

        View or Download as a PDF file.

        PDF

        eReader

        View online with eReader.

        eReader
        About Cookies On This Site

        We use cookies to ensure that we give you the best experience on our website.

        Learn more

        Got it!