Is seeing believing?: how recommender system interfaces affect users' opinions

ABSTRACT

Recommender systems use people's opinions about items in an information domain to help people choose other items. These systems have succeeded in domains as diverse as movies, news articles, Web pages, and wines. The psychological literature on conformity suggests that in the course of helping people make choices, these systems probably affect users' opinions of the items. If opinions are influenced by recommendations, they might be less valuable for making recommendations for other users. Further, manipulators who seek to make the system generate artificially high or low recommendations might benefit if their efforts influence users to change the opinions they contribute to the recommender. We study two aspects of recommender system interfaces that may affect users' opinions: the rating scale and the display of predictions at the time users rate items. We find that users rate fairly consistently across rating scales. Users can be manipulated, though, tending to rate toward the prediction the system shows, whether the prediction is accurate or not. However, users can detect systems that manipulate predictions. We discuss how designers of recommender systems might react to these findings.

References

1. T. Amoo and H. H. Friedman. Do numeric values influence subjects' responses to rating scales? Journal of International Marketing and Marketing Research, 26:41--46, February 2001.
2. S. E. Asch. Effects of group pressure upon the modification and distortion of judgements. In Groups, Leadership, and Men, pages 177--190, 1951.
3. J. S. Breese, D. Heckerman, and C. Kadie. Empirical analysis of predictive algorithms for collaborative filtering. In Proc. UAI '98, pages 43--52, July 1998.
4. C. Dellarocas. Immunizing online reputation reporting systems against unfair ratings and discriminatory behavior. In Proc. 2nd ACM Conference on Electronic Commerce, pages 150--157, Minneapolis, 2000.
5. P. Domingos and M. Richardson. Mining the network value of customers. In Proc. SIGKDD 2001., pages 57--66, San Francisco, 2001. ACM Press.
6. H. H. Friedman and T. Amoo. Rating the rating scales. Journal of Marketing Management, 9(3):114--123, 1999.
7. R. Garland. The midpoint on a rating scale: Is it desirable? Marketing Bulletin, 2:66--70, 1991.
8. K. Goldberg, T. Roeder, D. Gupta, and C. Perkins. Eigentaste: A constant time collaborative filtering algorithm. Information Retrieval, 4(2):133--151, 2001.
9. J. L. Herlocker, J. A. Konstan, and J. Riedl. Explaining collaborative filtering recommendations. In Proc. CSCW2000, pages 241--250, 2000.
10. W. Hill, L. Stead, M. Rosenstein, and G. Furnas. Recommending and evaluating choices in a virtual community of use. In Proc. CHI95, pages 194--201, 1995.
11. C. Nass and Y. Moon. Machines and mindlessness: Social responses to computers. Journal of Social Issues, 60(1):81--103, 2000.
12. D. M. Pennock, E. Horvitz, S. Lawrence, and C. L. Giles. Collaborative filtering by personality diagnosis: A hybrid memory and modelbased approach. In Proc. UAI'00, pages 473--480, San Francisco, July 2000.
13. P. Resnick, N. Iacovou, M. Suchak, P. Bergstrom, and J. Riedl. GroupLens: An open architecture for collaborative filtering of netnews. In Proc. CSCW94, pages 175--186, 1994.
14. B. M. Sarwar, G. Karypis, J. A. Konstan, and J. Reidl. Item based collaborative filtering recommendation algorithms. In WWW10, pages 285--295, 2001.
15. U. Shardanand and P. Maes. Social information filtering: Algorithms for automating "word of mouth". In Proc. CHI95, pages 210--217, 1995.
16. K. Swearingen and R. Sinha. Interaction design for recommender systems. In Designing Interactive Systems (DIS2002), London, June 25--28 2002.
17. A. H. Turpin and W. Hersh. Why batch and user evaluations do not give the same results. In Proc. SIGIR2001, pages 225--231, New Orleans, Sept. 9--13 2001.

Index Terms

• Computing methodologies

  • Artificial intelligence

    • Philosophical/theoretical foundations of artificial intelligence

      • Cognitive science

• Human-centered computing

  • Human computer interaction (HCI)

    • Empirical studies in HCI

    • HCI design and evaluation methods

• Information systems

  • Information retrieval

    • Retrieval tasks and goals

      • Document filtering

      • Information extraction