Abstract
Prior research suggests that public negative feedback on social knowledge sharing platforms can be powerfully demotivating to newcomers, particularly when it involves peer feedback mechanisms such as ratings and commenting systems. What is the impact on newcomer retention when feedback is private, and from a single peer reviewer? We study these effects using the example of the Humanitarian OpenStreetMap Team, a Wikipedia-style social mapping platform where the review process is closer to a teacher-learner model rather than a public peer review. We observe peer feedback for early contributions by 1,300 newcomers, and assess the impact of different classes of feedback, including performance feedback, corrective feedback, and verbal rewards. We find that verbal rewards and immediate feedback can have a powerful effect on newcomer retention. In order to better support such positive engagement effects, we recommend that system designers conceptually distinguish between mechanisms for quality control and for learner feedback.
- Panagiotis Adamopoulos. 2013. What makes a great MOOC? An interdisciplinary analysis of student retention in online courses ICIS.Google Scholar
- June Ahn, Brian S. Butler, Cindy Weng, and Sarah Webster. 2013. Learning to be a better Q'er in social Q&A sites: social norms and information artifacts. Proceedings of the American Society for Information Science and Technology, Vol. 50, 1 (2013), 1--10. Google Scholar
Digital Library
- Blake K Ashforth and Alan M Saks. 1996. Socialization tactics: Longitudinal effects on newcomer adjustment. Academy of management Journal Vol. 39, 1 (1996), 149--178.Google Scholar
- Jonathan Bishop. 2007. Increasing participation in online communities: A framework for human-computer interaction. Computers in human behavior Vol. 23, 4 (2007), 1881--1893. Google Scholar
Digital Library
- Joel Brockner, William R Derr, and Wesley N Laing. 1987. Self-esteem and reactions to negative feedback: Toward greater generalizability. Journal of Research in Personality Vol. 21, 3 (1987), 318--333.Google Scholar
Cross Ref
- Judy Cameron, Katherine M Banko, and W David Pierce. 2001. Pervasive negative effects of rewards on intrinsic motivation: The myth continues. The Behavior Analyst, Vol. 24, 1 (2001), 1.Google Scholar
Cross Ref
- Justin Cheng, Cristian Danescu-Niculescu-Mizil, and Jure Leskovec. 2014. How Community Feedback Shapes User Behavior. In Proc. ICWSM '14.Google Scholar
- Boreum Choi, Kira Alexander, Robert E Kraut, and John M Levine. 2010. Socialization tactics in Wikipedia and their effects Proc. CSCW '10. ACM, 107--116. Google Scholar
Digital Library
- Edward L Deci, Richard Koestner, and Richard M Ryan. 1999. A meta-analytic review of experiments examining the effects of extrinsic rewards on intrinsic motivation. Psychological bulletin Vol. 125, 6 (1999), 627.Google Scholar
- Chrysanthos Dellarocas. 2003. The digitization of word of mouth: Promise and challenges of online feedback mechanisms. Management science, Vol. 49, 10 (2003), 1407--1424. Google Scholar
Digital Library
- Martin Dittus, Giovanni Quattrone, and Licia Capra. 2016. Analysing volunteer engagement in humanitarian mapping: building contributor communities at large scale. In Proc. CSCW '16. Google Scholar
Digital Library
- Martin Dittus, Giovanni Quattrone, and Licia Capra. 2017. Mass participation during emergency response: event-centric crowd-sourcing in humanitarian mapping. In Proc. CSCW '17. Google Scholar
Digital Library
- Melanie Eckle and Jo ao Porto de Albuquerque. 2015. Quality assessment of remote mapping in OpenStreetMap for disaster management purposes Proc. ISCRAM '15.Google Scholar
- Laurence S Freedman. 1982. Tables of the number of patients required in clinical trials using the logrank test. Statistics in medicine Vol. 1, 2 (1982), 121--129.Google Scholar
- Aaron Halfaker, Aniket Kittur, and John Riedl. 2011. Don't bite the newbies: how reverts affect the quantity and quality of Wikipedia work Proc. WikiSym '11. ACM, 163--172. Google Scholar
Digital Library
- Judith M Harackiewicz. 1979. The effects of reward contingency and performance feedback on intrinsic motivation. Journal of personality and social psychology, Vol. 37, 8 (1979), 1352.Google Scholar
Cross Ref
- John Hattie and Helen Timperley. 2007. The power of feedback. Review of educational research Vol. 77, 1 (2007), 81--112.Google Scholar
- Aniket Kittur, Bryan Pendleton, and Robert E Kraut. 2009. Herding the cats: the influence of groups in coordinating peer production Proc. WikiSym '09. ACM, 7. Google Scholar
Digital Library
- Robert E Kraut, Paul Resnick, Sara Kiesler, Moira Burke, Yan Chen, Niki Kittur, Joseph Konstan, Yuqing Ren, and John Riedl. 2012. Building successful online communities: Evidence-based social design. Mit Press. Google Scholar
Digital Library
- J Richard Landis and Gary G Koch. 1977. The measurement of observer agreement for categorical data. biometrics (1977), 159--174.Google Scholar
- Jingyan Lu and Nancy Law. 2012. Online peer assessment: effects of cognitive and affective feedback. Instructional Science Vol. 40, 2 (2012), 257--275.Google Scholar
Cross Ref
- Richard S Lysakowski and Herbert J Walberg. 1982. Instructional effects of cues, participation, and corrective feedback: A quantitative synthesis. American Educational Research Journal Vol. 19, 4 (1982), 559--572.Google Scholar
Cross Ref
- Loizos Michael and Jahna Otterbacher. 2014. Write Like I Write: Herding in the Language of Online Reviews. Proc. ICWSM '14.Google Scholar
- Richard L Moreland and Paul D Sweeney. 1984. Self-expectancies and reactions to evaluations of personal performance. Journal of Personality Vol. 52, 2 (1984), 156--176.Google Scholar
Cross Ref
- Jonathan T Morgan, Siko Bouterse, Heather Walls, and Sarah Stierch. 2013. Tea and sympathy: crafting positive new user experiences on wikipedia Proc. CSCW '13. ACM, 839--848. Google Scholar
Digital Library
- David R Musicant, Yuqing Ren, James A Johnson, and John Riedl. 2011. Mentoring in Wikipedia: a clash of cultures. In Proc. WikiSym '11. ACM, 173--182. Google Scholar
Digital Library
- Leysia Palen, Robert Soden, T Jennings Anderson, and Mario Barrenechea. 2015. Success & scale in a data-producing organization: the socio-technical evolution of OpenStreetMap in response to humanitarian events Proc. SIGCHI '15. 4113--4122. Google Scholar
Digital Library
- Katherine Panciera, Aaron Halfaker, and Loren Terveen. 2009. Wikipedians are born, not made: a study of power editors on Wikipedia Proc. GROUP '09. ACM, 51--60. Google Scholar
Digital Library
- Laurence D Parnell, Pierre Lindenbaum, Khader Shameer, Giovanni Marco Dall'Olio, Daniel C Swan, Lars Juhl Jensen, Simon J Cockell, Brent S Pedersen, Mary E Mangan, Christopher A Miller, et almbox. 2011. BioStar: an online question & answer resource for the bioinformatics community. PLoS Comput Biol, Vol. 7, 10 (2011), e1002216.Google Scholar
Cross Ref
- Paul Resnick, Ko Kuwabara, Richard Zeckhauser, and Eric Friedman. 2000. Reputation systems. Commun. ACM Vol. 43, 12 (2000), 45--48. Google Scholar
Digital Library
- Robert Soden and Leysia Palen. 2014. From crowdsourced mapping to community mapping: the post-earthquake work of OpenStreetMap Haiti. (2014).Google Scholar
- Yla R Tausczik and James W Pennebaker. 2012. Participation in an online mathematics community: differentiating motivations to add Proc. CSCW '12. ACM, 207--216. Google Scholar
Digital Library
- Gershon Tenenbaum and Ellen Goldring. 1989. A meta-analysis of the effect of enhanced instruction: Cues, participation, reinforcement and feedback and correctives on motor skill learning. Journal of Research & Development in Education (1989).Google Scholar
- Clay Westrope, Robert Banick, and Mitch Levine. 2014. Groundtruthing OpenStreetMap building damage assessment. Procedia engineering Vol. 78 (2014), 29--39.Google Scholar
- Donghee Yvette Wohn. 2015. The Effects of Feedback and Habit on Content Posting in an Online Community. iConference 2015 Proceedings (2015).Google Scholar
- Haiyi Zhu, Amy Zhang, Jiping He, Robert E Kraut, and Aniket Kittur. 2013. Effects of peer feedback on contribution: a field experiment in Wikipedia Proc. SIGCHI '13. ACM, 2253--2262. Google Scholar
Digital Library
Index Terms
Private Peer Feedback as Engagement Driver in Humanitarian Mapping
Recommendations
Analysing Volunteer Engagement in Humanitarian Mapping: Building Contributor Communities at Large Scale
CSCW '16: Proceedings of the 19th ACM Conference on Computer-Supported Cooperative Work & Social ComputingOrganisers of large-scale crowdsourcing initiatives need to consider how to produce outcomes with their projects, but also how to build volunteer capacity. The initial project experience of contributors plays an important role in this, particularly when ...
Large-scale Volunteer Engagement in Humanitarian Mapping
CSCW '16 Companion: Proceedings of the 19th ACM Conference on Computer Supported Cooperative Work and Social Computing CompanionOrganisers of large crowdsourcing initiatives need to con-sider how to produce outcomes with their projects, but also how to build volunteer capacity. The initial contributor ex-perience plays an important role in this, particularly when contributions ...
Effects of peer feedback on contribution: a field experiment in Wikipedia
CHI '13: Proceedings of the SIGCHI Conference on Human Factors in Computing SystemsOne of the most significant challenges for many online communities is increasing members' contributions over time. Prior studies on peer feedback in online communities have suggested its impact on contribution, but have been limited by their ...






Comments