skip to main content
research-article

Private Peer Feedback as Engagement Driver in Humanitarian Mapping

Authors Info & Claims
Published:06 December 2017Publication History
Skip Abstract Section

Abstract

Prior research suggests that public negative feedback on social knowledge sharing platforms can be powerfully demotivating to newcomers, particularly when it involves peer feedback mechanisms such as ratings and commenting systems. What is the impact on newcomer retention when feedback is private, and from a single peer reviewer? We study these effects using the example of the Humanitarian OpenStreetMap Team, a Wikipedia-style social mapping platform where the review process is closer to a teacher-learner model rather than a public peer review. We observe peer feedback for early contributions by 1,300 newcomers, and assess the impact of different classes of feedback, including performance feedback, corrective feedback, and verbal rewards. We find that verbal rewards and immediate feedback can have a powerful effect on newcomer retention. In order to better support such positive engagement effects, we recommend that system designers conceptually distinguish between mechanisms for quality control and for learner feedback.

References

  1. Panagiotis Adamopoulos. 2013. What makes a great MOOC? An interdisciplinary analysis of student retention in online courses ICIS.Google ScholarGoogle Scholar
  2. June Ahn, Brian S. Butler, Cindy Weng, and Sarah Webster. 2013. Learning to be a better Q'er in social Q&A sites: social norms and information artifacts. Proceedings of the American Society for Information Science and Technology, Vol. 50, 1 (2013), 1--10. Google ScholarGoogle ScholarDigital LibraryDigital Library
  3. Blake K Ashforth and Alan M Saks. 1996. Socialization tactics: Longitudinal effects on newcomer adjustment. Academy of management Journal Vol. 39, 1 (1996), 149--178.Google ScholarGoogle Scholar
  4. Jonathan Bishop. 2007. Increasing participation in online communities: A framework for human-computer interaction. Computers in human behavior Vol. 23, 4 (2007), 1881--1893. Google ScholarGoogle ScholarDigital LibraryDigital Library
  5. Joel Brockner, William R Derr, and Wesley N Laing. 1987. Self-esteem and reactions to negative feedback: Toward greater generalizability. Journal of Research in Personality Vol. 21, 3 (1987), 318--333.Google ScholarGoogle ScholarCross RefCross Ref
  6. Judy Cameron, Katherine M Banko, and W David Pierce. 2001. Pervasive negative effects of rewards on intrinsic motivation: The myth continues. The Behavior Analyst, Vol. 24, 1 (2001), 1.Google ScholarGoogle ScholarCross RefCross Ref
  7. Justin Cheng, Cristian Danescu-Niculescu-Mizil, and Jure Leskovec. 2014. How Community Feedback Shapes User Behavior. In Proc. ICWSM '14.Google ScholarGoogle Scholar
  8. Boreum Choi, Kira Alexander, Robert E Kraut, and John M Levine. 2010. Socialization tactics in Wikipedia and their effects Proc. CSCW '10. ACM, 107--116. Google ScholarGoogle ScholarDigital LibraryDigital Library
  9. Edward L Deci, Richard Koestner, and Richard M Ryan. 1999. A meta-analytic review of experiments examining the effects of extrinsic rewards on intrinsic motivation. Psychological bulletin Vol. 125, 6 (1999), 627.Google ScholarGoogle Scholar
  10. Chrysanthos Dellarocas. 2003. The digitization of word of mouth: Promise and challenges of online feedback mechanisms. Management science, Vol. 49, 10 (2003), 1407--1424. Google ScholarGoogle ScholarDigital LibraryDigital Library
  11. Martin Dittus, Giovanni Quattrone, and Licia Capra. 2016. Analysing volunteer engagement in humanitarian mapping: building contributor communities at large scale. In Proc. CSCW '16. Google ScholarGoogle ScholarDigital LibraryDigital Library
  12. Martin Dittus, Giovanni Quattrone, and Licia Capra. 2017. Mass participation during emergency response: event-centric crowd-sourcing in humanitarian mapping. In Proc. CSCW '17. Google ScholarGoogle ScholarDigital LibraryDigital Library
  13. Melanie Eckle and Jo ao Porto de Albuquerque. 2015. Quality assessment of remote mapping in OpenStreetMap for disaster management purposes Proc. ISCRAM '15.Google ScholarGoogle Scholar
  14. Laurence S Freedman. 1982. Tables of the number of patients required in clinical trials using the logrank test. Statistics in medicine Vol. 1, 2 (1982), 121--129.Google ScholarGoogle Scholar
  15. Aaron Halfaker, Aniket Kittur, and John Riedl. 2011. Don't bite the newbies: how reverts affect the quantity and quality of Wikipedia work Proc. WikiSym '11. ACM, 163--172. Google ScholarGoogle ScholarDigital LibraryDigital Library
  16. Judith M Harackiewicz. 1979. The effects of reward contingency and performance feedback on intrinsic motivation. Journal of personality and social psychology, Vol. 37, 8 (1979), 1352.Google ScholarGoogle ScholarCross RefCross Ref
  17. John Hattie and Helen Timperley. 2007. The power of feedback. Review of educational research Vol. 77, 1 (2007), 81--112.Google ScholarGoogle Scholar
  18. Aniket Kittur, Bryan Pendleton, and Robert E Kraut. 2009. Herding the cats: the influence of groups in coordinating peer production Proc. WikiSym '09. ACM, 7. Google ScholarGoogle ScholarDigital LibraryDigital Library
  19. Robert E Kraut, Paul Resnick, Sara Kiesler, Moira Burke, Yan Chen, Niki Kittur, Joseph Konstan, Yuqing Ren, and John Riedl. 2012. Building successful online communities: Evidence-based social design. Mit Press. Google ScholarGoogle ScholarDigital LibraryDigital Library
  20. J Richard Landis and Gary G Koch. 1977. The measurement of observer agreement for categorical data. biometrics (1977), 159--174.Google ScholarGoogle Scholar
  21. Jingyan Lu and Nancy Law. 2012. Online peer assessment: effects of cognitive and affective feedback. Instructional Science Vol. 40, 2 (2012), 257--275.Google ScholarGoogle ScholarCross RefCross Ref
  22. Richard S Lysakowski and Herbert J Walberg. 1982. Instructional effects of cues, participation, and corrective feedback: A quantitative synthesis. American Educational Research Journal Vol. 19, 4 (1982), 559--572.Google ScholarGoogle ScholarCross RefCross Ref
  23. Loizos Michael and Jahna Otterbacher. 2014. Write Like I Write: Herding in the Language of Online Reviews. Proc. ICWSM '14.Google ScholarGoogle Scholar
  24. Richard L Moreland and Paul D Sweeney. 1984. Self-expectancies and reactions to evaluations of personal performance. Journal of Personality Vol. 52, 2 (1984), 156--176.Google ScholarGoogle ScholarCross RefCross Ref
  25. Jonathan T Morgan, Siko Bouterse, Heather Walls, and Sarah Stierch. 2013. Tea and sympathy: crafting positive new user experiences on wikipedia Proc. CSCW '13. ACM, 839--848. Google ScholarGoogle ScholarDigital LibraryDigital Library
  26. David R Musicant, Yuqing Ren, James A Johnson, and John Riedl. 2011. Mentoring in Wikipedia: a clash of cultures. In Proc. WikiSym '11. ACM, 173--182. Google ScholarGoogle ScholarDigital LibraryDigital Library
  27. Leysia Palen, Robert Soden, T Jennings Anderson, and Mario Barrenechea. 2015. Success & scale in a data-producing organization: the socio-technical evolution of OpenStreetMap in response to humanitarian events Proc. SIGCHI '15. 4113--4122. Google ScholarGoogle ScholarDigital LibraryDigital Library
  28. Katherine Panciera, Aaron Halfaker, and Loren Terveen. 2009. Wikipedians are born, not made: a study of power editors on Wikipedia Proc. GROUP '09. ACM, 51--60. Google ScholarGoogle ScholarDigital LibraryDigital Library
  29. Laurence D Parnell, Pierre Lindenbaum, Khader Shameer, Giovanni Marco Dall'Olio, Daniel C Swan, Lars Juhl Jensen, Simon J Cockell, Brent S Pedersen, Mary E Mangan, Christopher A Miller, et almbox. 2011. BioStar: an online question & answer resource for the bioinformatics community. PLoS Comput Biol, Vol. 7, 10 (2011), e1002216.Google ScholarGoogle ScholarCross RefCross Ref
  30. Paul Resnick, Ko Kuwabara, Richard Zeckhauser, and Eric Friedman. 2000. Reputation systems. Commun. ACM Vol. 43, 12 (2000), 45--48. Google ScholarGoogle ScholarDigital LibraryDigital Library
  31. Robert Soden and Leysia Palen. 2014. From crowdsourced mapping to community mapping: the post-earthquake work of OpenStreetMap Haiti. (2014).Google ScholarGoogle Scholar
  32. Yla R Tausczik and James W Pennebaker. 2012. Participation in an online mathematics community: differentiating motivations to add Proc. CSCW '12. ACM, 207--216. Google ScholarGoogle ScholarDigital LibraryDigital Library
  33. Gershon Tenenbaum and Ellen Goldring. 1989. A meta-analysis of the effect of enhanced instruction: Cues, participation, reinforcement and feedback and correctives on motor skill learning. Journal of Research & Development in Education (1989).Google ScholarGoogle Scholar
  34. Clay Westrope, Robert Banick, and Mitch Levine. 2014. Groundtruthing OpenStreetMap building damage assessment. Procedia engineering Vol. 78 (2014), 29--39.Google ScholarGoogle Scholar
  35. Donghee Yvette Wohn. 2015. The Effects of Feedback and Habit on Content Posting in an Online Community. iConference 2015 Proceedings (2015).Google ScholarGoogle Scholar
  36. Haiyi Zhu, Amy Zhang, Jiping He, Robert E Kraut, and Aniket Kittur. 2013. Effects of peer feedback on contribution: a field experiment in Wikipedia Proc. SIGCHI '13. ACM, 2253--2262. Google ScholarGoogle ScholarDigital LibraryDigital Library

Index Terms

  1. Private Peer Feedback as Engagement Driver in Humanitarian Mapping

        Recommendations

        Comments

        Login options

        Check if you have access through your login credentials or your institution to get full access on this article.

        Sign in

        Full Access

        PDF Format

        View or Download as a PDF file.

        PDF

        eReader

        View online with eReader.

        eReader
        About Cookies On This Site

        We use cookies to ensure that we give you the best experience on our website.

        Learn more

        Got it!