Abstract
How does group membership framing affect the feedback students provide learners? This paper presents two between-subjects experiments investigating the effect of Ingroup/Outgroup membership on effort spent in peer evaluations, and whether the group membership criterion type affects quality and stringency of evaluation. Two peer-review assignments were implemented in two separate classes. In the first study, students were nominally grouped by location they sat in class and non-nominally grouped by current class score; each was asked to review an Ingroup and Outgroup peer assignment. A second study randomly assigned students to one of four group types (random, score, motivation, and location); student reviewed two Ingroup assignments. In both studies, score-grouped students graded their peers more stringently than students grouped by location. These studies illustrate for system designers the impacts of group framing - and the disclosure of that-in peer review tasks.
- Daniel Balliet, Junhui Wu, and Carsten K. W. De Dreu. 2014. Ingroup favoritism in cooperation: A meta-analysis. Psychol. Bull. 140, 6 (2014), 1556--1581.Google Scholar
Cross Ref
- Douglas Bates, Martin Mächler, Ben Bolker, and Steve Walker. 2015. Fitting Linear Mixed-Effects Models Using lme4. J. Stat. Softw. Vol 1 Issue 1 2015 (2015).Google Scholar
- Erin Bradner and Gloria Mark. 2002. Why Distance Matters: Effects on Cooperation, Persuasion and Deception. In Proceedings of the 2002 ACM Conference on Computer Supported Cooperative Work (CSCW '02), 226--235. Google Scholar
Digital Library
- Marilynn B. Brewer. 1979. In-group bias in the minimal intergroup situation: A cognitive-motivational analysis. Psychol. Bull. 86, 2 (1979), 307--324.Google Scholar
Cross Ref
- Priyanka B. Carr and Gregory M. Walton. 2014. Cues of working together fuel intrinsic motivation. J. Exp. Soc. Psychol. 53, (2014), 169--184.Google Scholar
- Justin Cheng, Jaime Teevan, and Michael S. Bernstein. 2015. Measuring Crowdsourcing Effort with Error-Time Curves. In Proceedings of the 33rd Annual ACM Conference on Human Factors in Computing Systems (CHI '15), 1365--1374. Google Scholar
Digital Library
- Winnie Cheng and Martin Warren. 1997. Having second thoughts: Student perceptions before and after a peer assessment exercise. Stud. High. Educ. 22, 2 (January 1997), 233--239.Google Scholar
Cross Ref
- Kwangsu Cho and Christian D. Schunn. 2007. Scaffolded Writing and Rewriting in the Discipline: A Web-based Reciprocal Peer Review System. Comput Educ 48, 3 (April 2007), 409--426. Google Scholar
Digital Library
- Phil Davies. 2000. Computerized Peer Assessment. Innov. Educ. Train. Int. 37, 4 (January 2000), 346--355.Google Scholar
- Carol S. Dweck. 2008. Mindset: The New Psychology of Success. Ballantine Books, New York.Google Scholar
- Naomi Ellemers, S. Alexander Haslam, and Dick de Gilder. 2008. Motivating Individuals and Groups at Work in the 21st Century. In 21st Century Management: A Reference Handbook. SAGE Publications, Inc., Thousand Oaks, II--182--II--192. Retrieved from http://sk.sagepub.com/reference/management/n68.xmlGoogle Scholar
- Rosta Farzan, Laura A. Dabbish, Robert E. Kraut, and Tom Postmes. 2011. Increasing Commitment to Online Communities by Designing for Social Presence. In Proceedings of the ACM 2011 Conference on Computer Supported Cooperative Work (CSCW '11), 321--330. Google Scholar
Digital Library
- Dilrukshi Gamage, Mark E. Whiting, Thejan Rajapakshe, Haritha Thilakarathne, Indika Perera, and Shantha Fernando. 2017. Improving Assessment on MOOCs Through Peer Identification and Aligned Incentives. In Proceedings of the Fourth (2017) ACM Conference on Learning @ Scale ([email protected] '17), 315--318. Google Scholar
Digital Library
- Andrew Gelman and Jennifer Hill. 2006. Data Analysis Using Regression and Multilevel/Hierarchical Models. Cambridge University Press, Cambridge. Retrieved from https://www.cambridge.org/core/books/data-analysis-using-regression-and-multilevelhierarchical-models/32A29531C7FD730C3A68951A17C9D983Google Scholar
- Andrew Gelman, Jennifer Hill, and Masanao Yajima. 2012. Why We (Usually) Don't Have to Worry About Multiple Comparisons. J. Res. Educ. Eff. 5, 2 (April 2012), 189--211.Google Scholar
Cross Ref
- Stephanie J. Hanrahan and Geoff Isaacs. 2001. Assessing Self- and Peer-assessment: The students' views. High. Educ. Res. Dev. 20, 1 (May 2001), 53--70.Google Scholar
Cross Ref
- Catherine M. Hicks, Vineet Pandey, C. Ailie Fraser, and Scott Klemmer. 2016. Framing Feedback: Choosing Review Environment Features That Support High Quality Peer Assessment. In Proceedings of the 2016 CHI Conference on Human Factors in Computing Systems (CHI '16), 458--469. Google Scholar
Digital Library
- Julia H. Kaufman and Christian D. Schunn. 2011. Students' perceptions about peer assessment for writing: their origin and impact on revision work. Instr. Sci. 39, 3 (2011), 387--406.Google Scholar
Cross Ref
- Chinmay E. Kulkarni, Michael S. Bernstein, and Scott R. Klemmer. 2015. PeerStudio: Rapid Peer Feedback Emphasizes Revision and Improves Performance. In Proceedings of the Second (2015) ACM Conference on Learning @ Scale ([email protected] '15), 75--84. Google Scholar
Digital Library
- Chinmay Kulkarni, Koh Pang Wei, Huy Le, Daniel Chia, Kathryn Papadopoulos, Justin Cheng, Daphne Koller, and Scott R. Klemmer. 2013. Peer and Self Assessment in Massive Online Classes. ACM Trans Comput-Hum Interact 20, 6 (December 2013), 33:1--33:31. Google Scholar
Digital Library
- Kimberly Ling, Gerard Beenen, Pamela Ludford, Xiaoqing Wang, Klarissa Chang, Xin Li, Dan Cosley, Dan Frankowski, Loren Terveen, Al Mamunur Rashid, Paul Resnick, and Robert Kraut. 2005. Using social psychology to motivate contributions to online communities. J. Comput.-Mediat. Commun. 10, 4 (January 2005).Google Scholar
Cross Ref
- Steson Lo and Sally Andrews. 2015. To transform or not to transform: using generalized linear mixed models to analyse reaction time data. Front. Psychol. 6, (2015), 1171.Google Scholar
- Yanxin Lu, Joe Warren, Christopher Jermaine, Swarat Chaudhuri, and Scott Rixner. 2015. Grading the Graders: Motivating Peer Graders in a MOOC. In Proceedings of the 24th International Conference on World Wide Web (WWW '15), 680--690. Google Scholar
Digital Library
- Jennifer Marlow and Laura Dabbish. 2012. Designing Interventions to Reduce Psychological Distance in Globally Distributed Teams. In Proceedings of the ACM 2012 Conference on Computer Supported Cooperative Work Companion (CSCW '12), 163--166. Google Scholar
Digital Library
- M. M. Nelson and C. D. Schunn. 2009. The nature of feedback: How different types of peer feedback affect writing performance. Instr. Sci. 37, (July 2009), 375--401.Google Scholar
- Melissa M. Patchan, Brandi Hawk, Christopher A. Stevens, and Christian D. Schunn. 2013. The effects of skill diversity on commenting and revisions. Instr. Sci. 41, 2 (2013), 381--405.Google Scholar
Cross Ref
- Maria Perez-Ortiz and Rafal K. Mantiuk. 2017. A practical guide and software for analysing pairwise comparison experiments. ArXiv171203686 Cs Stat (December 2017). Retrieved from http://arxiv.org/abs/1712.03686Google Scholar
- Ken Reily, Pam Ludford Finnerty, and Loren Terveen. 2009. Two Peers Are Better Than One: Aggregating Peer Reviews for Computing Assignments is Surprisingly Accurate. In Proceedings of the ACM 2009 International Conference on Supporting Group Work (GROUP '09), 115--124. Google Scholar
Digital Library
- Yuqing Ren, F. Maxwell Harper, Sara Drenner, Loren Terveen, Sara Kiesler, John Riedl, and Robert E. Kraut. 2012. Building Member Attachment in Online Communities: Applying Theories of Group Identity and Interpersonal Bonds. MIS Q 36, 3 (September 2012), 841--864. Google Scholar
Digital Library
- Todd Rogers and Avi Feller. 2016. Discouraged by Peer Excellence: Exposure to Exemplary Peer Performance Causes Quitting. Psychol. Sci. 27, 3 (March 2016), 365--374.Google Scholar
- J. Sanchez-Burks, R. E. Nisbett, and O. Ybarra. 2000. Cultural styles, relational schemas, and prejudice against out-groups. J. Pers. Soc. Psychol. 79, 2 (August 2000), 174--189.Google Scholar
Cross Ref
- Stanley Schacter. 1959. The Psychology of Affiliation; Experimental Studies of the Sources of Gregariousness. Stanford University Press, Stanford, California.Google Scholar
- Natalie Sebanz, Harold Bekkering, and Günther Knoblich. 2006. Joint action: bodies and minds moving together. Trends Cogn. Sci. 10, 2 (February 2006), 70--76.Google Scholar
Cross Ref
- Garriy Shteynberg and Adam D. Galinsky. 2011. Implicit coordination: Sharing goals with similar others intensifies goal pursuit. J. Exp. Soc. Psychol. 47, 6 (November 2011), 1291--1294.Google Scholar
Cross Ref
- Walter W. Stroup. 2013. Generalized linear mixed models: modern concepts, methods and applications. CRC Press, Boca Raton, Fl.Google Scholar
- Henri Tajfel. 1974. Social identity and intergroup behaviour. Inf. Int. Soc. Sci. Counc. 13, 2 (April 1974), 65--93.Google Scholar
Cross Ref
- Yla R. Tausczik, Laura A. Dabbish, and Robert E. Kraut. 2014. Building Loyalty to Online Communities Through Bond and Identity-based Attachment to Sub-groups. In Proceedings of the 17th ACM Conference on Computer Supported Cooperative Work & Social Computing (CSCW '14), 146--157. Google Scholar
Digital Library
- UCLA: Statistical Consulting Group. Introduction to SAS. Retrieved from https://stats.idre.ucla.edu/sas/faq/how-can-i-interpret-log-transformed-variables-in-terms-of-percent-change-in-linear-regression/Google Scholar
- Siri Vinther, Ole Haagen Nielsen, Jacob Rosenberg, Niels Keiding, and Torben V. Schroeder. 2012. Same review quality in open versus blinded peer review in "Ugeskrift for Læger." Dan. Med. J. 59, 8 (August 2012), A4479.Google Scholar
- Gregory M. Walton and Shannon T. Brady. The Many Questions of Belonging. In Handbook of Competence and Motivation: Theory and Application (2nd ed.), Andrew J. Elliot, Carol S. Dweck and Yeager (eds.). Guilford Press, New York.Google Scholar
- Gregory M. Walton and Geoffrey L. Cohen. 2011. Sharing motivation. In Social motivation. Psychology Press, New York, NY, US, 79--101.Google Scholar
- Gregory M. Walton, Geoffrey L. Cohen, David Cwir, and Steven J. Spencer. 2012. Mere belonging: the power of social connections. J. Pers. Soc. Psychol. 102, 3 (March 2012), 513--532.Google Scholar
Cross Ref
Index Terms
Score-Group Framing Negatively Impacts Peer Evaluations
Recommendations
Accommodating field-dependence: a cross-over study
In a previous study, it was found that field-independent learners performed significantly better than field-dependent learners in a web based and in a computerised text-based environment. It was decided to adapt both a web interface and a traditional ...
Accommodating field-dependence: a cross-over study
ITiCSE '04: Proceedings of the 9th annual SIGCSE conference on Innovation and technology in computer science educationIn a previous study, it was found that field-independent learners performed significantly better than field-dependent learners in a web based and in a computerised text-based environment. It was decided to adapt both a web interface and a traditional ...
Exploring the impact of the rubric on peer feedback content quality in online learning
ICDEL '22: Proceedings of the 7th International Conference on Distance Education and LearningThis study examines the value of structuring the peer assessment process by providing students with a peer assessment rubric for the peer feedback content quality in an online discussion board in higher education. The study took place in the 1st year of ...






Comments