Peer ratings in group assessment: How to foster personal responsibility for independent learning

Group assessments breed the problem of free riders.  Each member within the group has varying propensities to do work and different objectives.  If you are lazy and you know it, the chances are that there is somebody in the group who is more hard-working than you are, so there is an incentive to slack off knowing that the hard-working person will probably cover for you as it is in their interests to make sure that the marks are as high as possible.  This is manifestly unfair, and is a problem to teachers as the marks no longer reflect the capabilities of each student within the group.

Yet it is possible to design systems that foster personal responsibility for independent learning. To address the ‘free-rider’ problem, my students are asked to rate each other’s performance on professionalism and effort in a group assignment. This peer grade is then used to weight individual performance within the group; thus each group member receives a mark commensurate to their contribution to the total.

Here's a quick visual overview: 

STEP ONE: Grade the paper as you would normally.








STEP TWO: Students rate each other on their respective performances. I use a simple Likert 1-5 scale where 5 is maximum.  Students should be given some rubric or guide.  The teacher should specify in what circumstances would someone deserve a 5 or in what circumstances would they receive a 1.




STEP THREE: The ratings are then summed and an average rating is awarded to each student.  In this case student A has worked the hardest, and so he receives an average scalar rating of  1.11.





STEP FOUR: The rating is applied as a scalar to the base mark, pushing the hardest working student up and vice versa for those that have not applied themselves.

Safety checks
Wherever there are new rules, enterprising individual may see opportunities to exploit the system.  This is a good thing, because it means that educators have the power to adjust the system to build in effective safety checks.

Problems	Solutions
Self-interest: Students might rate themselves higher	Self-ratings are required to ensure reflection but are ignored
One student might pressure a weaker student into an unfair rating	Ratings are collected during the invigilated final examination, effectively making it a secret ballot to foster honest peer ratings.
Ex ante harassment: Ratings may be used to victimise another student to the point of failing. (How can we avoid a popularity contest)	The algorithm ensures no student can fail solely due to poor peer ratings.
Ex post harassment: Students target someone that rates them poorly.	Group mark and individual final grades are disclosed, but not the individual peer ratings.
Ratings come too late to intervene	At mid project, students submit a short self-reflection on the group dynamic. Teachers use this qualitative data to match against quantitative peer ratings to detect unfair collusion.

Task rational: How can we explain this to students?
Students are often suspicious of new assessment mechanisms as ways for teachers to extract more work for little intrinsic reward. Therefore, it is important that you explain that this system reflects rewards in the real world. High wages and success are often achievable by simply being part of a growth industry, but likewise, high performing staff are often usually rewarded well regardless of what industry they join. The design of peer weighted grades refocuses attention on individual learning within the team, as the balance between teamwork and personal responsibility are crucial skills for professional success.

Other Providers
Peer evaluation mechanisms have been about for a long time, so I can not claim this system as being my own. There is a great program out of the UK called Web PA which seeks to do this exact system.
http://webpaproject.lboro.ac.uk/


In Australia, there is a program developed at University Technology of Sydney (UTS) called SPARK+.  Spark is quite an impressive piece of software, however is quite complex for teachers to use and the interface could be polished. SPARK is well researched and is methodologically solid and used internationally. If you are doing this for a large number of students, it is probably better to use dedicated software for this purpose.  SPARK is also quite cheap, as I understand the UTS is currently providing the software on a cost recovery basis.
http://spark.uts.edu.au/

Happy grading one and all
-Tetracarbon out.

References
Falchikov, Nancy, and Judy Goldfinch. "Student peer assessment in higher education: A meta-analysis comparing peer and teacher marks." Review of educational research 70.3 (2000): 287-322.

Goldfinch, Judy, and Robert Raeside. "Development of a peer assessment technique for obtaining individual marks on a group project." Assessment and evaluation in Higher Education 15.3 (1990): 210-231.

Willey, K. and A. Gardner (2010). "Investigating the capacity of self and peer assessment activities to engage students and promote learning." European Journal of Engineering Education 35(4): 429-443.