In this series of blog posts we will dive into the literature around peer review and peer feedback. Each post will summarize the main findings of a different academic paper, find all the research summaries here.
“Rethinking feedback practices in higher education: a peer review perspective” https://www.tandfonline.com/doi/abs/10.1080/02602938.2013.795518 Authors: David Nicol, Department of Education Enhancement, University of Strathclyde Avril Thomson, Department of Design, Manufacture and Engineering Management, University of Strathclyde Caroline Breslin, Department of Education Enhancement, University of Strathclyde
What were they trying to understand?
The 3 main questions that this paper is trying to answer, are:
- What were students’ experiences of and attitudes towards peer review in general?
- What were students’ perceptions of the learning benefits associated with the different components of the peer review process, giving and receiving feedback, and how did these processes influence their own assignment productions?
- What mental processes did students engage in whilst carrying out reviewing activities and whilst constructing feedback?
The authors mention a similar study by Cho and MacArthur (2011) where the effect on writing quality was investigated for a group of students just receiving feedback and a group of students just providing feedback. The positive effects were significantly better for the group of students that produced reviews themselves. This paper tries to understand this process better through a qualitative research approach.
How did they did they approach their research?
The authors implemented peer review in a first-year engineering class of 82 students at University of Strathclyde. The students had to complete a large assignment where they had to produce a “product design specification” (PDS). Students submitted a draft of their PDS individually then anonymously reviewed 2 submissions by other students. Lastly, they reviewed their own submission. After the peer review process the students were able to revise their submission before turning it in for teacher assessment. The peer review activity did not directly have any impact on the grade, but participation was a stated course requirement.
The peer review was done according to teacher specified criteria which included:
- Do you feel the PDS is complete in the range of headings covered? If not, can you suggest any headings that would contribute towards the completeness of the PDS and then explain why they are important?
- Is the PDS specific enough? Does it specify appropriate target values or ranges of values? Please suggest aspects that would benefit from further detail and explain.
- To what extent do you think the rationale is convincing for the PDS? Can you make any suggestions as to how it might be more convincing? Please explain.
- Can you identify one main improvement that could be made to the PDS? Provide reason(s) for your answer.
To answer the research questions, the students were asked to answer a survey with 21 questions about their perceptions of the process (64 students answered), and additionally 15 students were interviewed in 4 focus groups. The focus groups were based around the following questions:
- How did you go about doing the review of the other students’ work?
- When you were doing it what was going on in your head, can you remember?
- What was the sequence of steps you took in carrying out the review?
- What were you thinking as you were carrying out the review?
What did they find?
Students were generally positive towards peer review with 76% saying that they would definitely participate in a peer review activity in the future and 86% confirming that their peer review experience had been positive.
Some students commented on the positive effects of anonymity:
“I felt fine as you didn’t know who was looking at yours or whose you were looking at” and “glad it was anonymous”.
53% of students rated the reviews they received as excellent or good quality and 65% rated the reviews they produced as excellent or good quality. Students had a few ideas about how they might receive even better reviews including recieving more reviews and having a teacher mark the quality of the reviews.
According to the survey, students felt that they learned slightly more from receiving feedback than from producing it, but the focus groups provided more insight into what came out of the different activities. Some students mentioned that providing reviews gave them a better understanding of standards: “I learned what the standards were and what other people’s standards were”. While some students felt that providing feedback taught them more – and half of the students in the focus groups improved their own work after having provided peer feedback but before receiving their own peer feedback.
Some students also noted that producing reviews got them to reflect on their own work:
“I had a chance to see other people’s work and aspects of their work that I felt were lacking in my own work – this helped me improve my work”.
Conceptually the open-ended survey responses and focus groups highlighted that receiving and producing feedback leads to different types of learning. Receiving feedback helps more in relation to subject content, whereas producing feedback was about critical thinking and taking the assessors perspective. Basically, the research indicates that producing feedback unlocks learning potential that is beyond what is possible through receiving feedback.
Some students noted that the act of reviewing provides an opportunity to move beyond a common limitation of received feedback, namely that it is limited to talk about what has been produced:
“For me it would probably be to give feedback because I think seeing what other people have done is more helpful than getting other people’s comments on what you have already done. By looking at other people’s work you can see for yourself what you have forgotten or not even thought about. When people give feedback on yours they generally just talk about what is there. They don’t say, well I did this on mine and you could put that in yours.”
Finally students echoed a classical sentiment in peer review, namely that of seeing how good the work by other students were:
“Seeing the quality of other’ work was a bit of a shock. I was, yes, I really need to step mine up, but then it was fine because we could then go and improve on it”.
The authors argue that reviewers engage in a series of comparative judgements, both of submissions against each other, of submissions against criteria and of their own work against that of peers. The authors describe that these comparative judgements leads to “producing explanations” which provides benefits of peer review that are not accessible from simply receiving feedback. Since the act of comparison is important for learning, this points to the importance of receiving and producing multiple reviews.
The authors end this paper on the note that this and other studies did not show a significant impact of having the self-review in addition to the peer review. They also partially conclude that the process of self-reflection can be triggered simply by the peer review itself since students will compare a peer’s work to their own anyways.
“I think when you are reviewing (the work of peers), it’s more a self-learning process, you’re teaching yourself, well, I can see somebody’s done that and that’s a strength, and I should maybe try and incorporate that somehow into my work. Whereas getting (teacher) feedback you’re kind of getting told what to do. You’re getting told this is the way you should be doing it, and this is the right way to do it. You’re not really thinking for yourself.”