Close
An even better peer feedback experience with the Moodle Workshop activity
By Mira Vogel, on 21 December 2015
This is the third and final post in a series about using the Moodle Workshop activity for peer feedback, in which I’ll briefly summarise how we acted on recommendations from the second iteration which in turn built on feedback from the first go. The purpose is to interpret pedagogical considerations as Moodle activity settings.
To refresh your memories, the setting is the UCL Arena Teaching Association Programme in which postgraduate students, divided into three cognate cohorts, give and receive peer feedback on case studies they are preparing for their Higher Education Academy Associate Fellowship application. Since the activity was peer feedback only, we weren’t exploiting the numeric grades, tutor grades, or grade weighting capabilities of Moodle Workshop on this occasion.
At the point we last reported on Moodle Workshop there were a number of recommendations. Below I revisit those and summarise the actions we took and their consequences.
Improve signposting from the Moodle course area front page, and maybe the title of the Workshop itself, so students know what to do and when.
We changed the title to a friendly imperative: “Write a mini case study, give peer feedback”. That is how the link to it now appears on the Moodle page.
Instructions: let students know how many reviews they are expected to do; let them know if they should expect variety in how the submissions display.
Noting that participants may need to click or scroll for important information, we used the instructions fields for submissions and for assessment to set out what they should expect to see and do, and how. In instructions for Submission this included word count, how to submit, and that their names would appear with their submission. Then the instructions for Assessment included how to find the allocation, a rough word count for feedback, and that peer markers’ names would appear with their feedback (see below for more on anonymity). The Conclusion included how to find both the original submission and the feedback on it.
In the second iteration some submissions had been attachments while others had been typed directly into Moodle. This time we set attachments to zero, instead requiring all participants to paste their case studies directly into Moodle. We hoped that the resulting display of submission and its assessment on the same page would help with finding the submission and with cross-referencing. Later it emerged that there were mixed feelings about this: one participant reported difficulties with footnotes and another said would have preferred a separate document so he could arrange the windows in relation to each other, rather than scrolling. In future we may allow attachments, and include a line in the instructions prompting participants to look for an attachment if they can’t see the submission directly in Moodle.
Since the participants were entirely new to the activity, we knew we would need to give more frequent prompts and guidance than if they were familiar with it. Over the two weeks we sent out four News Forum posts in total at fixed times in relation to the two deadlines. The first launched the activity, let participants know where to find it, and reminded them about the submission deadline; the second, a couple of days before the submission deadline, explained that the deadline was hard and let them know how and when to find the work they had been allocated to give feedback; the third reminded them of the assessment deadline; the fourth let them know where and when to find the feedback they had been given. When asked whether these emails had been helpful or a nuisance, the resounding response was that they had been useful. Again, if students had been familiar with the process, we would have expected to take a much lighter touch on the encouragement and reminders, but first times are usually more effort.
Consider including an example case study & feedback for reference.
We linked to one rather than including it within the activity (which is possible) but some participants missed the link. There is a good case for including it within the activity (with or without the feedback). Since this is a low-stakes, voluntary activity, we would not oblige participants to carry out a practice assessment.
Address the issue that, due to some non-participation during the Assessment phase, some students gave more feedback than they received.
In our reminder News Forum emails we explicitly reminded students of their role in making sure every participant received feedback. In one cohort this had a very positive effect with participants who didn’t make the deadline (which is hard for reasons mentioned elsewhere) using email to give feedback on their allocated work. We know that, especially with non-compulsory activities and especially if there is a long time between submitting, giving feedback and receiving feedback, students will need email prompts to remind them what to do and when.
We originally had a single comments field but will now structure the peer review with some questions aligned to the relevant parts of the criteria.
Feedback givers had three question prompts to which they responded in free text fields.
Decide about anonymity – should both submissions and reviews be anonymous, or one or the other, or neither? Also to consider – we could also change Permissions after it’s complete (or even while it’s running) to allow students to access the dashboard and see all the case studies and all the feedback.
We decided to even things out by making both the submissions and reviews attributable, achieving this by changing the permissions for that Moodle Workshop activity before it ran. We used the instructions for submissions and assessment to flag this to participants.
A lead tutor for one of the cohorts had been avoiding using Moodle Workshop because she felt it was too private between a participant their few reviewees. We addressed this after the closure of the activity by proposing to participants that we release all case studies and their feedback to all participants in the cohort (again by changing the permissions for that Moodle Workshop activity). We gave them a chance to raise objections in private, but after receiving none we went ahead with the release. We have not yet checked the logs to see whether this access has been exploited.
Other considerations.
Previously we evaluated the peer feedback activity with a questionnaire, but this time we didn’t have the opportunity for that. We did however have the opportunity to discuss the experience with one of the groups. This dialogue affirmed the decisions we’d taken. Participants were positive about repeating the activity, so we duly ran it again after the next session. They also said that they preferred to receive feedback from peers in their cognate cohort, so we maintained the existing Moodle Groupings (Moodle Groups would also work if the cohorts had the same deadline date, but ours didn’t, which is why we had three separate Moodle Workshop instances with Groupings applied).
The staff valued the activity but felt that without support from ELE they would have struggled to make it work. ELE is responding by writing some contextual guidance for that particular activity, including a reassuring checklist.