Assessment born digital – Sian Bayne at UCL

Sian Bayne is Professor of Digital Education in the School of Education at the University of Edinburgh. She convenes the Digital Cultures and Education research group and teaches on the MSc in Digital Education, a fully-online course. At an earlier ELE Assessment & Feedback Special Interest Group (link for UCL people), Tony McNeill from SELCS – a graduate of that MSc – recommended we invite Sian to talk about assessment in a digital age, she kindly accepted, and Anthony Smith (UCL’s Vice Provost Education & Student Affairs) chaired the event. The abstract:

“The study and production of text is a defining academic activity, yet the way in which texts are shaped and shared in internet spaces presents an intriguing set of challenges to teachers and learners. Pedagogic work with the new generation of web artefacts requires us to work within a textual domain which is unstable, multilinear, driven by a visual logic and informed by authorship practices which are multimodal, public and sometimes collective. How can we critically approach these new writing spaces, as learners, teachers and scholars? Drawing on experience of conducting such assessment within a large, online Masters programme, the talk will demonstrate how assignments born digital can be rich, critical and creative. It will also consider how as teachers we can manage, mark and organise for these assessment forms.”

Sian’s MSc students have a range of digital skills. As a fully-online course contact is crucial, so students are required to blog frequently for a term, privately by default but shared if preferred, receiving individual feedback in the form of comments on posts. This is necessarily labour intensive for the teaching team since it is intended to replicate the one-to-one tutorial within the blog space, as far as possible. To build students’ confidence and skills with multimodal presentation they’re set a number of formative tasks in advance of higher stakes assessment – for example to rework a passage from Plato’s Phaedrus.

For high stakes assessment students have a choice – they can submit work in established essay form but have the option to instead work on digital artefacts out on the Web. Where these are public they can bring new and exhilarating kinds of attention, sometimes from the thinkers whose work they are referencing. Increasing numbers of students are choosing this multimodal alternative (a side effect is that the public nature of the work also raises the profile of the MSc).

Proposals to assess beyond the essay often prompt questions about the appropriateness of other modes for academic communication – as one person asked during the discussion, don’t images and music fall within a cultural domain apart from academia, an emotional realm of implicit meaning and taste – isn’t it more art than scholarly communication? Sian emphasised that multimodal assessment shouldn’t be treated as a special case, and that the MSc assessment criteria are conventional and shared with other postgraduate courses in Edinburgh. Moreover the student work we saw was sophisticated. A student used a screen capture of his explorations in Google Earth and Google Streetview, rhetorical forms attuned to the content of his work on flaneurship. To pose questions about the meaning of originality in a copy-paste age, another fabricated a plagiarised essay with each section linked to its source, juxtaposed with an essay on the same subject which adhered to established norms of academic integrity.

There was a question about whether assessment criteria conceived with text in mind could adequately comprehend the sensuality and interpretive ambiguity of multimodal work. Sian observed that the MSc assessors were alive to their burden of responsibility to interpret the work. There is a single holistic mark rather than breaking down by criteria, and there is moderation and sometimes third marking. Trust between marker and student is important; students and tutors need to know each other because assessing this kind of work depends on building a relationship between tutor and students. Sian explained that students are asked to propose their own assessment criteria in addition to the regulated ones. There may be much to learn from assessment practices in visual arts when assessing multimodal work in humanities and social sciences. There was a discussion about the role of images – it was clear that they needed to be doing rhetorical work, and students who simply used them illustratively or ornamentally tended to be marked down.

On more than one occasion Sian observed that “text is not being toppled”. Digital modes aren’t taking over; it’s more a case of what exceeds, rather than what comes after, ‘the essay’. Programmes and institutions who are doing this now are the ones which are willing to experiment.

If you’re at UCL and want to experiment with multimodal assessment, E-Learning Environments looks forward to working with you. Contact your school’s E-Learning Facilitator to discuss – Jessica Gramp (BEAMS), Natasa Perovic (SLMS), and Mira Vogel (SLASH). At UCL there are plenty of precedents, including Making History (History Department), Internet Cultures (Institute of Education), Digital anthropology, the BEng, and an object-based learning module called Object Lessons (more on the latter to come). See also Laura Gibbs from the University of Oklahoma in a short conversation with Howard Rheingold about how her students retell old stories in new ways.