X Close

Digital Education team blog

Home

Ideas and reflections from UCL's Digital Education team

Menu

Archive for the 'COVID' Category

Marking 24 hour exams

Steve Rowett5 May 2020

Please note this post is being regularly updated with additional resources.
+ New on Friday 8 May: Guide to online marking from Mary Richardson
+ New on Friday 8 May: Microsoft Drawboard PDF demo from Dewi Lewis, UCL Chemistry
+ New on Friday 15 May: Updated details on Microsoft Drawboard PDF

The move to online 24 hour assessments that replace traditional exams leads to a challenge for those that have to grade and mark the work.

We start from a place of two knowns:

  • Students are submitting work to Turnitin in Moodle during the 24 hour window; and
  • Final grades need to be stored in Portico, our student records system.

But in between those two endpoints, there are many different workflows by which marking can take place. These are set out by the UCL’s Academic Manual but encompasses a range of choices, particularly in how second marking is completed. One key difference between regular courseworks is that this is not about providing feedback to students, but instead about supporting the marking process, the communication between markers and the required record of the marking process. At the end of the marking process departments will need to ensure that scripts are stored securely but can be accessed by relevant staff as required, much in line with requirements for paper versions over previous years.

Neither SRS nor Digital Education mandate any particular way that marking should take place and there is considerable flexibility for departments to use processes that work best for them. So we are suggesting a menu of options which provide a basis for departments to build on if they so choose. We are also running daily training sessions which as listed at the foot of this document.

The menu options are:

  • Markers review the scripts and mark or annotate them using Turnitin Feedback Studio
  • Digital Education will provide PDF copies of scripts for departments to annotate using PDF annotation software on a computer or tablet device.
  • Markers review the scripts using Turnitin Feedback Studio, but keep a ‘marker file’ or notes and comments on the marking process.
  • Markers print the scripts and mark them, then scan them for storage or keep them for return to the department on paper.

The rest of this post goes into these options in more detail.


Turnitin Feedback Studio

Turnitin Feedback Studio provides a web-based interface where comments can be overlaid on a student’s work. QuickMarks provide a bank of comments that occur regularly that can be just drag and dropped onto the work. In addition, the traditional Turnitin Similarity Report is also available. This method probably works best for text-based documents like essays and reports. Turnitin is integrated into Moodle and set up for you as part of the exam process for students to submit their work, but it’s your choice if you wish to use the marking tools available after the work has been submitted.

You can also allocate marks using grading forms or rubrics.  On the whole we think that these could be a bit of a ‘sledgehammer to crack a nut’ solution for a single paper, but it is an option available to you if you are familiar with them and you have a more granular set marking criteria for each question. We recommend hiding the assignment before adding the grading form or rubric so that students cannot see it.

If you want to know if this method is for you, you can watch a short video demo or  try marking up an example paper provided by Turnitin. A video tailored to UCL’s 24 hour exam process is given below. This video has captions.

Things to think about with this approach:

  • Rubrics and grading forms take a little bit of setting up, and are probably best used where you have previous experience in doing them.
  • In some exams it is common to put a mark (e.g. a tick) on each page to indicate that the page has been read. To replicate this you might define a QuickMark called ‘page read’ and put it on each page, or annotate with the same words
  • The marked paper often becomes a resource to go back to if there are any errors or omissions in the grading process. You might wish to both write the marks on the paper using the annotation tools or in the general feedback area, and also lodge them in a spreadsheet for uploading to Portico.
  • Turnitin does not support double blind marking effectively. It is rarely used for paper-based exams (since the second marker could always see the markings of the first marker on the paper) but if it was needed one marker could mark online and the second could download the papers for offline marking (e.g the ‘marker form’ method below).

You can view additional guidance on using Turnitin Feedback Studio.


Annotation using PDF documents

Where you annotation needs are more sophisticated, or you want to ‘write’ on the paper using a graphics tablet or a tablet and pencil/stylus, then this option may suit you better.

Upon notification (notification form) Digital Education will supply your department with PDF copies of the students’ work, uploaded to a OneDrive account set up by your department.

For this to happen, Exam Liaison Officers / Teaching Administrators will need to set up a OneDrive folder and notify Digital Education that they wish to have PDF copies of the files. We have a video tutorial (with captions) on this process below.

You can then use tools you already have or prefer to use to do your marking. There is more flexibility here, and we will not be able to advise and support every PDF tool available or give precise instructions for every workflow used by departments, but we give some examples here.

Marking on an iPad using OneDrive

Many staff have reported using an iPad with Apple Pencil or an Android tablet with a stylus to be a very effective marking tool. The Microsoft OneDrive app supports both platforms and provides rapid access to scripts and some annotation tools as shown in the video below (which also has captions). The OneDrive app is free, and connects to your UCL OneDrive account via Single Sign On.

There’s further guidance from Microsoft on each individual annotation tool.

The Apple Files app can also connect to OneDrive and has a similar (and perhaps more powerful) annotation tool. Thanks to David Bowler for mentioning this in the first comment on this blog post.

Marking on a Mac using Preview

Preview on a Mac is often taken for granted but is actually quite a sophisticated tool and includes some basic annotation functions. Here is some guidance from Apple on using it.

Marking on a PC or Surface Pro using Microsoft Drawboard PDF

Microsoft Drawboard PDF is a very comprehensive annotation tool, but is only available for Windows 10 and is really designed to be used with a Surface Pro or a desktop with a graphics tablet. Dewi Lewis from UCL Chemistry has produced a video illustrating the annotation tools available and how to mark a set of files easily. UCL does not have a site-wide licence for Drawboard PDF, but it is available at a very modest price if departments choose to buy it.

Marking on a PC, Mac or Linux machine using a PDF annotation program.

Of course there are plenty of third party tools that support annotating PDF documents. Some requirement payment to access the annotation facilities (or to save files that have been annotated) but two that do not are Xodo and Foxit PDF.

Things to think about with this approach:

  • Your marking process: if you use double blind marking you might need to make two copies of the files, one for each marker. If you use check marking then a single copy will suffice.
  • You will need to ensure the files are stored securely and can be accessed by the relevant departmental staff in case of any query. You might share the exam submission files with key contacts such as teaching administrators or directors of teaching.
  • Some of the products listed above have a small charge, as would any stylus or pencil that staff would need. These cannot be supplied centrally, so you may need a process for staff claiming back the costs from departments.

Using a ‘marker file’

Accessing the students’ scripts is done using Turnitin in Moodle, which allows all the papers to be viewed online individually or downloaded in one go. Then a separate document is kept (either one per script, or one overall) containing the marks and marker feedback for each comment. If double-blind marking is being used, then it is easy to see that two such documents or sets of documents could be kept in this way.


Printing scripts and marking on paper

Although we have moved to online submission this year, colleagues are still welcome to print documents and mark on paper. However there is no central printing service available for completed scripts to be printed, and this would have to be managed individually or locally by departments.


The evidence about marking online

In this video Dr Mary Richardson, Associate Professor in Educational Assessment at the IOE, gives a guide to how online marking can differ from paper-based marking and offers some tips for those new to online marking. The video has captions.


Training sessions and support

Digital Education will be running daily training sessions for teachers covering the ground in this blog post. These will run at 12-1pm every weekday from Tuesday 12 May.

No booking necessary.

We are also providing additional support for students during the exam period. Our support hours will be (UK time):

  • Monday 11.30am-8.30pm
  • Tuesday-Thursday 8am-8pm
  • Friday: 7.30am-3.30pm

Details of support mechanisms are given in the exam section on each Moodle module where an exam is taking place.

Writing when teaching remotely

Steve Rowett22 April 2020

We start this blog post with a short exercise for the reader: search Google Images or an image library for the word teacher.

I hope that you see a range of diverse people in the pictures. But I bet that most of them will have a pen or chalk in their hand, and a board or surface at their side. The act of writing and talking as a form of instruction is perhaps the most commonly perceived characteristic of who a teacher is and what they do.

The move to remote teaching has made writing as a form of teaching more difficult. A mouse may be great for controlling a computer, but it’s hard to use it to write smoothly on screen.

So in this post, we will look at how we might continue to use writing as a form of teaching, even from home.

Before we start, a note on accessibility. Whilst we teach remotely, so of the facilities that our students might have relied upon (like captions, transcripts, note-takers or signers) might not be available, so we may need to make alternative accommodations. Please do think about this before you plan your teaching and contact Digital Education if you would like advice.

Using a visualiser at home

In the classroom or lecture hall, many rooms will have a visualiser (also called a document camera) for showing writing and objects. These are typically hooked up to a projection screen through the AV system and are also captured by lecture capture. UCL uses a mix of table-top visualisers and ceiling mounted visualisers from a company called Wolfvision.

You can also buy more consumer-level products in this category. At home I use an IPEZO VZ-R camera, which costs about £200.

The IPEVO VZ-R visualiser - a sturdy base with a webcam mounted on a flexible hinge.

IPEVO VZ-R Visualiser (source: ipevo.com)

The camera connects via USB and can be used in two ways:

Either method works well, but I’d recommend the latter as webcam video is often heavily compressed. This method works well with all our supported platforms, including Lecturecast (Echo 360) Universal Capture Personal, Blackboard Collaborate and Microsoft Teams.

The camera is compact but sturdy. It has an in-built light, and facilities such as zoom and autofocus, and can capture a surface slightly less than A3. It also has an inbuilt microphone which is likely to be closer to you when you are speaking than one in a laptop.

Control buttons on the IPEVO VZ-R including power, zoom and focus

Control buttons on the IPEVO VZ-R (source: ipevo.com)

The software can control the features of the camera, and can make recordings that you could upload to Lecturecast, for example. The controls neatly get out of the way when they aren’t being used, so that you can focus on the content. The image below shows the auto-focus in operation, clearly capturing a book cover.

IPEVO Visualizer software showing controls and a picture of a book under the camera

IPEVO Visualizer software showing controls

The Visualizer application can easily be shared in a Blackboard Collaborate room, and you can then move the window out of the way and focus on the participants. The image below shows me writing some simple maths on a notepad, captured by the camera, using the screen sharing feature of Collaborate to show this to other participants.

Writing equations which are shared within a Blackboard Collaborate room

Writing equations which are shared within a Blackboard Collaborate room.

IPEVO offer another model, the V4K, for £100, which has a lower resolution sensor (but still good) and no light. Of course other companies are available, and I’ve heard recommendations of Genee and Aver models. These types of visualisers are often used in secondary schools, so any teachers you know may be able to give suitable advice.

One thing I particularly like about the IPEVO models is the design. The one I have folds fairly flat, so is easy to store away, so my living room doesn’t have to look like a lecture theatre. Here’s a comparison of its size when folded compared to a bottle of wine. And yes, it does fit in a wine rack.

An IPEVO VZ-R is about the size of a bottle of wine when folded

The IPEVO VZ-R folds down to the size of a bottle of wine.

Using a webcam

If you have a webcam that isn’t built in to your computer, you can use this too. We’ve tried a couple of webcams and it can work well, but may not have the full controls or handy features like the light that the VZ-R has.

Using a mobile phone

You may have seen tweets where inventive teachers have rigged up their phones to use as cameras. But how do you do this?

Well, IPEVO provide an app for that too. To make this work, you need three things:

  • An Apple or Android phone on the same network as your computer (PC, Mac, Chromebook and Linux supported)
  • An installation of IPEVO iDocCam on your phone
  • An installation of IPEVO Visualizer on your computer

Launch the app on your phone, and then the Visualizer software on your computer, and your phone will appear in the list of available cameras.

The phone can be selected from the list of cameras available in the IPEVO Visualizer software

Your phone shows as a camera in the IPEVO Visualizer software

And here is an example of it in use, with my old IPhone SE acting as the camera:

The final bit is to rig the phone up in a secure way, about 40cm above your desk. You might use a desk mount, or a pile of books, or some blu tack and string, but I’m afraid that is up to you.

Learning to Teach Online [LinkedIn Learning course]

Clive Young6 April 2020

As you may know, UCL has access to LinkedIn Learning (formerly Lynda.com), a huge range of video tutorials supporting learning in software, creative and business skills – all free to UCL staff and currently enrolled students.

One of these is Learning to Teach Online, a recently-updated 48 minute long introduction to help instructors and teachers to understand the approaches and skills required to teach effectively online. 

As we are going through what is likely to be a significant shift to online learning at UCL, would I recommend it? Yes, I would. Although some of the ideas should already be familiar to UCL colleagues, the course provides a practical (and quick) overview. Most of the main points you need to know to get started are covered, and the course neatly highlights the difference between face-to-face, blended and online methods.

The emphasis on online organisation and communication, ensuring teacher presence, fostering collaboration, accessibility and so on are all particularity sound, and align with UCL’s E-learning Baseline. Much of the advice is in bullet-pointed checklists and only two theoretical frameworks are mentioned, the familiar Bloom’s taxonomy for learning outcomes and the rather useful SAMR model to help us think about technology integration.

I’d obviously suggest ABC to help structure the course, too, but all in all a worthwhile use of 48 minutes (or much less if you speed up the video, as I do!).

Learning Design in an Emergency Part 1

Clive Young2 April 2020

Our teachers are being forced into moving their teaching online very quickly. The initial focus for academics is inevitably be on access, getting themselves and their students online and doing what they can immediately with the institutional tools provided. For most teachers this will be far from full, integrated ‘online learning’ but more an initial digitisation of their existing face-to-face methods.

UCL is mainly focusing on Moodle (already used for resource distribution and to a lesser extent communication), Blackboard Collaborate (for online ‘tutorials’) and Lecturecast (Echo) Universal Personal Capture (short video recordings to replace lectures). The term we use at UCL is Teaching Continuity, emphasising the ‘business as usual’ elements for both teachers and students. Similarly, current assessment plans largely focus on digitising existing time-limited exam practices, although finding alternatives is always recommended.

We might consider this a ‘first stage’ in moving online, the analogy being with the well-known JISC model of stages in development of digital literacies (see image on right). In this model, academics may soon move on from issues of access to tools and simple how-to skills guides. The next stage is likely to how to enhance their online teaching and learning practice and towards attributes such as their role as online teachers. In parallel, our students and staff may soon demand a more sophisticated ‘next stage’ learning experience.

As we progress, learning design becomes especially important. In order to develop true ‘online learning’ in even a simple form, courses will have to be redesigned around online activities. This is likely to be an opportunity to create richer learning designs, and for academics to consider a wider range of practices. Although academics may now have a motivation (albeit external) and focus, colleagues will still be stressed and time-limited, so any (re)design will require very simple workflows, step-by-step guides, checklists and so on.

At UCL and across the HE sector, the ABC ’sprint’ method of learning design has proved a well-evaluated, simple, engaging and productive practical framework to guide academic colleagues this process. The method is built around a collaborative and quite intensive 90’ workshop in which modules teams work together to produce a paper-based storyboard describing the student journey. The key pedagogical core of ABC is an operationalisation the six Learning Types framework of Prof Diana Laurillard (UCL Institute of Education). Using Learning Types has proved a remarkably robust and accessible route into teaching and learning discussion and reflection and shows how pedagogically-informed rapid development learning design is achievable.

The main issue is that ABC was designed as a social, face to face, group-based activity, so we have to rethink how the pedagogic ideas might be presented. The UCL ABC team, working with our ABCtoVLE Erasmus+ project partners are currently investigating online alternatives, with an aim to producing a Toolkit that institutions can localise and use in their own context. Some practicalities will be discussed in the next blog post, but from our experience in the European project we believe ABC should be considered not in isolation but as part of a process of institutional engagement. The full toolkit will most likely comprise three parts,

Pedagogy – Present the Learning Types to academic colleagues as a common language. As we can’t do this as a workshop, we are working on different approaches (see next post).

Technology – What tools are available to teachers? The Tool Wheel is a quick visual approach, linking pedagogy to technology. There are two stages for institutional support teams that can be can be done in either order,

  • Sketch a ‘snapshot’ of supported technologies, comprising the VLE, videoconferencing/virtual classroom, media tools etc. using the App wheel template, the ‘wheel of opportunities’.
  • Make a list of the tools identified and link to local support documentation (videos, web sites, tool guides, ‘how-to’s etc) – can also link to external guides from Canvas, Moodle etc.

Practice – What is effective pedagogic practice using these tools? We want to make use of the guides etc produced by the domain. We are redeveloping the Erasmus+ project site to represent good practice in the six learning types. The project team will list any resources we find that address each of the types. The first stage is to curate, we may be able to write something more definitive in the final report. More information to follow.