X Close

Digital Education team blog

Home

Ideas and reflections from UCL's Digital Education team

Menu

Archive for the 'Turnitin' Category

Marking 24 hour exams

Steve Rowett5 May 2020

Please note this post is being regularly updated with additional resources.
+ New on Friday 8 May: Guide to online marking from Mary Richardson
+ New on Friday 8 May: Microsoft Drawboard PDF demo from Dewi Lewis, UCL Chemistry
+ New on Friday 15 May: Updated details on Microsoft Drawboard PDF

The move to online 24 hour assessments that replace traditional exams leads to a challenge for those that have to grade and mark the work.

We start from a place of two knowns:

  • Students are submitting work to Turnitin in Moodle during the 24 hour window; and
  • Final grades need to be stored in Portico, our student records system.

But in between those two endpoints, there are many different workflows by which marking can take place. These are set out by the UCL’s Academic Manual but encompasses a range of choices, particularly in how second marking is completed. One key difference between regular courseworks is that this is not about providing feedback to students, but instead about supporting the marking process, the communication between markers and the required record of the marking process. At the end of the marking process departments will need to ensure that scripts are stored securely but can be accessed by relevant staff as required, much in line with requirements for paper versions over previous years.

Neither SRS nor Digital Education mandate any particular way that marking should take place and there is considerable flexibility for departments to use processes that work best for them. So we are suggesting a menu of options which provide a basis for departments to build on if they so choose. We are also running daily training sessions which as listed at the foot of this document.

The menu options are:

  • Markers review the scripts and mark or annotate them using Turnitin Feedback Studio
  • Digital Education will provide PDF copies of scripts for departments to annotate using PDF annotation software on a computer or tablet device.
  • Markers review the scripts using Turnitin Feedback Studio, but keep a ‘marker file’ or notes and comments on the marking process.
  • Markers print the scripts and mark them, then scan them for storage or keep them for return to the department on paper.

The rest of this post goes into these options in more detail.


Turnitin Feedback Studio

Turnitin Feedback Studio provides a web-based interface where comments can be overlaid on a student’s work. QuickMarks provide a bank of comments that occur regularly that can be just drag and dropped onto the work. In addition, the traditional Turnitin Similarity Report is also available. This method probably works best for text-based documents like essays and reports. Turnitin is integrated into Moodle and set up for you as part of the exam process for students to submit their work, but it’s your choice if you wish to use the marking tools available after the work has been submitted. The Turnitin submission boxes have been set up for you, and we ask that you don’t change the settings or set up any grading templates to mark student work before submission, as this could prevent students from submitting.

You can also allocate marks using grading forms or rubrics.  On the whole we think that these could be a bit of a ‘sledgehammer to crack a nut’ solution for a single paper, but it is an option available to you if you are familiar with them and you have a more granular set marking criteria for each question. We recommend hiding the assignment before adding the grading form or rubric so that students cannot see it.

If you want to know if this method is for you, you can watch a short video demo or  try marking up an example paper provided by Turnitin. A video tailored to UCL’s 24 hour exam process is given below. This video has captions.

Things to think about with this approach:

  • Rubrics and grading forms take a little bit of setting up, and are probably best used where you have previous experience in doing them.
  • In some exams it is common to put a mark (e.g. a tick) on each page to indicate that the page has been read. To replicate this you might define a QuickMark called ‘page read’ and put it on each page, or annotate with the same words
  • The marked paper often becomes a resource to go back to if there are any errors or omissions in the grading process. You might wish to both write the marks on the paper using the annotation tools or in the general feedback area, and also lodge them in a spreadsheet for uploading to Portico.
  • Turnitin does not support double blind marking effectively. It is rarely used for paper-based exams (since the second marker could always see the markings of the first marker on the paper) but if it was needed one marker could mark online and the second could download the papers for offline marking (e.g the ‘marker form’ method below).

You can view additional guidance on using Turnitin Feedback Studio.


Annotation using PDF documents

Where you annotation needs are more sophisticated, or you want to ‘write’ on the paper using a graphics tablet or a tablet and pencil/stylus, then this option may suit you better.

Upon notification (notification form) Digital Education will supply your department with PDF copies of the students’ work, uploaded to a OneDrive account set up by your department.

For this to happen, Exam Liaison Officers / Teaching Administrators will need to set up a OneDrive folder and notify Digital Education that they wish to have PDF copies of the files. We have a video tutorial (with captions) on this process below.

You can then use tools you already have or prefer to use to do your marking. There is more flexibility here, and we will not be able to advise and support every PDF tool available or give precise instructions for every workflow used by departments, but we give some examples here.

Marking on an iPad using OneDrive

Many staff have reported using an iPad with Apple Pencil or an Android tablet with a stylus to be a very effective marking tool. The Microsoft OneDrive app supports both platforms and provides rapid access to scripts and some annotation tools as shown in the video below (which also has captions). The OneDrive app is free, and connects to your UCL OneDrive account via Single Sign On.

There’s further guidance from Microsoft on each individual annotation tool.

The Apple Files app can also connect to OneDrive and has a similar (and perhaps more powerful) annotation tool. Thanks to David Bowler for mentioning this in the first comment on this blog post.

Marking on a Mac using Preview

Preview on a Mac is often taken for granted but is actually quite a sophisticated tool and includes some basic annotation functions. Here is some guidance from Apple on using it.

Marking on a PC or Surface Pro using Microsoft Drawboard PDF

Microsoft Drawboard PDF is a very comprehensive annotation tool, but is only available for Windows 10 and is really designed to be used with a Surface Pro or a desktop with a graphics tablet. Dewi Lewis from UCL Chemistry has produced a video illustrating the annotation tools available and how to mark a set of files easily. UCL does not have a site-wide licence for Drawboard PDF, but it is available at a very modest price if departments choose to buy it.

Marking on a PC, Mac or Linux machine using a PDF annotation program.

Of course there are plenty of third party tools that support annotating PDF documents. Some requirement payment to access the annotation facilities (or to save files that have been annotated) but two that do not are Xodo and Foxit PDF.

Things to think about with this approach:

  • Your marking process: if you use double blind marking you might need to make two copies of the files, one for each marker. If you use check marking then a single copy will suffice.
  • You will need to ensure the files are stored securely and can be accessed by the relevant departmental staff in case of any query. You might share the exam submission files with key contacts such as teaching administrators or directors of teaching.
  • Some of the products listed above have a small charge, as would any stylus or pencil that staff would need. These cannot be supplied centrally, so you may need a process for staff claiming back the costs from departments.

Using a ‘marker file’

Accessing the students’ scripts is done using Turnitin in Moodle, which allows all the papers to be viewed online individually or downloaded in one go. Then a separate document is kept (either one per script, or one overall) containing the marks and marker feedback for each comment. If double-blind marking is being used, then it is easy to see that two such documents or sets of documents could be kept in this way.


Printing scripts and marking on paper

Although we have moved to online submission this year, colleagues are still welcome to print documents and mark on paper. However there is no central printing service available for completed scripts to be printed, and this would have to be managed individually or locally by departments.


The evidence about marking online

In this video Dr Mary Richardson, Associate Professor in Educational Assessment at the IOE, gives a guide to how online marking can differ from paper-based marking and offers some tips for those new to online marking. The video has captions.


Training sessions and support

Digital Education will be running daily training sessions for teachers covering the ground in this blog post. These will run at 12-1pm every weekday from Tuesday 12 May.

No booking necessary.

We are also providing additional support for students during the exam period. Our support hours will be (UK time):

  • Monday 11.30am-8.30pm
  • Tuesday-Thursday 8am-8pm
  • Friday: 7.30am-3.30pm

Details of support mechanisms are given in the exam section on each Moodle module where an exam is taking place.

Moving activities online – as easy as ABC?

Clive Young10 March 2020

ABC and learning types

As we focus on Teaching continuity, UCL’s ABC method of learning design can help us consider how to move learning activities online. 

Many colleagues will already be familiar with the ABC sprint workshops for programme and module (re) design. During the high-energy 90’ workshop, academic teams work together to create a visual ‘storyboard’ showing the type and sequence learning activities required to meet the module’s learning outcomes and also how these will be assessed. Over 1000 UCL colleagues have now participated in ABC workshops since we started in 2015 and report it is particularly useful for new programmes or those moving to an online or more blended format.

The storyboard represents the learner journey and is constructed from pre-printed cards representing six types of learning.

The learning types are derived from the highly respected ‘Conversational Framework’ model of adult learning developed by Prof Diana Laurillard of the Institute of Education, UCL.

Video: Prof Laurillard introduces the Conversational Framework (Only the title is in Italian!)

The ABC cards list ‘conventional’ and digital examples activity associated with each of Prof Laurillard’s learning types, but teams are able and encouraged to add their own activities to the cards. Extensive testing at UCL and elsewhere has showed the creative hands-on, analogue format of the workshop stimulates a wide-ranging discussion. This includes the purpose of the course or programme, teaching methods, alternative technologies and assessment methods and above all the student experience. Even if you are not able to organise a ‘full’ ABC learning design event for your team, the cards themselves can help you identify digital alternatives to current activities.

Image: Example activities from the ABC cards.

Video: Prof Laurillard introduces the six learning types (2′) Note: ‘Inquiry’ is used here instead of ‘Investigation’.

How can the  six learning types guide us to consider digital alternatives to ‘conventional’ teaching and learning?


Acquisition

What learners do when they read books and articles, listen to lectures and podcasts, watch demos or videos. In this way learners acquire new concepts, models, vocabulary, models, and methodologies. Acquisition should be reflective as learners align new ideas to their existing knowledge. Conventional methods often include face-to-face presentations, demos and master classes.

Moving acquisition online: reading multimedia, websites, digital documents and resources listening to podcasts, webcasts watching animations, videos. Online quizzes can be used to check learner progress.

Key UCL tools:


Investigation

Encourages the learner to take an active and exploratory approach to learning, to search for and evaluate a range of new information and ideas. Students are guided to analyse, compare and critique the texts, data, documents and resources within the concepts and ideas being taught.

Moving investigation online: in many disciplines using digital resources and analytical tools are already part of students’ activities.

Key UCL tools:


Practice

Enables knowledge to be applied in context. The learner modifies actions according to the task and uses feedback to improve. Feedback may come from self-reflection, peers, the teacher, or from the activity outcomes. Practice often includes significant face-to-face components including labs, field trips, placements, practice-based projects and face-to-face role-play and groupwork.

Moving practice online: The most challenging of the six activity types, some activities are hard to substitute without losing important learning outcomes. Videos of methods, simulations, models, sample data sets, image and video banks, online role-play and case studies may be used to address some of the learning aims. Online quizzes can be used to test application and understanding.

Key UCL tools:


Discussion

Requires the learner to articulate their ideas and questions, and to challenge and respond to the ideas and questions from the teacher, and/or from their peers. Conventionally this is achieved through face-to-face tutorials, seminars and class discussion.

Moving discussion online: There are a number of good online options, including Moodle discussion forums which can be real-time (synchronous) or run over an extended period (asynchronous). Online forums can be even more productive than conventional tutorials as more students may contribute. For a richer discussion, Blackboard Collaborate can be run as a synchronous session.

Key UCL tools:


Collaboration

Requires students to work together in small groups to achieve a common project goal. Building on investigations and acquisition it is about taking part in the process of knowledge building itself. Learning through collaboration therefore includes elements of discussion, practice, and production.

Moving collaboration online: Some parts of group and project working lend themselves to digital communication to help discussion and planning of project outputs. The practical elements depend on the discipline but in some areas it will be possible to build a joint digital output and complete the task entirely online.

Key UCL tools:


Production

How the teacher motivates the learner to consolidate what they have learned by articulating their current conceptual understanding and reflect how they used it in practice. Production is usually associated with formative and summative assessment and can cover a wide range of items; essays, reports, designs, performances, articles, models etc.

Moving production online: In some disciplines, digital representations are already common such as presentations, videos, slideshows, blogs and e-portfolios.

Key UCL tools:

Assessment

During the ABC workshop, assessment is usually addressed as a part of the (re) design process. Online formative assessment can be included in the learner experience using many of the tools and approaches listed above, such as Moodle Forum and Moodle Quiz. Online summative assessment is more complex and separate guidance is being prepared.

Turnitin and Moodle Assignment training

Eliot Hoving7 February 2019

The Digital Education team is running two new training courses, Hands on with Turnitin Assignment, and Hands on with Moodle Assignment. Each session is practical, from a staff and student perspective you will experience the process of submitting, marking, returning marks, engaging with feedback and managing records. Both courses are applicable to Tutors and Course Administrators new to online marking or needing to refresh their knowledge.

Register through the HR Single Training Booking System or follow the links below:
Hands on with Turnitin
Hands on with Moodle Assignment

Email digi-ed@ucl.ac.uk for more information or to inquire about specific training for your Department.

Jisc student digital tracker 2017 and BLE consortium

Moira Wright10 August 2017

computer-767776_1920UCL participated in the 2017 Jisc Digital Student Tracker Survey as part of a consortium with the Bloomsbury Learning Environment (BLE) made up of SOAS, Birkbeck, LSHTM and RVC. 74 UK institutions ran the tracker with their students collecting 22,593 student responses, while 10 international universities collected an additional 5,000 student responses

We were the only consortium to participate in the survey and had come together as a result of institutional surveys, such as the National Student Survey, meaning that the time available to run it independently was short (a month) and we therefore felt that our individual sample sizes would be too small. We treated the survey as a pilot and advertised a link to it on each College’s Moodle landing page as well as some promotion via social media and the Student Unions. The survey generated 330 responses, which given our constraints was much more than we expected.

The survey comprises five broad areas: Digital access, digital support and digital learning. Most questions were quantitatively recorded, but there were four open questions, which produced qualitative data. We were also able to choose two additional questions to the survey and we selected e-assessment, since that was a previous shared enhancement project (see www.bloomsbury.ac.uk/assessment) and Moodle, since all members of the consortium use the platform for their Virtual Learning Environment (VLE).

Once the survey closed and we had access to the benchmarking report we ran a workshop for representatives from each of the Colleges in July 2017 whereby the results corresponding to the survey’s open questions were analysed in institutional groups, which facilitated interesting discussions over commonalities and potential implications.

Sarah Sherman, the BLE Manager and myself, have been working to produce a report which will examine our collective responses to the survey in comparison with the national survey population with a recommendation that individual Colleges independently analyse their own results in more detail. For confidentiality, each College will be presented with a version of this document, which contains the relevant data for their institution only and not the complete BLE data set. A disadvantage of the consortium approach was that we were not able to benchmark individual Colleges to the survey population as the resources would not allow for this. In the future, the participating Colleges may wish to run the survey individually rather than as part of a collective as it was not possible to conduct deep analysis with this data set. 

markus-spiske-221494

Although the sample size collected by the Bloomsbury Colleges was small and not statistically viable, there is much we can extract and learn from this exercise. For the most part, our collective responses tended to fall within the margins set by the national survey population, which means we are all at a similar phase in our student’s digital capability and development.

You will have to wait for the full report for more information on the UCL data collected but just to whet the appetite you can see the key findings from Jisc in this 2 page report: Student digital experience tracker at a glance .

Finally, you can see this collection of case studies, which features the Bloomsbury Colleges consortium, here.

Please get in touch with me if you would like to get involved (moira.wright @ ucl.ac.uk)

Sarah Sherman and Moira Wright

Jisc/ NUS student digital experience benchmarking tool 

Jisc guide to enhancing the digital student experience: a strategic approach

 

Moving to Turnitin Feedback Studio

Janice Kiugu5 June 2017

A few months ago we alerted you to the fact that Turnitin will be moving all users to its new grading interface known as Feedback Studio.

At present, users can toggle between using Turnitin’s old interface – Turnitin Classic and its latest version Feedback Studio. From 1st August 2017, all users will have to use Feedback Studio to view originality reports, grade work and provide feedback.

Though a number of you have already moved to using Feedback Studio as it is the default version available on UCL Moodle, some staff have chosen to revert back to the Classic version. In preparation for the switch off of the ‘Classic’ version, we recommend that all Moodle users have a go at using Feedback Studio.

The 2-minute video below is a brief walkthrough of Turnitin Feedback Studio.


Learn more about Feedback Studio

  • Quick Tips for mastering Feedback Studio
  • Trial Feedback Studio: Please note that not all functionality is enabled on this demo version. You can however enable feedback studio on assignments that you may currently be in the process of marking by clicking on the ‘Try Turnitin feedback Studio’ link located at the top of an open assignment.
  • For additional guidance on using Feedback studio see our user guides
  • Watch this video comparison between Feedback Studio and Turnitin Classic
  • View our past communication regarding the move to Feedback Studio

Moving to Turnitin Feedback Studio – multiple ways to mark with an updated look

Annora Eyt-Dessus28 April 2017

From summer 2017 Turnitin will be moving all users to their updated viewing and grading tool, Feedback Studio. Most UCL staff and students are already using Feedback Studio so will not see any change, but if you’re still using the ‘Classic’ version of the tool you will no longer have this option from late July 2017. You’ll still find the functionality you’re used to, but with an updated look and feel.

So what does Feedback Studio offer? For a quick tour of the new features of Feedback Studio, and differences with the ‘Classic’ version, you can watch the short video above from Turnitin. (NB. Multiple markers feature will not be enabled initially, and the section shown 4:00mins+ is not relevant to Turnitin through Moodle.)

Beyond being able to view and navigate similarity reports, it also offers staff a variety of ways to mark – including audio comments, rubrics and saved re-usable comments. Most of this functionality has been available to UCL staff for some time, but Turnitin have been working on the design of their interface.

For instance, selection of the wide variety of tools Turnitin offers is now done by using icons set alongside the paper for easier faster marking. You may also be looking for a ‘Save’ button, but Turnitin now saves your comments as you move between papers using the arrows in the top right of the screen.

This new version also aims to be more accessible, with viewing and grading easier on a greater variety of devices using a responsive design, as well as for those using screen readers and keyboards for navigation.

If you want to try out the new version of Feedback studio, without logging in to Moodle and setting up an assignment, you can explore an interactive demo from Turnitin.

Universities across the UK will soon be making the change as well as UCL, and we had the opportunity to hear from colleagues at the University of Kent, in partnership with Turnitin, at the recent MoodleMoot.ie conference earlier this April. They spoke of their success in moving to online marking in conjunction with the move to the new Feedback Studio, with over 70% overall of all marks now returned online. Key elements in their success were offering guidance for both staff and students (ours can be found here for staff and students), and offering repeated reminders of the change throughout the summer, so that all staff had a chance to be made aware.

If you have any concerns or questions about this change, please consult the guidance, and email digi-ed@ucl.ac.uk with any issues.