Digital Education team blog
  • We support Staff and Students using technology to enhance education at UCL.

    Here you'll find updates on institutional developments, projects we're involved in, updates on educational technology, events, case studies and personal experiences (or views!).

    Subscribe to our elearning newsletters.

  • Subscribe to this blog

  • Meta

  • Tags

  • A A A

    UCL 2034: Improving the ‘Student Experience’ with Digital Exams

    By Karen Shackleford-Cesare, on 6 April 2017

    Rationale

    One may argue that UCL students are over-assessed. But, regardless of your take on this, it stands to reason that where examinations are used to assess learning, it is worth investing in delivery systems that give students the best possible experience and outcomes. These systems could conceivably have the following aims, to:

    1. Improve the functionality, usability and accessibility of the questions on the exam
    2. Offer greater flexibility in terms of the time, place and resources needed to host an exam
    3. Improve the support given to both students sitting exams and the staff facilitating and/or grading them
    4. Potentially reduce the time between sitting some exams and obtaining results/feedback
    5. Potentially increase transparency in marking and confidence in the grades awarded
    6. Reduce potential for errors in marking and from transcribing/transferring marks
    7. Reduce the workloads of both academic and administrative staff
    8. Reduce costs around printing, transporting, securing and storing paper

    Acknowledging that assessment and feedback are ongoing areas of student concern, and that the Education Strategy commits us to addressing and resolving persistent challenges in this area (https://www.ucl.ac.uk/teaching-learning/education-strategy), Brunel University’s approach to digitising some examinations is interesting.

    E-Assessment Management at Brunel

    Whereas UCL uses the quiz tool in Moodle for online exams, Brunel University has subscribed to a cloud-based digital exam platform called WISEflow for theirs. Like Turnitin this subscribed service can be linked to various VLEs including Moodle. In the case of Brunel, this happens to be Blackboard. Tutors create their quizzes/exams, which can be accessed indirectly via a link on a course page in their VLE or directly by logging into the secure WISEflow website.

    WISEflow-Select Login

    WISEflow provides tutors with 50 varied question types with quite sophisticated features allowing for different media to be embedded and for example, virtual tools, such as a ruler and a protractor to be used to measure the lines and angles of diagrams in questions. It also supports essay questions that allow a student to respond with a 1,000 to 3,000 word essay. This advantageous for exams in subjects such as law, history and literature, etc. WISEflow also has a number of features to ensure that student’s responses aren’t lost whilst being written.

    According to Niels Berg Conradsen of UNIwise,

    “WISEflow features a very robust lockdown browser module. It carries a text editor in the lockdown browser that also serves as a word processor. The students can structure their essay with headlines and even embed pictures from their webcam. Moreover, the student has a revision tool at their disposal, so they can revert the essay to former editions if they so desire”.

    This platform is sufficiently secure to support bring-your-own-device (BYOD) examinations. At a 1-day event at Brunel on March 17, 2017 attendees, (myself included), had an opportunity to hear positive feedback from Brunel’s staff and students about their experience using WISEflow, and to use it ourselves on our own devices.

    WISEflow enables tutors to create and manage digital exams, which they can assign to specific students. They can then mark the materials that the students submit in WISEflow. The marking tools are similar to those in the Moodle and Turnitin assignments. WISEflow can also be used for the submission of coursework.

    Benefits of digital exams generally for various stakeholders are listed below and may be also achieved with Moodle quizzes, (although in Moodle’s case additional software would be needed to lock the browser).

    Advantages for students:

    • It’s feasible for students to take exams remotely
    • No need to write long hand for hours (increasingly students may have little sustained handwriting practice)
    • It’s much easier to correct mistakes and make changes to responses on online exam scripts than on paper
    • It’s feasible for students to get feedback beyond just a grade on exams taken
    • Students may be able to get their results faster
    • It’s feasible to take the exam on a familiar device, their own laptop
    • Accessibility features easier to incorporate

    Advantages for tutors, administrators, invigilators:

    • Exam papers don’t have to be printed, packaged, secured and distributed to exam centres
    • No challenging handwriting to decipher
    • No paper scripts to collect, secure or transport
    • No scripts to store for the short to long term
    • Scripts can be printed as required
    • Tools exist for invigilators to monitor students’ progress through a paper they’re sitting
    • Tools exist for tutors to analyse students’ responses to each question
    • Easy to facilitate access to scripts by moderators, second markers and external examiners
    • The cost of a streamlined infrastructure for online exams may not exceed that of the current paper based examination system in place.

    Importantly, Brunel also benefited from the involvement and support of their Registry and Examinations Office in the launch of this project.

    What WISEflow can do for you…

    There are other Providers of Digital Examination Systems that may also be considered.

    Some Cons of Acquiring Another App

    This piece has highlighted many of the pros of acquiring specialist, purpose-built kit. However, some of the problems that would need to be addressed include:

    • Providing students with a “one-stop shop” for accessing their results and feedback. UCL has part sponsored the development of the MyFeedback tool in Moodle for this purpose but could it retrieve this data from a third party product?
    • Additional training and support needed
    • The integration of a new app with Moodle
    • Identifying and equipping enough rooms with adequate WiFi and power points to support BYOD
    • Adequate provision of suitable computer suites
    • Cost

    Cost Differentials

    It would be informative to compare the estimated cost of running paper-based exams with that of online exams. Instinctively, one may focus on the costs associated with the actually staging of an exam. By so doing, the paper option may seem much cheaper, if only because computing paraphernalia isn’t required. However, costs may start to even out when preparatory printing, storing and distribution expenses, (not to mention the man-hours needed), are taken into account. Plus, cost associated with moving, securing and storing hard-copy exam scripts.

    Where the computing infrastructure is exists for other purposes and would be present even if no online exams to use it, then one could argue in favour of discounting the cost it may otherwise present.

    Observations by Attendees

    See what others had to say:


    Assessment & feedback – links from the Joint Faculty E-Learning Forum

    By Mira Vogel, on 20 November 2014

    This morning UCL’s Joint Faculty E-Learning Forum – that’s Arts & Humanities and Social & Historical Sciences – met for the second time. The first meeting had focused on assessment and feedback, so ELE gave a brief presentation on our actions since then and recommended avenues colleagues in departments could pursue.

    This post provides some resources to support those, which you can also find in the presentation from the meeting below.

    Student engagement with assessment feedback

    Our discussions with students suggested low awareness of feedback release dates and our investigations revealed patchy engagement with feedback.

    • What can we find out about student engagement with feedback? Turnitin provides some basic information to staff about student engagement with feedback. Each assignment inbox has a student response column containing either a dot (no engagement) or in the case of students who reviewed their graded paper in GradeMark for longer than 30 seconds, an icon of a person with a check mark. For a fuller picture of how long it takes students to visit their feedback, check more than once – for example, one day, one week and one month after feedback is released. Moodle Assignment has a different process: in each assignment’s Settings block, click Logs and filter actions by View.
    • Since Moodle and Turnitin don’t alert the students automatically, it’s important to use the News Forum or other communication channel to draw students’ attention to feedback when it becomes available.
    • ELE have guidance for Moodle Assignment on how to delay providing a numeric mark, to encourage students to engage with feedback. With Turnitin this cannot be done as a bulk process, though there are workarounds.
    • Turnitin UserVoice and Moodle Tracker are available for users to contribute and vote for ideas (to create an account on Turnitin UserVoice, enter your UCL email and you should get an option to create account). For example, on Turnitin UserVoice see ‘Feedback released prior to grades’, with a corresponding item on the Moodle Tracker. Do contribute your ideas and votes.
    • The solution to low engagement may lie in rethinking assessment design so as to incorporate dialogue about earlier feedback. Jisc has gathered assessment and feedback principles and provides support for the design of assessment such as the University Of Ulster’s Viewpoints. Where there is anonymous submission, ELE has guidance on Turnitin aligned with the marking policy which enables you to lift anonymity between marking and external examining, so as to enable dialogue with students. We are in the process of preparing corresponding guidance for Moodle Assignment.

    External examining

    Efficiency gains? Efficiency losses?

    Advocacy with third party software providers

     

     

     

     

     

    Finding good example rubrics

    By Rod Digges, on 8 February 2013

     

    A well constructed rubric is a great way of  communicating, both to students and fellow assessors, exactly how the marking of an assignment is being approached but the construction of effective and comprehensive rubrics can be very time consuming.

    Rubistar rubrics

    Recently, while looking through the instructions for entering rubrics in the peer assessment workshop activity in Moodle, I came across a link to a site with hundreds of example rubrics.
    Usefully, the site also allows users to construct their own rubrics in a rich variety of subject areas. The site is free, easy to use and well worth a look for anyone needing inspiration in this area.

    http://rubistar.4teachers.org/