Digital Education team blog
  • We support Staff and Students using technology to enhance education at UCL.

    Here you'll find updates on institutional developments, projects we're involved in, updates on educational technology, events, case studies and personal experiences (or views!).

  • Subscribe to the Digital Education blog

  • Meta

  • Tags

  • A A A

    Archive for the 'e-Assessment' Category

    Jisc student digital tracker 2017 and BLE consortium

    By Moira Wright, on 10 August 2017

    computer-767776_1920UCL participated in the 2017 Jisc Digital Student Tracker Survey as part of a consortium with the Bloomsbury Learning Environment (BLE) made up of SOAS, Birkbeck, LSHTM and RVC. 74 UK institutions ran the tracker with their students collecting 22,593 student responses, while 10 international universities collected an additional 5,000 student responses

    We were the only consortium to participate in the survey and had come together as a result of institutional surveys, such as the National Student Survey, meaning that the time available to run it independently was short (a month) and we therefore felt that our individual sample sizes would be too small. We treated the survey as a pilot and advertised a link to it on each College’s Moodle landing page as well as some promotion via social media and the Student Unions. The survey generated 330 responses, which given our constraints was much more than we expected.

    The survey comprises five broad areas: Digital access, digital support and digital learning. Most questions were quantitatively recorded, but there were four open questions, which produced qualitative data. We were also able to choose two additional questions to the survey and we selected e-assessment, since that was a previous shared enhancement project (see www.bloomsbury.ac.uk/assessment) and Moodle, since all members of the consortium use the platform for their Virtual Learning Environment (VLE).

    Once the survey closed and we had access to the benchmarking report we ran a workshop for representatives from each of the Colleges in July 2017 whereby the results corresponding to the survey’s open questions were analysed in institutional groups, which facilitated interesting discussions over commonalities and potential implications.

    Sarah Sherman, the BLE Manager and myself, have been working to produce a report which will examine our collective responses to the survey in comparison with the national survey population with a recommendation that individual Colleges independently analyse their own results in more detail. For confidentiality, each College will be presented with a version of this document, which contains the relevant data for their institution only and not the complete BLE data set. A disadvantage of the consortium approach was that we were not able to benchmark individual Colleges to the survey population as the resources would not allow for this. In the future, the participating Colleges may wish to run the survey individually rather than as part of a collective as it was not possible to conduct deep analysis with this data set. 

    markus-spiske-221494

    Although the sample size collected by the Bloomsbury Colleges was small and not statistically viable, there is much we can extract and learn from this exercise. For the most part, our collective responses tended to fall within the margins set by the national survey population, which means we are all at a similar phase in our student’s digital capability and development.

    You will have to wait for the full report for more information on the UCL data collected but just to whet the appetite you can see the key findings from Jisc in this 2 page report: Student digital experience tracker at a glance .

    Finally, you can see this collection of case studies, which features the Bloomsbury Colleges consortium, here.

    Please get in touch with me if you would like to get involved (moira.wright @ ucl.ac.uk)

    Sarah Sherman and Moira Wright

    Jisc/ NUS student digital experience benchmarking tool 

    Jisc guide to enhancing the digital student experience: a strategic approach

     

    Turnitin assignments not resetting properly – issue now resolved!

    By Janice K M Kiugu, on 31 July 2017

    At the start of the academic year and in preparation for the next cohort of students on a course, staff are required to ‘Reset’ their Moodle courses. This removes students work and grades but leaves course resources and activities in place.

    An issue was identified at the end of July that was affecting Turnitin assignments when a Moodle course was ‘Reset’. This has now been resolved. Staff can now reset their Moodle courses. HOWEVER, the process of resetting courses has changed slightly so please read through the guidance provided via this link carefully, paying particular attention to step 4 of the process.

    If you reset your course/s containing Turnitin assignments before 11th August, the Digital Education team suggest resetting them again, to ensure the issue described below does not occur.

    *********************************************************************

    An issue has been identified that is affecting Turnitin assignments when a Moodle course is ‘Reset’.

    At the start of the academic year and in preparation for the next cohort of students on a course, staff are required to ‘Reset’ their Moodle courses. This removes students work and grades but leaves course resources and activities in place.

    Issue

    The ‘Reset’ function in Moodle normally creates a new class ID for a Turnitin assignment and staff should then be able to edit the assignment settings accordingly.  The issue that has been identified is that resetting the course seems to ‘lock’ the anonymous marking setting to  ‘Yes’  making it un-editable.  However, even if the Post date is edited and no submission has been made, student’s names are visible.

    We have reported the issue to Turnitin and they have acknowledged that there is problem and indicated that they hope to have a solution we can implement by the end of the week.

    Action Required (temporary workaround)

    We recommend, where feasible, that you wait until we have a fix in place and refrain from ‘resetting’  your course until we advise otherwise.

    Staff who have reset their courses in preparation for the next cohort of students or any staff planning to reset their courses before the issue is resolved should take the following steps:

    After resetting your Moodle course:

    • Delete the Turnitin assignment(s) that currently exist
    • Create new Turnitin assignment (s) with the required settings

      Guidance and instructions on creating Turnitin assignments are available from our Moodle Resource Centre: https://wiki.ucl.ac.uk/display/MoodleResourceCentre/M20+-+Turnitin+Assignment

    We apologise for the inconvenience caused and will advise when the issue has been resolved.

    If you have any questions of concerns, please contact the Digital Education team by emailing
    digi-ed@ucl.ac.uk

     

    Moving to Turnitin Feedback Studio

    By Janice K M Kiugu, on 5 June 2017

    A few months ago we alerted you to the fact that Turnitin will be moving all users to its new grading interface known as Feedback Studio.

    At present, users can toggle between using Turnitin’s old interface – Turnitin Classic and its latest version Feedback Studio. From 1st August 2017, all users will have to use Feedback Studio to view originality reports, grade work and provide feedback.

    Though a number of you have already moved to using Feedback Studio as it is the default version available on UCL Moodle, some staff have chosen to revert back to the Classic version. In preparation for the switch off of the ‘Classic’ version, we recommend that all Moodle users have a go at using Feedback Studio.

    The 2-minute video below is a brief walkthrough of Turnitin Feedback Studio.


    Learn more about Feedback Studio

    • Quick Tips for mastering Feedback Studio
    • Trial Feedback Studio: Please note that not all functionality is enabled on this demo version. You can however enable feedback studio on assignments that you may currently be in the process of marking by clicking on the ‘Try Turnitin feedback Studio’ link located at the top of an open assignment.
    • For additional guidance on using Feedback studio see our user guides
    • Watch this video comparison between Feedback Studio and Turnitin Classic
    • View our past communication regarding the move to Feedback Studio

    Moving to Turnitin Feedback Studio – multiple ways to mark with an updated look

    By Annora Eyt-Dessus, on 28 April 2017

    From summer 2017 Turnitin will be moving all users to their updated viewing and grading tool, Feedback Studio. Most UCL staff and students are already using Feedback Studio so will not see any change, but if you’re still using the ‘Classic’ version of the tool you will no longer have this option from late July 2017. You’ll still find the functionality you’re used to, but with an updated look and feel.

    So what does Feedback Studio offer? For a quick tour of the new features of Feedback Studio, and differences with the ‘Classic’ version, you can watch the short video above from Turnitin. (NB. Multiple markers feature will not be enabled initially, and the section shown 4:00mins+ is not relevant to Turnitin through Moodle.)

    Beyond being able to view and navigate similarity reports, it also offers staff a variety of ways to mark – including audio comments, rubrics and saved re-usable comments. Most of this functionality has been available to UCL staff for some time, but Turnitin have been working on the design of their interface.

    For instance, selection of the wide variety of tools Turnitin offers is now done by using icons set alongside the paper for easier faster marking. You may also be looking for a ‘Save’ button, but Turnitin now saves your comments as you move between papers using the arrows in the top right of the screen.

    This new version also aims to be more accessible, with viewing and grading easier on a greater variety of devices using a responsive design, as well as for those using screen readers and keyboards for navigation.

    If you want to try out the new version of Feedback studio, without logging in to Moodle and setting up an assignment, you can explore an interactive demo from Turnitin.

    Universities across the UK will soon be making the change as well as UCL, and we had the opportunity to hear from colleagues at the University of Kent, in partnership with Turnitin, at the recent MoodleMoot.ie conference earlier this April. They spoke of their success in moving to online marking in conjunction with the move to the new Feedback Studio, with over 70% overall of all marks now returned online. Key elements in their success were offering guidance for both staff and students (ours can be found here for staff and students), and offering repeated reminders of the change throughout the summer, so that all staff had a chance to be made aware.

    If you have any concerns or questions about this change, please consult the guidance, and email digi-ed@ucl.ac.uk with any issues.

    UCL 2034: Improving the ‘Student Experience’ with Digital Exams

    By Karen A M Shackleford-Cesare, on 6 April 2017

    Rationale

    One may argue that UCL students are over-assessed. But, regardless of your take on this, it stands to reason that where examinations are used to assess learning, it is worth investing in delivery systems that give students the best possible experience and outcomes. These systems could conceivably have the following aims, to:

    1. Improve the functionality, usability and accessibility of the questions on the exam
    2. Offer greater flexibility in terms of the time, place and resources needed to host an exam
    3. Improve the support given to both students sitting exams and the staff facilitating and/or grading them
    4. Potentially reduce the time between sitting some exams and obtaining results/feedback
    5. Potentially increase transparency in marking and confidence in the grades awarded
    6. Reduce potential for errors in marking and from transcribing/transferring marks
    7. Reduce the workloads of both academic and administrative staff
    8. Reduce costs around printing, transporting, securing and storing paper

    Acknowledging that assessment and feedback are ongoing areas of student concern, and that the Education Strategy commits us to addressing and resolving persistent challenges in this area (https://www.ucl.ac.uk/teaching-learning/education-strategy), Brunel University’s approach to digitising some examinations is interesting.

    E-Assessment Management at Brunel

    Whereas UCL uses the quiz tool in Moodle for online exams, Brunel University has subscribed to a cloud-based digital exam platform called WISEflow for theirs. Like Turnitin this subscribed service can be linked to various VLEs including Moodle. In the case of Brunel, this happens to be Blackboard. Tutors create their quizzes/exams, which can be accessed indirectly via a link on a course page in their VLE or directly by logging into the secure WISEflow website.

    WISEflow-Select Login

    WISEflow provides tutors with 50 varied question types with quite sophisticated features allowing for different media to be embedded and for example, virtual tools, such as a ruler and a protractor to be used to measure the lines and angles of diagrams in questions. It also supports essay questions that allow a student to respond with a 1,000 to 3,000 word essay. This advantageous for exams in subjects such as law, history and literature, etc. WISEflow also has a number of features to ensure that student’s responses aren’t lost whilst being written.

    According to Niels Berg Conradsen of UNIwise,

    “WISEflow features a very robust lockdown browser module. It carries a text editor in the lockdown browser that also serves as a word processor. The students can structure their essay with headlines and even embed pictures from their webcam. Moreover, the student has a revision tool at their disposal, so they can revert the essay to former editions if they so desire”.

    This platform is sufficiently secure to support bring-your-own-device (BYOD) examinations. At a 1-day event at Brunel on March 17, 2017 attendees, (myself included), had an opportunity to hear positive feedback from Brunel’s staff and students about their experience using WISEflow, and to use it ourselves on our own devices.

    WISEflow enables tutors to create and manage digital exams, which they can assign to specific students. They can then mark the materials that the students submit in WISEflow. The marking tools are similar to those in the Moodle and Turnitin assignments. WISEflow can also be used for the submission of coursework.

    Benefits of digital exams generally for various stakeholders are listed below and may be also achieved with Moodle quizzes, (although in Moodle’s case additional software would be needed to lock the browser).

    Advantages for students:

    • It’s feasible for students to take exams remotely
    • No need to write long hand for hours (increasingly students may have little sustained handwriting practice)
    • It’s much easier to correct mistakes and make changes to responses on online exam scripts than on paper
    • It’s feasible for students to get feedback beyond just a grade on exams taken
    • Students may be able to get their results faster
    • It’s feasible to take the exam on a familiar device, their own laptop
    • Accessibility features easier to incorporate

    Advantages for tutors, administrators, invigilators:

    • Exam papers don’t have to be printed, packaged, secured and distributed to exam centres
    • No challenging handwriting to decipher
    • No paper scripts to collect, secure or transport
    • No scripts to store for the short to long term
    • Scripts can be printed as required
    • Tools exist for invigilators to monitor students’ progress through a paper they’re sitting
    • Tools exist for tutors to analyse students’ responses to each question
    • Easy to facilitate access to scripts by moderators, second markers and external examiners
    • The cost of a streamlined infrastructure for online exams may not exceed that of the current paper based examination system in place.

    Importantly, Brunel also benefited from the involvement and support of their Registry and Examinations Office in the launch of this project.

    What WISEflow can do for you…

    There are other Providers of Digital Examination Systems that may also be considered.

    Some Cons of Acquiring Another App

    This piece has highlighted many of the pros of acquiring specialist, purpose-built kit. However, some of the problems that would need to be addressed include:

    • Providing students with a “one-stop shop” for accessing their results and feedback. UCL has part sponsored the development of the MyFeedback tool in Moodle for this purpose but could it retrieve this data from a third party product?
    • Additional training and support needed
    • The integration of a new app with Moodle
    • Identifying and equipping enough rooms with adequate WiFi and power points to support BYOD
    • Adequate provision of suitable computer suites
    • Cost

    Cost Differentials

    It would be informative to compare the estimated cost of running paper-based exams with that of online exams. Instinctively, one may focus on the costs associated with the actually staging of an exam. By so doing, the paper option may seem much cheaper, if only because computing paraphernalia isn’t required. However, costs may start to even out when preparatory printing, storing and distribution expenses, (not to mention the man-hours needed), are taken into account. Plus, cost associated with moving, securing and storing hard-copy exam scripts.

    Where the computing infrastructure is exists for other purposes and would be present even if no online exams to use it, then one could argue in favour of discounting the cost it may otherwise present.

    Observations by Attendees

    See what others had to say:


    You said, we did

    By Jessica Gramp, on 22 March 2017

    A number of recommendations emerged from the E-Learning Reports developed in 2013 across the Bartlett, Engineering and Maths and Physical Sciences (BEAMS) departments. Here’s what you asked for and what the Digital Education Advisor for the faculty arranged in response, in collaboration with staff from across the Information Services Division.

     

    quoteYou wanted to import module timetable information from the Common Timetable into Moodle.
    We developed a Common Timetable iCal feed to import module timetables for displaying in the Moodle calendar.

     

    quoteYou wanted a simplified quiz creation process with guidelines and checklists for importing questions.
    We purchased the Moodle Word Table format plugin to help staff quickly develop quizzes with simple question types (not calculated or drag and drop) in Word, including those with images and LaTeX.

     

    quoteYou wanted us to run staff workshops and demos to increase knowledge of e-learning tools & their potential use.
    We ran workshops across the faculties and in individual departments catered to the needs of the departments.

     

    quoteYou wanted to simplify the process for exporting grades out of Moodle and into Portico.
    We imported the UCL student number into Moodle and added this column to the Moodle Gradebook export, simplifying the uploading of grades from Moodle into Portico. A video explaining how to move grades from Moodle to Portico is now available on the UCL E-Learning Wiki – a space for staff to share their e-learning practice: