Digital Education team blog
  • We support Staff and Students using technology to enhance education at UCL.

    Here you'll find updates on institutional developments, projects we're involved in, updates on educational technology, events, case studies and personal experiences (or views!).

  • Subscribe to the Digital Education blog

  • Meta

  • Tags

  • A A A

    Archive for the 'Case Studies' Category

    A good peer review experience with Moodle Workshop

    By Mira Vogel, on 18 March 2015

    Update Dec 2015: there are now three posts on our refinements to this peer feedback activity: one, two, and three.

    Readers have been begging for news of how it went with the Moodle Workshop activity from this post.

    Workshop is an activity in Moodle which allows staff to set up a peer assessment or (in our case) peer review. Workshop collects student work, automatically allocates reviewers, allows the review to be scaffolded with questions, imposes deadlines on the submission and assessment phase, provides a dashboard so staff can follow progress, and allows staff to assess the reviews/assessments as well as the submissions.

    However, except for some intrepid pioneers, it is almost never seen in the wild.

    The reason for that is partly to do with daunting number and nature of the settings – there are several pitfalls to avoid which aren’t obvious on first pass – but also the fact that because it is a process you can’t easily see a demo and running a test instance is pretty time consuming. If people try once and it doesn’t work well they rarely try again.

    Well look no further – CALT and ELE have it working well now and can support you with your own peer review.

    What happened?

    Students on the UCL Arena Teaching Associate Programme reviewed each others’ case studies. 22 then completed a short evaluation questionnaire in which they rated their experience of giving and receiving feedback on a five-point scale and commented on their responses. The students were from two groups with different tutors running the peer review activity. A third group leader chose to run the peer review on Moodle Forum since it would allow students to easily see each others’ case studies and feedback.

    The students reported that giving feedback went well (21 respondents):

    Pie chart - giving feedback

    Satisfaction with reviewing work – click to enlarge

    This indicates that the measures we took – see previous post – to address disorientation and participation were successful. In particular we were better aware of where the description, instructions for submission, instructions for assessment, and concluding comments would display, and put the relevant information into each.

    Receiving feedback also went well (22 respondents) though with a slightly bigger spread in both directions:

    Pie chart - receiving feedback

    Satisfaction with receiving reviews – click to enlarge

     

    Students appreciated:

    • Feedback on their work.
    • Insights about their own work from considering others’ work.
    • Being able to edit their submission in advance of the deadline.
    • The improved instructions letting them know what to do, when and where.

    Staff appreciated:

    This hasn’t been formally evaluated, but from informal conversations I know that the two group leaders appreciate Moodle taking on the grunt work of allocation. However, this depends on setting a hard deadline with no late submissions (otherwise staff have to keep checking for late submissions and allocating those manually) and one of the leaders was less comfortable with this than the other. Neither found it too onerous to write diary notes to send reminders and alerts to students to move the activity along – in any case this manual messaging will hopefully become unnecessary with the arrival of Moodle Events in the coming upgrade.

    For next time:

    • Improve signposting from the Moodle course area front page, and maybe the title of the Workshop itself, so students know what to do and when.
    • Instructions: let students know how many reviews they are expected to do; let them know if they should expect variety in how the submissions display – in our case some were attachments while others were typed directly into Moodle (we may want to set attachments to zero); include word count guidance in the instructions for submission and assessment.
    • Consider including an example case study & review for reference (Workshop allows this).
    • Address the issue that, due to some non-participation during the Assessment phase, some students gave more feedback than they received.
    • We originally had a single comments field but will now structure the peer review with some questions aligned to the relevant parts of the criteria.
    • Decide about anonymity – should both submissions and reviews be anonymous, or one or the other, or neither? These can be configured via the Workshop’s Permissions. Let students know who can see what.
    • Also to consider – we could also change Permissions after it’s complete (or even while it’s running) to allow students to access the dashboard and see all the case studies and all the feedback.

    Have you had a good experience with Moodle Workshop? What made it work for you?

    Meet Jess, Jack, Stuart & Heather – realistic voices for free* download

    By Jessica Gramp, on 3 March 2015

    I have recently started listening to my books and papers, rather than reading them. This frees me up to do other things while I listen, such as cook, take a bath or do some tidying up. It also gives my eyes a well needed break from staring at a computer screen or paper.

    As part of an online e-learning course I am helping to develop, I am using the TechDis Jess voice to provide audio files of the commentary, as an alternative to reading. I have had to tweak some of the text – for example, UCL needs to be written with spaces between each letter in order for Jess to pronounce each letter individually and I needed to add a hyphen to CMALT (C-MALT) for it to be pronounced correctly. But for the most part I can leave the text much as it is typed. I then run it through a free, open source software called Balabolka to produce an audio file that participants on the course can download and listen to.

    TechDis Jess and other UK voices (including Scottish and Welsh options) are available from www.heacademy.ac.uk/jisc-techdis-voices.

    Balabolka is available from: www.cross-plus-a.com/balabolka.htm.
    Listen to a sample:

    Listen on SoundCloud…

    *Staff and learners studying at England’s UK and FE institutions can download the voices free of charge and those at Scottish and Welsh institutions can download local voices.

    HEA Senior Fellowship Case Study Series: 4 – Researching learner interaction and engagement with in-class response systems

    By Matt Jenner, on 15 August 2014

    As a four-part series I am openly publishing my case studies previously submitted for my Senior Fellowship of the Higher Education Academy. I submitted my application in February 2014. If you’re interested in this professional recognition programme, please visit their webpages and look through the Professional Standards Framework (PSF). UCL runs an institutional model for fellowships called ARENA, your institution may run one too – speak to people!

    Case Study 4 – Researching learner interaction and engagement with in-class response systems

    In 2012 I conducted research, in parallel with my job at UCL, focusing on increasing student interaction and staff engagement of an in-class question and response system colloquially known as ‘clickers’. Evidence suggests clickers provide interaction opportunities to stimulate and engage learners[1] and have a benign or positive effect in student performance[2]. Clickers are popular across many disciplines, in particular the physical sciences, but there is a particularly low interest in medical sciences.

    I wanted to directly address this shortcoming so I enlisted two academics in the UCL Medical School. I assimilated the current method of teaching, and the materials used (K1). From here we adapted a learning activity to align with the new tool being applied (A1). I underpinned the use of the technology with existing literature and the evidence of realigning the ‘sage on the stage’ to the ‘guide on the side’ [3](K2), which evidence suggests is an effective method for learning and teaching (K3, V3). I provided pre-lecture technical support to reduce technical barriers and was on-hand in the lecture to support as/when needed (A2). Questions were designed into the lectures and the clickers provide immediate feedback (A3). Staff react to clicker data with an approach called ‘contingent teaching’[4] where they dynamically respond to the answers/feedback provided (A3).

    I designed evaluation questions for each lecture based on Bloom’s Taxonomy[5] for learners-based evaluation of the teaching approach and learning outcomes (A4). Questions were derived from categorising Bloom into three sub-categories; remember or understand, apply or analyse the topic and evaluate or create new knowledge (K5). When questioned, 74% of students agreed or strongly agreed that the clickers and the related teaching approach encouraged interaction and helped to achieve metacognitive learning (K5). I integrated these data with post-lecture interviews for the lecturers. Using this analysis, we designed next steps for future use and identified gaps and areas for improvement (A5).

    I conducted evidence-based research and followed best practice around clickers to ensure inclusion was academically merited (V3). Measuring (and increasing) engagement within the traditional lecture was aiming to promote participation for learners (V2). It was understood that clickers do not directly enhance learning but can lead to higher-order learning. I used my understanding of the wider field of evidence to define their most appropriate use within the lectures (V1, V3).

    By implementing a technology which was new to staff and guiding them with appropriate techniques known to increase interaction and engagement, I provided an evidence-informed approach which could be used to transform didactic content delivery into something more engaging. My research adds to a disproportionately small body of knowledge for clickers in medical education and the study overall was positive. Staff involved still use the clickers, the impact I measured plus the evidence collected, can be further used to promote clickers within UCL, the Medical School and beyond. It earned me a Distinction in my MSc Learning Technologies and furthered my ambition to make a lasting, positive difference to higher education.

    (493 words)

    HEA Professional Standards Framework links referenced in this case study:

    Areas of Activity

    • A1 Design and plan learning activities and/or programmes of study
    • A2 Teach and/or support learning
    • A3 Assess and give feedback to learners
    • A4 Develop effective learning environments and approaches to student support and guidance
    • A5 Engage in continuing professional development in subjects/disciplines and their pedagogy, incorporating research, scholarship and the evaluation of professional practices

    Core Knowledge

    • K1 The subject material
    • K2 Appropriate methods for teaching, learning and assessing in the subject area and at the level of the academic programme
    • K3 How students learn, both generally and within their subject/disciplinary area(s)
    • K5 Methods for evaluating the effectiveness of teaching

    Professional Values

    • V1 Respect individual learners and diverse learning communities
    • V2 Promote participation in higher education and equality of opportunity for learners
    • V3 Use evidence-informed approaches and the outcomes from research, scholarship and continuing professional development


    [1] Bligh, D.A., (2000). What’s the use of Lectures? London/San Francisco; Jossey-Bass

    [2] http://w.lifescied.org/content/6/1/9.short

    [3] King, A. (1993). From Sage on the Stage to Guide on the Side. College Teaching, Vol. 41, No. 1, p30- 35. Taylor & Francis Ltd.

    [4] Beatty I. D., Gerace W. J., Leonard W. J. and Dufresne R. J., (2006). Designing effective questions for classroom response teaching, American Journal of Physics. Vol. 74, p31-39.

    [5] Bloom B.S., (1956). Taxonomy of Educational Objectives, Handbook I: The Cognitive Domain. New York: David McKay Co Inc.

    HEA Senior Fellowship Case Study Series: 3 – Facilitating communities of practice at UCL and beyond

    By Matt Jenner, on 14 August 2014

    As a four-part series I am openly publishing my case studies previously submitted for my Senior Fellowship of the Higher Education Academy. I submitted my application in February 2014. If you’re interested in this professional recognition programme, please visit their webpages and look through the Professional Standards Framework (PSF). UCL runs an institutional model for fellowships called ARENA, your institution may run one too – speak to people!

    Case study 3 – Facilitating communities of practice at UCL and beyond

    At UCL I have facilitated and been involved in two active communities of practice. One external group called ‘Moodle User Group Greater London’ (MUGGL)[1] is for e-learning professionals with an interest in ‘Moodle’ – an online learning environment. More historically, I helped this community have an agenda for meetings with a colleague, Sarah Sherman in a neighbouring university consortia the ‘Bloomsbury Learning Environment’ [2]. The other, the ‘Distance Learning, CPD and Short Courses Network’[3], is an internal network I help steer for colleagues at UCL. These communities focus on “sharing best practices and creating new knowledge to advance a domain of professional practice”[4]. Drawing on my enthusiasm for e-learning and connecting people, the communities I facilitate take me beyond my desk and into the wider sector (A5, V3, V4).

    People often disappear into enclaves, with daily priorities overarching valuable reflective opportunities. I work responsively to sustain and grow communities with common needs, values, locale and domain of discourse. All too often communities around information technology end up as one-way monologs beset by dry jargon. Instead I plan active pedagogically-focused sessions and encourage sharing and collaboration. This provides a vital source of information, future utility (V3) and cross-sector discovery (A4).

    I regularly participate in wider CPD events and read literature in the areas of distance learning, MOOCs (Massive Online Open Courses) and e-learning to remain current in my understanding and share my experiences within networks and social media (V3). For the internal community, I organise and deliver centralised updates from professional services and the wider context of relevant developments so colleagues can gain purview of the changing landscape of higher education and e-learning (V4).

    Both communities started three to four years ago as special interest groups created around particular foci. I have encouraged expansion of both networks, with events often attracting 50–150 people and a core membership of many times that figure. With this increasing range of members I must ensure to maintain respect for the needs of the group when planning three-four hours of their professional development time (V1). Communities are grown, not constructed, and sharing and learning cannot be “legislated into existence”[5]. I sideline my leadership in alignment with members’ interests to ensure events are planned with consultation and encouragement for active participation in each session. I make events accessible by adding remote attendance options and capturing them on video, to encourage participation and preserve inclusion for the diverse members of the community (V2).

    I have received feedback stating I “did a great job organizing everyone and your presentation was beautiful”. Comments for MUGGL events have noted a “big crowd for the #muggl Moodle 2 meeting today” and “spent the morning at #muggl good presentations from @mattjenner […] we watch and learn!” Connecting people is an integral part, with one group visit commenting “how proud we are to work in such an engaging institution. Most participants left inspired”. In relation to expanding the community a senior member of staff spoke of a well-attended event that “the increase in numbers and interest across campus reflects the importance of this area”. Feedback is can be sporadic, but I take comments on board for future sessions and resolving any issues.

    (515 words)

    HEA Professional Standards Framework links referenced in this case study:

    Areas of Activity

    • A4 Develop effective learning environments and approaches to student support and guidance
    • A5 Engage in continuing professional development in subjects/disciplines and their pedagogy, incorporating research, scholarship and the evaluation of professional practices

    Professional Values

    • V1 Respect individual learners and diverse learning communities
    • V2 Promote participation in higher education and equality of opportunity for learners
    • V3 Use evidence-informed approaches and the outcomes from research, scholarship and continuing professional development
    • V4 Acknowledge the wider context in which higher education operates recognising the implications for professional practice


    [1] https://sites.google.com/site/moodlelondon/

    [2] http://www.bloomsbury.ac.uk/ble

    [3] https://www.mailinglists.ucl.ac.uk/mailman/listinfo/distancelearning

    [4] http://net.educause.edu/ir/library/pdf/nli0531.pdf

    [5] Dubé, L., Bourhis, A. & Jacob, R. (2005). The impact of structuring characteristics on the launching of virtual communities of practice. Journal of Organizational Change Management, 18(2): 145-166.

    HEA Senior Fellowship Case Study Series: 2 – Bringing lecture flipping to a new interdisciplinary programme

    By Matt Jenner, on 13 August 2014

    As a four-part series I am openly publishing my case studies previously submitted for my Senior Fellowship of the Higher Education Academy. I submitted my application in February 2014. If you’re interested in this professional recognition programme, please visit their webpages and look through the Professional Standards Framework (PSF). UCL runs an institutional model for fellowships called ARENA, your institution may run one too – speak to people!

    Case Study 2 – Bringing lecture flipping to a new interdisciplinary programme

    As an experienced member of staff I have a wide range of institutional e-learning responsibilities. In 2011 I was an integral part in designing, delivering and evaluating a new teaching style called the ‘flipped lecture’. It’s increasingly thought that lectures do not provide “students a rich and rewarding educational experience”[1]. The flipped model is where “students gain first-exposure learning prior to class and focus on the processing part of learning (synthesizing, analyzing, problem-solving, etc.) in class”[2]. The first academic to explicitly flip their lectures, and put the concept in the UCL spotlight, was Carl Gombrich, director of UCL’s flagship interdisciplinary degree the Bachelor of Arts and Sciences (BASc).

    Carl wanted to put “people back at the centre of the learning”12 and I supported and advised him while we created an active learning, ‘flipped lecture’, environment (A2, A4). UCL’s institutional teaching and learning strategy envisaged the BASc providing “a model for a substantial proportion of UCL’s undergraduate teaching”[3]. Our plan was to create a teaching approach with innovative and sustainable qualities.

    Carl is a capable teacher but he was less comfortable with the technological change required to achieve ‘flipped lectures’. I mentored him through the virtues, and downsides, of the available technological landscape. A barrier emerged within institutional systems which did not support his approach. While retaining alignment with the planned learning activities, (A1, K4) I researched available options and evaluated a new tool called Hot Question[4] based on research from Purdue University[5]. My research outcome delivered a solution which matched the requirements for the BASc and was designed to scale for the institution if flipping increased in popularity. It is now used across 81 courses at UCL (A4).

    Feedback received from Carl has been “In many (most) cases it really does get students thinking about the content of the lectures” and “how sophisticated” some of the student’s questions are. A student on the course commented “once you have the additional time in lectures, you’re going to be able to take advantage of that”9. Another academic commented ‘flipping’ uncovers “the impact of the information we provide for students, and their understanding at a much finer temporal resolution than what was available previously”[6]. One issue that’s arisen is that “so many questions are submitted and so many are of a high standard that it is a bit bewildering to look through them all”.

    I have shared the flipped approach within external communities[7] and Carl’s work has become an inspirational exemplar for staff to experiment with flipped lectures. The technological changes, and teaching approaches have been shared via blog posts[8] and case studies[9], they provide evidence for colleagues to learn from, discuss and adopt. Seeing this as a potential future component of teaching at UCL, it was my role to ensure we built a sustainable model not just for the BASc but for the wider UCL community. By ensuring the flipped lecture was well supported we have seen a growth in popularity across the university’s faculties of Engineering, Social and Medical Sciences.

    (505 words)

    HEA Professional Standards Framework links referenced in this case study:

    Areas of Activity

    • A1 Design and plan learning activities and/or programmes of study
    • A2 Teach and/or support learning
    • A3 Assess and give feedback to learners
    • A4 Develop effective learning environments and approaches to student support and guidance
    • A5 Engage in continuing professional development in subjects/disciplines and their pedagogy, incorporating research, scholarship and the evaluation of professional practices

    Core Knowledge

    • K4 The use and value of appropriate learning technologies


    [1] http://www.brookes.ac.uk/services/ocsld/resources/20reasons.html

    [3] http://www.ucl.ac.uk/teaching-learning/secure-downloads/ILTS.pdf

    [4] https://moodle.org/plugins/view.php?plugin=mod_hotquestion

    [5] https://www.purdue.edu/hotseat

    [6] https://www.ucl.ac.uk/stream/media/swatch?v=5461b59f4751

    [7] https://twitter.com/AndyKons/status/278446517537353728

    [8] http://blogs.ucl.ac.uk/digital-education/tag/flipping/

    [9] http://www.ucl.ac.uk/teaching-learning/news/carl-gombrich-flipping-the-lecture-theatre

    HEA Senior Fellowship Case Study Series: 1 – Creating a public-facing e-learning environment

    By Matt Jenner, on 12 August 2014

    As a four-part series I am openly publishing my case studies previously submitted for my Senior Fellowship of the Higher Education Academy. I submitted my application in February 2014. If you’re interested in this professional recognition programme, please visit their webpages and look through the Professional Standards Framework (PSF). UCL runs an institutional model for fellowships called ARENA, your institution may run one too – speak to people!

    Case Study 1 – Creating a public facing e-learning environment

    With 10 years’ experience, I joined UCL’s central e-learning team in 2008. My role requires me to advise a diverse academic community traversing a diverse technological landscape. I build strong relationships with colleagues and contribute to technical developments and institutional strategies. My specialisation is distance education, an area experiencing accelerated growth across the sector due to demand for flexible learning, increasing technological grasp and the questionable future of Massive Online Open Courses.

    My activity focuses around UCLeXtend – a new public-facing online learning environment offering free and premium courses. I advocate open education and am passionate about the opportunities universities have for social enterprise, global impact and widening participation. A core component of our institutional e-learning strategy is to “raise UCL’s profile as a global education leader”[1]; by opening the rich and varied corpus of UCL to a wider audience I am enabling this reality.

    UCLeXtend is built on familiar and established e-learning software which eases the transition for staff (K4). Staff leave their comfort zones when developing distance learning so I mentor them throughout the process. Course development approaches are less familiar; so I encourage course teams to follow a customised course development framework based on an existing model named ADDIE (Analysis, Design, Development, Implementation and Evaluation)[2] (K5). I facilitate open discussion of ideas and assimilate their subject material from the outset, advising on suitable development approaches (K1).

    I struggle with the demand for ‘rapid’ (i.e. quick and cheap) course development, some colleagues need convincing that high quality interactive and engaging learning is not guaranteed by nice fonts and shiny graphics. I adapted the ADDIE course development framework to focus on the needs and approaches of individuals’ learning, tailored for the appropriate market and teaching style (K2). I encourage good practice and make recommendations for course design and structure, especially factoring in learner’s who are going to “consider the potential educational benefits”[3] of each resource or activity (K3).

    Creating courses open to the public exposes UCL, so to protect our brand and standards I developed and lead on integrating quality assurance processes. All courses are scrutinised via an academic and rigorous review process (K6). Initially this was too much like a ‘course approval’ system, creating unnecessary pressure on both sides. I therefore matured it into a critical friend review. One academic commented they were “really pleased by the positive reactions and by the very useful suggestions we got from the panel” and another noted it was “a very constructive meeting”.

    In eight months UCLeXtend has nine live courses and over 2000 learners from 68 countries. Although the evaluation phase has yet to commence, one learner commented “how fantastic the better conversations tool for aphasia is […] and has so many benefits”. In the longer term, UCLeXtend will become positively disruptive to UCL. I have senior level support and interest across the university; my challenge now is to lead UCLeXtend into a sustainable model and integral to future institutional priorities. For me, the strong start was critical to success; my on-going leadership in this area will ensure the initial quality sets the baseline for future growth.

    (516 words)

    HEA Professional Standards Framework links referenced in this case study:

    Core Knowledge

    • K1 The subject material
    • K2 Appropriate methods for teaching, learning and assessing in the subject area and at the level of the academic programme
    • K3 How students learn, both generally and within their subject/disciplinary area(s)
    • K4 The use and value of appropriate learning technologies
    • K5 Methods for evaluating the effectiveness of teaching
    • K6 The implications of quality assurance and quality enhancement for academic and professional practice with a particular focus on teaching

     


    [1] http://www.ucl.ac.uk/teaching-learning/strategic_priorities/e-learning-strategy

    [2] http://www.nwlink.com/~donclark/history_isd/addie.html

    [3] http://oro.open.ac.uk/10072/1/Assessment_and_student_learning-HO.pdf