Digital Education team blog
  • We support Staff and Students using technology to enhance education at UCL.

    Here you'll find updates on institutional developments, projects we're involved in, updates on educational technology, events, case studies and personal experiences (or views!).

    Subscribe to our elearning newsletters.

  • Subscribe to this blog

  • Meta

  • Tags

  • A A A

    Archive for the 'Case Studies' Category

    Students’ intellectual property, open nitty gritty

    By Mira Vogel, on 19 May 2015

    Brass tacks by MicroAssist on FlickrWhat happened when staff on one module encouraged students to openly license the online products of their assessed group work?

    Object Lessons is a module on Bachelor of Arts and Sciences at UCL. In keeping with its object-based nature and emphasis on inquiry and collaboration, part of the assessment is a group research project to produce a media-rich online exhibition. Because the exhibitions are lovely and shine a light on multimodal assessment, the teaching team are frequently approached by colleagues across UCL with requests to view them. In considering how to get students’ permission for this, Leonie Hannan (now at QUB), Helen Chatterjee and I quickly realised a few things. One, highlighted by an exchange with UCL’s Copyright specialist Chris Holland, was that the nature of the permission was hard to define and therefore hard to get consent for, so we needed to shift the emphasis away from staff and the nuances of their possible use scenarios, and onto the status of the work itself. Another was that since the work was the product of a group and could not be decomposed into individual contributions without breaking the whole, consent would need to be unanimous. Then there was the question of administrative overhead related to obtaining consent and actually implementing what students had consented to – potentially quite onerous. And finally the matter presented us with some opportunities we shouldn’t miss, namely to model taking intellectual property seriously and to engage students in key questions about contemporary practices.

    We came up with four alternative ways for students to license their work ranging incrementally from open to private. We called these:

    1. Open;
    2. Publish;
    3. Show;
    4. Private.

    You can read definitions of each alternative in the document ‘Your groupwork project – requesting consent for future use scenarios’ which we produced to introduce them to students. As part of their work students were required to discuss these, reach a unanimous consensus on one, and implement it by publishing (or selectively, or not at all) the exhibition and providing an intellectual property notice on its front page. That way staff would not have to collect consent forms nor gate-keep access.

    Before we released it to students I circulated the guidance to two Jiscmail discussion groups (Open Educational Resources and Association for Learning Technology) and worked in some of their suggestions. A requirement that students include a statement within the work itself reduces the administrative overhead and, we hoped, would be more future-proof than staff collecting, checking off and filing paper records. While making it clear that students would not be at any deficit if they chose not to open their work, we also took a clear position in favour of Creative Commons licensing – the most open of our alternatives, since as well as flexibility and convenience it would potentially lend the work more discoverability and exposure.

    What did the students choose? In the first iteration, out of ten groups:

    • Five opted for Open. Between them they used 3 different varieties of Creative Commons licence, and one submitted their work to Jorum;
    • Two opted for Publish;
    • None opted for Show;
    • Three opted for Private (including one which didn’t make a statement; since the group kept the work hidden this defaults to Private).

    We haven’t yet approached the students to ask about their decision-making processes, but from informal conversations and reading some of the intellectual property statements we know that there are different reasons why half the students decided not to make their work open. One was the presence of elements which were not themselves open, and therefore could not be opened in turn. From evaluations of a number of other modules, we know that the students were not generally all that enthusiastic about the platform they were asked to use for their exhibition (Mahara, which is serviceable but vanishingly rare outside educational settings). This may have contributed to another factor, which was that not all group members felt the work reflected well on them individually.

    Then there’s the matter of deciding to revoke consent, which is something individual students can do at any time. In the context of group work we decided that what this would mean is that if any group member decides at a later date that they want to reduce openness, then this effectively overrides other group members’ preferences. That doesn’t work in reverse though – a student can’t increase openness without the consent of all other group members. So here we are privileging individuals who want to close work, although we do encourage them to consider instead simply ending their association with it. We have yet to find out how this state of affairs works out, and it may take quite a while to find out. But so far it seems stable and viable.

    We would be very interested in your views, suggestions and any experiences you have had with this kind of thing – please do comment below.

    Particular thanks to Pat Lockley and Javiera Atenas for their input.

    Image source: MicroAssist, 2012. Brass tacks. Work found at https://www.flickr.com/photos/microassist/7136725313/. Licensed as CC BY-SA.

    A good peer review experience with Moodle Workshop

    By Mira Vogel, on 18 March 2015

    Update Dec 2015: there are now three posts on our refinements to this peer feedback activity: one, two, and three.

    Readers have been begging for news of how it went with the Moodle Workshop activity from this post.

    Workshop is an activity in Moodle which allows staff to set up a peer assessment or (in our case) peer review. Workshop collects student work, automatically allocates reviewers, allows the review to be scaffolded with questions, imposes deadlines on the submission and assessment phase, provides a dashboard so staff can follow progress, and allows staff to assess the reviews/assessments as well as the submissions.

    However, except for some intrepid pioneers, it is almost never seen in the wild.

    The reason for that is partly to do with daunting number and nature of the settings – there are several pitfalls to avoid which aren’t obvious on first pass – but also the fact that because it is a process you can’t easily see a demo and running a test instance is pretty time consuming. If people try once and it doesn’t work well they rarely try again.

    Well look no further – CALT and ELE have it working well now and can support you with your own peer review.

    What happened?

    Students on the UCL Arena Teaching Associate Programme reviewed each others’ case studies. 22 then completed a short evaluation questionnaire in which they rated their experience of giving and receiving feedback on a five-point scale and commented on their responses. The students were from two groups with different tutors running the peer review activity. A third group leader chose to run the peer review on Moodle Forum since it would allow students to easily see each others’ case studies and feedback.

    The students reported that giving feedback went well (21 respondents):

    Pie chart - giving feedback

    Satisfaction with reviewing work – click to enlarge

    This indicates that the measures we took – see previous post – to address disorientation and participation were successful. In particular we were better aware of where the description, instructions for submission, instructions for assessment, and concluding comments would display, and put the relevant information into each.

    Receiving feedback also went well (22 respondents) though with a slightly bigger spread in both directions:

    Pie chart - receiving feedback

    Satisfaction with receiving reviews – click to enlarge

     

    Students appreciated:

    • Feedback on their work.
    • Insights about their own work from considering others’ work.
    • Being able to edit their submission in advance of the deadline.
    • The improved instructions letting them know what to do, when and where.

    Staff appreciated:

    This hasn’t been formally evaluated, but from informal conversations I know that the two group leaders appreciate Moodle taking on the grunt work of allocation. However, this depends on setting a hard deadline with no late submissions (otherwise staff have to keep checking for late submissions and allocating those manually) and one of the leaders was less comfortable with this than the other. Neither found it too onerous to write diary notes to send reminders and alerts to students to move the activity along – in any case this manual messaging will hopefully become unnecessary with the arrival of Moodle Events in the coming upgrade.

    For next time:

    • Improve signposting from the Moodle course area front page, and maybe the title of the Workshop itself, so students know what to do and when.
    • Instructions: let students know how many reviews they are expected to do; let them know if they should expect variety in how the submissions display – in our case some were attachments while others were typed directly into Moodle (we may want to set attachments to zero); include word count guidance in the instructions for submission and assessment.
    • Consider including an example case study & review for reference (Workshop allows this).
    • Address the issue that, due to some non-participation during the Assessment phase, some students gave more feedback than they received.
    • We originally had a single comments field but will now structure the peer review with some questions aligned to the relevant parts of the criteria.
    • Decide about anonymity – should both submissions and reviews be anonymous, or one or the other, or neither? These can be configured via the Workshop’s Permissions. Let students know who can see what.
    • Also to consider – we could also change Permissions after it’s complete (or even while it’s running) to allow students to access the dashboard and see all the case studies and all the feedback.

    Have you had a good experience with Moodle Workshop? What made it work for you?

    Meet Jess, Jack, Stuart & Heather – realistic voices for free* download

    By Jessica Gramp, on 3 March 2015

    I have recently started listening to my books and papers, rather than reading them. This frees me up to do other things while I listen, such as cook, take a bath or do some tidying up. It also gives my eyes a well needed break from staring at a computer screen or paper.

    As part of an online e-learning course I am helping to develop, I am using the TechDis Jess voice to provide audio files of the commentary, as an alternative to reading. I have had to tweak some of the text – for example, UCL needs to be written with spaces between each letter in order for Jess to pronounce each letter individually and I needed to add a hyphen to CMALT (C-MALT) for it to be pronounced correctly. But for the most part I can leave the text much as it is typed. I then run it through a free, open source software called Balabolka to produce an audio file that participants on the course can download and listen to.

    TechDis Jess and other UK voices (including Scottish and Welsh options) are available from www.heacademy.ac.uk/jisc-techdis-voices.

    Balabolka is available from: www.cross-plus-a.com/balabolka.htm.
    Listen to a sample:

    Listen on SoundCloud…

    *Staff and learners studying at England’s UK and FE institutions can download the voices free of charge and those at Scottish and Welsh institutions can download local voices.

    HEA Senior Fellowship Case Study Series: 4 – Researching learner interaction and engagement with in-class response systems

    By Matt Jenner, on 15 August 2014

    As a four-part series I am openly publishing my case studies previously submitted for my Senior Fellowship of the Higher Education Academy. I submitted my application in February 2014. If you’re interested in this professional recognition programme, please visit their webpages and look through the Professional Standards Framework (PSF). UCL runs an institutional model for fellowships called ARENA, your institution may run one too – speak to people!

    Case Study 4 – Researching learner interaction and engagement with in-class response systems

    In 2012 I conducted research, in parallel with my job at UCL, focusing on increasing student interaction and staff engagement of an in-class question and response system colloquially known as ‘clickers’. Evidence suggests clickers provide interaction opportunities to stimulate and engage learners[1] and have a benign or positive effect in student performance[2]. Clickers are popular across many disciplines, in particular the physical sciences, but there is a particularly low interest in medical sciences.

    I wanted to directly address this shortcoming so I enlisted two academics in the UCL Medical School. I assimilated the current method of teaching, and the materials used (K1). From here we adapted a learning activity to align with the new tool being applied (A1). I underpinned the use of the technology with existing literature and the evidence of realigning the ‘sage on the stage’ to the ‘guide on the side’ [3](K2), which evidence suggests is an effective method for learning and teaching (K3, V3). I provided pre-lecture technical support to reduce technical barriers and was on-hand in the lecture to support as/when needed (A2). Questions were designed into the lectures and the clickers provide immediate feedback (A3). Staff react to clicker data with an approach called ‘contingent teaching’[4] where they dynamically respond to the answers/feedback provided (A3).

    I designed evaluation questions for each lecture based on Bloom’s Taxonomy[5] for learners-based evaluation of the teaching approach and learning outcomes (A4). Questions were derived from categorising Bloom into three sub-categories; remember or understand, apply or analyse the topic and evaluate or create new knowledge (K5). When questioned, 74% of students agreed or strongly agreed that the clickers and the related teaching approach encouraged interaction and helped to achieve metacognitive learning (K5). I integrated these data with post-lecture interviews for the lecturers. Using this analysis, we designed next steps for future use and identified gaps and areas for improvement (A5).

    I conducted evidence-based research and followed best practice around clickers to ensure inclusion was academically merited (V3). Measuring (and increasing) engagement within the traditional lecture was aiming to promote participation for learners (V2). It was understood that clickers do not directly enhance learning but can lead to higher-order learning. I used my understanding of the wider field of evidence to define their most appropriate use within the lectures (V1, V3).

    By implementing a technology which was new to staff and guiding them with appropriate techniques known to increase interaction and engagement, I provided an evidence-informed approach which could be used to transform didactic content delivery into something more engaging. My research adds to a disproportionately small body of knowledge for clickers in medical education and the study overall was positive. Staff involved still use the clickers, the impact I measured plus the evidence collected, can be further used to promote clickers within UCL, the Medical School and beyond. It earned me a Distinction in my MSc Learning Technologies and furthered my ambition to make a lasting, positive difference to higher education.

    (493 words)

    HEA Professional Standards Framework links referenced in this case study:

    Areas of Activity

    • A1 Design and plan learning activities and/or programmes of study
    • A2 Teach and/or support learning
    • A3 Assess and give feedback to learners
    • A4 Develop effective learning environments and approaches to student support and guidance
    • A5 Engage in continuing professional development in subjects/disciplines and their pedagogy, incorporating research, scholarship and the evaluation of professional practices

    Core Knowledge

    • K1 The subject material
    • K2 Appropriate methods for teaching, learning and assessing in the subject area and at the level of the academic programme
    • K3 How students learn, both generally and within their subject/disciplinary area(s)
    • K5 Methods for evaluating the effectiveness of teaching

    Professional Values

    • V1 Respect individual learners and diverse learning communities
    • V2 Promote participation in higher education and equality of opportunity for learners
    • V3 Use evidence-informed approaches and the outcomes from research, scholarship and continuing professional development


    [1] Bligh, D.A., (2000). What’s the use of Lectures? London/San Francisco; Jossey-Bass

    [2] http://w.lifescied.org/content/6/1/9.short

    [3] King, A. (1993). From Sage on the Stage to Guide on the Side. College Teaching, Vol. 41, No. 1, p30- 35. Taylor & Francis Ltd.

    [4] Beatty I. D., Gerace W. J., Leonard W. J. and Dufresne R. J., (2006). Designing effective questions for classroom response teaching, American Journal of Physics. Vol. 74, p31-39.

    [5] Bloom B.S., (1956). Taxonomy of Educational Objectives, Handbook I: The Cognitive Domain. New York: David McKay Co Inc.

    HEA Senior Fellowship Case Study Series: 3 – Facilitating communities of practice at UCL and beyond

    By Matt Jenner, on 14 August 2014

    As a four-part series I am openly publishing my case studies previously submitted for my Senior Fellowship of the Higher Education Academy. I submitted my application in February 2014. If you’re interested in this professional recognition programme, please visit their webpages and look through the Professional Standards Framework (PSF). UCL runs an institutional model for fellowships called ARENA, your institution may run one too – speak to people!

    Case study 3 – Facilitating communities of practice at UCL and beyond

    At UCL I have facilitated and been involved in two active communities of practice. One external group called ‘Moodle User Group Greater London’ (MUGGL)[1] is for e-learning professionals with an interest in ‘Moodle’ – an online learning environment. More historically, I helped this community have an agenda for meetings with a colleague, Sarah Sherman in a neighbouring university consortia the ‘Bloomsbury Learning Environment’ [2]. The other, the ‘Distance Learning, CPD and Short Courses Network’[3], is an internal network I help steer for colleagues at UCL. These communities focus on “sharing best practices and creating new knowledge to advance a domain of professional practice”[4]. Drawing on my enthusiasm for e-learning and connecting people, the communities I facilitate take me beyond my desk and into the wider sector (A5, V3, V4).

    People often disappear into enclaves, with daily priorities overarching valuable reflective opportunities. I work responsively to sustain and grow communities with common needs, values, locale and domain of discourse. All too often communities around information technology end up as one-way monologs beset by dry jargon. Instead I plan active pedagogically-focused sessions and encourage sharing and collaboration. This provides a vital source of information, future utility (V3) and cross-sector discovery (A4).

    I regularly participate in wider CPD events and read literature in the areas of distance learning, MOOCs (Massive Online Open Courses) and e-learning to remain current in my understanding and share my experiences within networks and social media (V3). For the internal community, I organise and deliver centralised updates from professional services and the wider context of relevant developments so colleagues can gain purview of the changing landscape of higher education and e-learning (V4).

    Both communities started three to four years ago as special interest groups created around particular foci. I have encouraged expansion of both networks, with events often attracting 50–150 people and a core membership of many times that figure. With this increasing range of members I must ensure to maintain respect for the needs of the group when planning three-four hours of their professional development time (V1). Communities are grown, not constructed, and sharing and learning cannot be “legislated into existence”[5]. I sideline my leadership in alignment with members’ interests to ensure events are planned with consultation and encouragement for active participation in each session. I make events accessible by adding remote attendance options and capturing them on video, to encourage participation and preserve inclusion for the diverse members of the community (V2).

    I have received feedback stating I “did a great job organizing everyone and your presentation was beautiful”. Comments for MUGGL events have noted a “big crowd for the #muggl Moodle 2 meeting today” and “spent the morning at #muggl good presentations from @mattjenner […] we watch and learn!” Connecting people is an integral part, with one group visit commenting “how proud we are to work in such an engaging institution. Most participants left inspired”. In relation to expanding the community a senior member of staff spoke of a well-attended event that “the increase in numbers and interest across campus reflects the importance of this area”. Feedback is can be sporadic, but I take comments on board for future sessions and resolving any issues.

    (515 words)

    HEA Professional Standards Framework links referenced in this case study:

    Areas of Activity

    • A4 Develop effective learning environments and approaches to student support and guidance
    • A5 Engage in continuing professional development in subjects/disciplines and their pedagogy, incorporating research, scholarship and the evaluation of professional practices

    Professional Values

    • V1 Respect individual learners and diverse learning communities
    • V2 Promote participation in higher education and equality of opportunity for learners
    • V3 Use evidence-informed approaches and the outcomes from research, scholarship and continuing professional development
    • V4 Acknowledge the wider context in which higher education operates recognising the implications for professional practice


    [1] https://sites.google.com/site/moodlelondon/

    [2] http://www.bloomsbury.ac.uk/ble

    [3] https://www.mailinglists.ucl.ac.uk/mailman/listinfo/distancelearning

    [4] http://net.educause.edu/ir/library/pdf/nli0531.pdf

    [5] Dubé, L., Bourhis, A. & Jacob, R. (2005). The impact of structuring characteristics on the launching of virtual communities of practice. Journal of Organizational Change Management, 18(2): 145-166.

    HEA Senior Fellowship Case Study Series: 2 – Bringing lecture flipping to a new interdisciplinary programme

    By Matt Jenner, on 13 August 2014

    As a four-part series I am openly publishing my case studies previously submitted for my Senior Fellowship of the Higher Education Academy. I submitted my application in February 2014. If you’re interested in this professional recognition programme, please visit their webpages and look through the Professional Standards Framework (PSF). UCL runs an institutional model for fellowships called ARENA, your institution may run one too – speak to people!

    Case Study 2 – Bringing lecture flipping to a new interdisciplinary programme

    As an experienced member of staff I have a wide range of institutional e-learning responsibilities. In 2011 I was an integral part in designing, delivering and evaluating a new teaching style called the ‘flipped lecture’. It’s increasingly thought that lectures do not provide “students a rich and rewarding educational experience”[1]. The flipped model is where “students gain first-exposure learning prior to class and focus on the processing part of learning (synthesizing, analyzing, problem-solving, etc.) in class”[2]. The first academic to explicitly flip their lectures, and put the concept in the UCL spotlight, was Carl Gombrich, director of UCL’s flagship interdisciplinary degree the Bachelor of Arts and Sciences (BASc).

    Carl wanted to put “people back at the centre of the learning”12 and I supported and advised him while we created an active learning, ‘flipped lecture’, environment (A2, A4). UCL’s institutional teaching and learning strategy envisaged the BASc providing “a model for a substantial proportion of UCL’s undergraduate teaching”[3]. Our plan was to create a teaching approach with innovative and sustainable qualities.

    Carl is a capable teacher but he was less comfortable with the technological change required to achieve ‘flipped lectures’. I mentored him through the virtues, and downsides, of the available technological landscape. A barrier emerged within institutional systems which did not support his approach. While retaining alignment with the planned learning activities, (A1, K4) I researched available options and evaluated a new tool called Hot Question[4] based on research from Purdue University[5]. My research outcome delivered a solution which matched the requirements for the BASc and was designed to scale for the institution if flipping increased in popularity. It is now used across 81 courses at UCL (A4).

    Feedback received from Carl has been “In many (most) cases it really does get students thinking about the content of the lectures” and “how sophisticated” some of the student’s questions are. A student on the course commented “once you have the additional time in lectures, you’re going to be able to take advantage of that”9. Another academic commented ‘flipping’ uncovers “the impact of the information we provide for students, and their understanding at a much finer temporal resolution than what was available previously”[6]. One issue that’s arisen is that “so many questions are submitted and so many are of a high standard that it is a bit bewildering to look through them all”.

    I have shared the flipped approach within external communities[7] and Carl’s work has become an inspirational exemplar for staff to experiment with flipped lectures. The technological changes, and teaching approaches have been shared via blog posts[8] and case studies[9], they provide evidence for colleagues to learn from, discuss and adopt. Seeing this as a potential future component of teaching at UCL, it was my role to ensure we built a sustainable model not just for the BASc but for the wider UCL community. By ensuring the flipped lecture was well supported we have seen a growth in popularity across the university’s faculties of Engineering, Social and Medical Sciences.

    (505 words)

    HEA Professional Standards Framework links referenced in this case study:

    Areas of Activity

    • A1 Design and plan learning activities and/or programmes of study
    • A2 Teach and/or support learning
    • A3 Assess and give feedback to learners
    • A4 Develop effective learning environments and approaches to student support and guidance
    • A5 Engage in continuing professional development in subjects/disciplines and their pedagogy, incorporating research, scholarship and the evaluation of professional practices

    Core Knowledge

    • K4 The use and value of appropriate learning technologies


    [1] http://www.brookes.ac.uk/services/ocsld/resources/20reasons.html

    [3] http://www.ucl.ac.uk/teaching-learning/secure-downloads/ILTS.pdf

    [4] https://moodle.org/plugins/view.php?plugin=mod_hotquestion

    [5] https://www.purdue.edu/hotseat

    [6] https://www.ucl.ac.uk/stream/media/swatch?v=5461b59f4751

    [7] https://twitter.com/AndyKons/status/278446517537353728

    [8] http://blogs.ucl.ac.uk/digital-education/tag/flipping/

    [9] http://www.ucl.ac.uk/teaching-learning/news/carl-gombrich-flipping-the-lecture-theatre