E-Learning Environments team blog
  • ELE Group
    We support Staff and Students using technology to enhance teaching & learning.

    Here you'll find updates on developments at UCL, links & events as well as case studies and personal experiences. Let us know if you have any ideas you want to share!

  • Subscribe to the ELE blog

  • Meta

  • Tags

  • A A A

    Archive for the 'Learning designs' Category

    ABC Curriculum Design Workshops

    By Natasa Perovic, on 30 September 2015

    Arena Blended Connected Curriculum Design


    • A 90 minute hands-on workshop to help module teams design engaging learning activities.
    • Teams work together to create a visual ‘storyboard’ showing the type and sequence learning activities required to meet the module’s learning outcomes and how these will be assessed.
    • ABC is particularly useful for new programmes or those changing to an online or more blended format.

    ABC CD workshop resources:

    • ABC CD leaflet (PDF)
    • ABC CD Workshop presentation (PPT)
    • ABC CD graph blend (PDF)
    • ABC learning types cards (PDF)
    • ABC additional online activities (PDF)
    • ABC CC and learning types (PDF)
    • ABC CD worksheet  lansdcape A1 (PDF) (PPT)
    • ABC CD workshop facilitation plan (Word)
    • ABC Action plan (Word)
    • resources checklist

    The resources are also adapted for ABC CPD and Life learning courses.

    Between March and September we had 11 workshops with 37 teams from SLMS and BEAMS.

    The feedback from participants:

    • “This process was really useful. It helps us think about the modules in their entirety. It is really good how everything maps out in a clear framework like this.“
    •  “We haven’t had such level of detailed discussion as a team. I think the structure and the materials are facilitated well. “
      “It is a good way of focusing on creating the balance within a course.“
    • “It makes you think about: OK , we are going to use this technique, but where, how, for what and how does it fit with everything else? And this is the way into that, I think.“
    • “It helped us formulate in our own mind the course structure. Yes, very useful.“
    • “Made me more conscious of a formative assessment, which really did not occur to me before. “
    • “This has been extremely useful. Not only that we start to think about individual modules and how we can use electronic resources, but it makes us think about the degree together, rather than as separate modules. “
    • “It reminds you of all different formats that you can use, rather than sticking to the same old same old.“
    • “I think it was good to take a step back from the content and look at the varied type of activity. “
    • “We are not trying to be very innovative, but it is a question of being open to new ideas“








    *Viewpoints project JISC

    **UCL IoE: Laurillard, D. (2012). Teaching as a Design Science: Building Pedagogical Patterns for Learning and Technology. New York and London: Routledge.

    What is the cost of developing e-learning? Try our calculator

    By Matt Jenner, on 22 July 2015

    Q: What is the cost of developing e-learning?

    A: It depends

    Arghthis answer is not good enough. 

    E-Learning is a big industry, so why does the cost of making ‘some’ feel so mysterious? Increasingly the question of ‘how much will this cost?’ is cropping up. This is a perfectly valid question and one that really demands a better answer than the one above. For too long the response of ‘it depends’ comes up, or something about a piece of string. This isn’t cutting it so after some research (there isn’t much out there) we created an E-Learning Costing Calculator so you can start putting in some numbers and start to see some cold, hard, financials. Hurray?

    Go – play with what we’ve created

    Access E-Learning Costing Calculator on Google Sheets 

    Warning: multiple users will obviously see one another’s calculations but I couldn’t find a better way of doing this while also retaining Alpha status for testing. Ideas welcome in the comments below…

    Images / captures (of the above sheet)

    Main tool, questions and numbers input:

    E-Learning Costing Calculator

    Cost and recovery

    E-Learning Costing Calculator - financials

    Charts for the boss

    Charts for the boss


    Breakdown by role

    Breakdown by role


    If you spend any time in the sheet you’ll notice there are some approximations going on in there (quite a few). It doesn’t produce an exact answer (because it really does depend). I think we’ve been asking the wrong question. We still need to ask – what data do we have to suggest how much e-learning might cost? How can we generalise and remain detailed enough to find ballparks? How close can we get to accuracy? and finally, What are we missing to increase accuracy?

    Disclaimer: so far all the work on this comes from smaller, shorter courses (CPD, continuing education). Moocs and fully accredited courses are slightly different. The biggest problem is to add in some economy of scale (more on this in Maths).

    Seeking improvement

    Firstly – I want people to roadtest this spreadsheet. So please contact me and we can collaborate in Google Sheets (for now). I’m confident we could get a little closer to understanding why and it involves maths, early solutions and more questions.


    Bryan Chapman, Chief Learning Strategist for Chapman Alliance asked in 2010 how long does it take to create e-learning:

    Bryan surveyed 4000 learning development professionals and obtained data (US-based) on CPD and short courses. He created a series of development hour timeframes based on teaching approaches of f2f and three-level e-learning (basic, intermediate and advanced). For each approach he discovered the number of development hours required to create one hour of ‘e-learning’ (vague as it depends on your teaching approach). These numbers were the primary driver to start calculating an idea of costing, and the questions to ask.

    This is the only data found. There’s corporations offering consultancy, and sure they have their ROI models (of course, it’s business). There’s bloggers and co. with their ideas and comments – but nothing with much evidence, especially when compared to Bryan’s work.

    Economy of scale / new vs old

    One problem with all this is that all costs tend to follow the rules of economies of scale. Producing one of anything tends to be proportionally more expensive than 10, then 100, and so on. Logically one hour of e-learning would cost a fair whack – say £15k. But the second should be cheaper, say £10k. Then from here you should see some sliding scales of efficiency. This isn’t so easy to build, so I omitted it in the sheet (for now). Idea welcome on this part.

    New content is probably not the same cost as reusing old. Converting old content vs producing new content both come with different costs. To try and not complicate things it’s best to avoid this question for now, but see a sliding scale could help here – but I don’t know how to calculate the cost of conversion and comparing it to the cost of creation – so it’s lumped in together (for now).


    Running a few generalisations – the data from the Chapman Alliance can be used to start calculating the cost of courses. By taking some known courses, and their approximate costs, we simulated with some UCL courses how much they cost. During a project (UCLeXtend) we had provided some seeding resource to prime the new platform and provide examples to the wider community of what’s possible. Due to the transparency of these courses we could also see how much they all cost, and whether any calculations made were accurate. Sometimes the numbers hurt (never making a profit in this corner…) they also looked kinda accurate.

    This motivated the creation of an E-Learning Costing Calculator – which we’re now crowdsourcing people’s opinions on to improve.


    Armed with one data source (dangerous, I know) I looked to break it back down and discover if it could be reverse-engineered to build a calculator for everyday use. The idea was to ask broad questions within the calculation to then align with the data from the Chapman Alliance’s research. I think there are more questions to ask, but how to also generalise for calculating answers?

    See also

    UCL recently become friends with the IOE. A tool they have is the Course Resource Appraisal Modeller  -it’s much more detailed than this and I think it goes a long way to answering some of the questions I have posed. It also takes a fair amount of time and information to complete it. I can see the validity of both, or (better) one feeding into the other / merging. What do you think? Have you used CRAM? 

    An Example Module in the IOE CRAM tool



    What’s next?

    Please comment on this, in the sheet or in this post (or Twitter). I feel a bit stuck on this now, so feedback is essential to move forward.


    Games, gamification and games-based learning SIG

    By Domi C Sinclair, on 18 June 2015

    Do you have an interest in games, gamification and games-based learning?

    If so we would be really interested in hearing from you, we are looking to put together a special interest group at UCL around these areas. The aim of the SIG will be to encourage interaction and discussion on these topics and others, ranging from research on games and play to their implementation within teaching practice (plus hopefully have a bit of fun along the way).

    Please join via our Moodle page if you are interested in taking part along and we will arrange an initial meeting of the group soon.

    ABC (Arena Blended Connected) curriculum design

    By Natasa Perovic, on 9 April 2015

    The ABC curriculum design method is a ninety-minute hands-on workshop for module (and programme) teams. This rapid-design method starts with your normal module (programme) documentation and will help you create a visual ‘storyboard’. A storyboard lays out the type and sequence learning activities required to meet the module’s learning outcomes and how these will be assessed. ABC is particularly useful for new programmes or those changing to an online or a more blended format.

    The method uses an effective and engaging paper card-based approach based on research from the JISC* and UCL IoE**. Six common types of learning activities are represented by six cards. These types are acquisition, inquiry, practice, production, discussion and collaboration.


    The team starts by writing a very short ‘catalogue’ description of the module to highlight its unique aspects. The rough proportion of each type is agreed (e.g. how much practice, or collaboration) and the envisaged blend of face-to-face and online.


    Next the team plan the distribution of each learning type by arranging the postcard-sized cards along the timeline of the module. With this outline agreed participants turn over the cards. Each card lists online and conventional activities associated with each learning types and the team can pick from this list and add their own.

    workshop team selecting activities

    The type and range of learner activities soon becomes clear and the cards often suggest new approaches. The aim of this process is not to advocate any ‘ideal’ mix but to stimulate a structured conversation among the team.

    Participants then look for opportunities for formative and summative assessment linked to the activities, and ensure these are aligned to the module’s learning outcomes.



    The final stage is a review to see if the balance of activities and the blend have changed, agree and photograph the new storyboard. graph_s

    The storyboard can then be used to develop detailed student documentation or outline a Moodle course (a module in Mooodle).



    The ABC team is developing a program-level version based on the Connected Curriculum principles.

    Participants’ thoughts about ABC curriculum design workshop:


    *Viewpoints project JISC

    **UCL IoE: Laurillard, D. (2012). Teaching as a Design Science: Building Pedagogical Patterns for Learning and Technology. New York and London: Routledge.

    Etymology of the e- in e-learning? Get out.

    By Matt Jenner, on 12 January 2015

    Based on a Christmas conversation about the etymology of emotion (e- = out, motion = move) my mum blurted out, “ah yes, like e-learning?” I wish! The idea of an externalised expression of one’s own learning, a variant on ‘visible learning’ as a colleague would put it, sounds like a no-brainer. I fear, however, that I must have either never clearly explained what e-learning means to my own mother and perhaps I’ve never really thought about it that much myself.


    I presume e-learning meant the same as email, but evidence suggests it might not be. Electronic-learning, mail, commerce or cigarettes are not necessarily using the same e-concatenation. Wikipedia didn’t have the origin or etymology of e-learning, so in true journalistic style, I added the following:

    “The origin or etymology of e-learning is contested, with the e- part not necessarily meaning electronic as per e-mail or e-commerce. Coined between 1997 and 1999, e-learning became first attached to either a distance learning service or it was used for the first time at the CBT systems seminar. Since then the term has been used extensively to describe the use of online, personalised, interactive or virtual education.”

    Others in the educational technology space have suggested more expressive terms for the mysterious e-. These include “exciting, energetic, enthusiastic, emotional, extended, excellent, educational” by Bernard Luskin or “everything, everyone, engaging, easy” by Eric Parks. If there’s no correct answer, we should enjoy that for as long as it lasts. There’s roots into historical computing and educational theory, but the term e-learning doesn’t even seem that old, which is surprising.

    Externalising learning

    In my experience; too much ‘e-learning’ is still long, scrolling pages of PDFs ad infinitum, raw materials made available via online tools and networks. If it’s supporting traditional face to face, I can live with it. But it’s not learning, not without well-constructed, meaningful learning outcomes and activities. Learning outcomes are critical, they link these resources into genuine learning activities that ‘make visible’ or indeed, put an ‘out’ type of e- into e-learning.

    In an online learning environment how do you, or a learner, know anyone has learnt, or done, anything? Externalised learning is surely the key. The idea of ‘making visible’ is critically important, learners should probably not work in isolation for too long. Personal study can still be highly interactive, and have ample opportunities to externalise thoughts, developments, questions, ideas etc. This is all done via the ‘out’, the externalised visible learning.

    Getting there – the importance of learning outcomes

    I’ve seen far too many course descriptions where the learning outcome is to ‘To be able to understand concept X’. Below is an example of how learning outcomes can vary, while all trying to achieve the same goal.


    By the end of this program, successful students will:

      Learning Outcome Analysis
    Option 1: Not an outcome Be given opportunities to learn effective communication skills Describes program content, not the attributes of successful students
    Option 2: Vague Have a deeper appreciation for good communication practices Does not start with an action verb or define the level of learning; subject of learning has no context and is not specific
    Option 3: Less vague Understand principles of effective communication Starts with an action verb, but does not define the level of learning; subject of learning is still too vague for assessment
    Option 4: Specific Communicate effectively in a professional environment through technical reports and presentations Starts with an action verb that defines the level of learning; provides context to ensure the outcome is specific and measurable

    Source – Examples of Learning Outcomes: Good and Bad

    I’m always so happy when I see one that even includes a challenging verb like analyse, classify, interpret, define, create or evaluate and more, more, more, more, etc.

    Writing good outcomes – the foundations of learning

    Writing good learning outcomes still seems like a continuous struggle, but it will be cracked. It will then result in improved online learning environments, structured learning, planned activities and more visible ‘out’ for the e- in e-learning. Or, well, that’s the plan.

    It’s in your Job Description

    Hopefully by next Christmas I’ll be able to explain to my mother what I do for a living, but she still thinks I work in IT. Which reminds me, I don’t think I finished updating her virus definitions either


    Image credit:

    [1] – Out of my mind 2 – Creative Commons openclipart / Creator: mondspeer

    UCL Engineering’s learning technologist initiative – one year on

    By Jessica Gramp, on 9 October 2014

    UCL Engineering’s Learning Technologists have been supporting rapid changes within the faculty. Changes include the development of several new programmes and helping the uptake of technology to improve the turnaround of feedback.

    In late 2013, the UCL Engineering faculty invested in a Learning Technologist post in order to support the Integrated Engineering Programme (IEP), as well as the other programmes within Engineering departments. Since then two Engineering departments, Science, Technology, Engineering and Public Policy (STEaPP) and Management Science and Innovation (MS&I) have both employed Learning Technologists to help develop their e-learning provision. These posts have had a significant impact on the e-learning activities. To evaluate impact on the student learning experience we are collecting information and feedback from students throughout the academic year.

    These three roles complement the UCL-wide support provided by the E-Learning Environments (ELE) team and the Learning Technologists work closely with the central ELE team. This relationship is facilitated by Jess Gramp, the E-Learning Facilitator for BEAMS (Built Environment, Engineering, Maths and Physical Sciences) who co-manages these roles with a manager from each faculty/department. This arrangement enables both formal input from ELE to the departmental activities and plans; and for the learning technologists to receive central mentoring and assistance. Without this structure in place it would be difficult to keep these roles aligned with the many central e-learning initiatives and for the learning technologists to liaise with the technical teams within ISD.

    The initiatives developed by these staff include: designing and implementing Moodle course templates; ensuring adherence to the UCL Moodle Baseline; running training needs analysis and developing staff training plans; delivering e-learning workshops; working with staff to redesign courses, as well as developing them from the ground up, to incorporate blended learning principles; delivering one-to-one support; and working with academics on e-learning projects.

    Moodle Templates
    Engineering now have a Moodle template that provides a consistent experience for students using UCL Moodle to support their learning. This template is now being used on all IEP, MS&I and STEaPP courses and all new Engineering Moodle courses from 2014/15 onwards will also use this template. In some cases the template has been modified to meet departmental requirements.

    Engineering Faculty Moodle template (click to enlarge)

    Engineering Faculty template

    See how MS&I have modified this template and described each feature in their MS&I Moodle Annotated Template document.

    Moodle Baseline course audit
    In MS&I all Moodle courses have been audited against the UCL Moodle Baseline. This has enabled the department’s Learning Technologist to modify courses to ensure every course in the department now meets the Baseline. The template document that was used to audit the courses has been shared on the UCL E-Learning Wiki, so other departments may use it if they wish to do similar. You can also download it here: Baseline Matrix MSI-template.

    Training Needs Analysis
    In STEaPP a Training Needs Analysis was conducted using both a survey and interviews with academics to develop individual training plans for academics and run training workshops specific to the department’s needs. The survey used for this has been shared with colleagues on the UCL E-Learning Wiki.

    Staff e-learning training and support
    In STEaPP a Moodle Staff Hub course has been developed to support staff in their development of courses, including links to e-learning support materials; curriculum development advice; and links to professional development resources. This course has now been duplicated and modified to assist staff across Engineering and within MS&I. If any other UCL faculties or departments would like a similar course set up they can request this be duplicated for them, so they may tailor it to their own requirements. This and other courses are being used to induct new staff to departments and are supported by face to face and online training programmes. The training is delivered using a combination of central ELE training courses and bespoke workshops delivered by Engineering Learning Technologists.

    E-assessment tools to improve the speed of feedback to students
    In MS&I the average turn around for feedback to students is now just one week, significantly shorter than the four week target set by UCL. In order to support this initiative, the department has adopted a fully online assessment approach. This has been achieved predominately using Turnitin, a plagiarism prevention tool that also provides the ability to re-use comments; use weighted grading criteria to provide consistent feedback to students (in the form of rubrics and grading forms); and mark offline using the iPad app. The use of this tool has helped staff to reach the one week feedback target and to streamline the administrative processes that were slowing the feedback process. The Learning Technologist in MS&I has recently arranged workshops with the majority of MS&I staff (including those who are new to UCL) to demonstrate how Turnitin can be used to deliver feedback quickly to students. Several modules within the IEP are also using Moodle’s Workshop tool to enable peer assessment to be managed automatically online. The use of this and other e-assessment tools is saving academics and support staff significant time that used to be spent manually managing the submission, allocation and marking of assessments.

    Technical e-learning support
    While the ELE Services team continues to be the main point of contact for technical e-learning support within Engineering, the local Learning Technologists are able to provide just-in-time support for staff working on local projects. The Learning Technologists are also able to provide assistance beyond what is supported by the central E-Learning team. This includes any development work, such as setting up specific tools within Moodle courses (like the Workshop tool for peer assessment) and setting up groups in MyPortfolio. Development work like these activities fall outside the remit of the central E-Learning Environments team. Also, because the Engineering Learning technologists are based within the faculty, they obtain a better knowledge of local working practices, and are therefore better equipped to understand and support department specific requirements than the central team is able to.

    Project support and funding
    The local Learning Technologists have worked with academics within Engineering to develop bids for Engineering Summer Studentships and other projects, including the E-Learning Development Grants that are distributed yearly by ELE. The successful project proposals have been supported by the local Learning Technologists, which has meant a greater level of support has been provided to the grant winners than has been possible in previous years.

    Using technology to support scenario-based learning
    The Learning Technologist for STEaPP had a unique opportunity to work with staff during the development of their curriculum to ensure that technology was considered at the very outset of the programme’s development. In MS&I the local Learning Technologist has helped to develop a scenario-based, blended-learning course that is now being used as an exemplar of how other academics may redesign their own courses to empower students in their own learning (both electronically and face to face) and provide authentic learning experiences. Many Engineering programmes are already using project-based work to provide students with authentic learning experiences and assessments and this is something the Learning Technologists can work with academics to develop and enhance further.

    Trialing new technologies
    Several e-learning systems have been trialed within Engineering significant input from the Engineering Learning Technologists, including the mobile e-voting system (TurningPoint ResponseWare) for up to 1000 students; and peer assessment of upwards of 700 student videos within the IEP. The successful implementation of such large scale trials would have been difficult without the support of the Learning Technologists.

    E-Learning equipment loans
    One of the common problems with technology uptake is ensuring staff have access to it. Engineering have invested in a number of devices to enable (amongst other things) offline marking; video capture and editing; and presentation of hand drawn figures during lectures. Equipment is available for loan across Engineering and also within STEaPP and MS&I. These include laptops, video recording and editing kit (such as cameras, tripods, microphones and editing software) and iPads. The maintenance and loaning of these are managed by the local Learning Technologists. They are also able to provide advice and assistance with the use of these devices, especially in terms of multimedia creation, including sound recording and filming, and editing of videos to enhance learning resources.

    Working closely with E-Learning Environments and each other
    One important aspect of these roles is that they have close ties to the ELE team, allowing for important two way communication to occur. The Engineering Learning Technologists are able to keep abreast of changes to centrally supported processes and systems and can obtain support from the central E-Learning Environments Services team when required, including receiving train-the-trainer support in order to run workshops locally within Engineering departments. Similarly, ELE benefit by an improved understanding of the activities occurring within faculties and departments, and accessing the materials that are developed and shared by the Learning Technologists.

    Each week the Engineering Learning Technologists share any developments, issues, and updates with each other and the E-Learning Facilitator for BEAMS. The result is a strong network of support for helping to problem solve and resolve issues. It also enables resources, such as the staff hub Moodle course and Moodle auditing matrix, to be shared across the Faculty and more widely across UCL, enabling the re-use of materials and avoiding duplication of effort. The importance of the strong working relationship between the Engineering Learning Technologists became apparent during UCL Engineering’s How to change the world series. During an important final-day session all three Learning Technologists were involved in resolving technical issues to ensure the voting system operated correctly in a venue with incompatible wireless provision.

    UCL staff and students today operate within a rapidly changing educational environment. Both staff and students are expected to understand how to use technology in order to operate within an increasingly digital society. There is a huge number of self directed online learning resources available (such as MOOCs and YouTube videos) and increasingly flexible work and study arrangements are being supported by enhanced technology use. As more staff see the benefits that technology can bring to the classroom, and true blended learning becomes the norm in many areas, it is going to be more important to implement appropriate support structures so staff have the resources to understand and work with these emerging technologies. It is equally important that students are supported in their use of these tools.

    The Learning Technologists within Engineering are in a unique position to understand the opportunities and issues arising in the classroom, and react to these quickly and effectively. We have already seen numerous outputs from these roles. These include a video editing guide to help academics produce professional looking videos for their students; the use of tools within Moodle and MyPortfolio on a scale not seen before with large cohorts of over 700 IEP students; and an exemplar of how scenario-based learning can be supported by technology in MS&I. While these outputs have been developed in reaction to local needs, they have been shared back for others to use and reference, and therefore they benefit the wider UCL community.

    As we see more of these roles implemented across UCL, we will begin to see more dramatic change than has been achievable in the past. One of the plans for the future involves running student focus groups and interviews to better understand how Moodle and other e-learning systems are helping students with their studies and how provision can be improved. The Engineering Learning Technologists will continue their work with local staff to help their departments to use technology more effectively and improve the student experience.