X Close

Digital Education team blog

Home

Ideas and reflections from UCL's Digital Education team

Menu

What IT Directors care about

By Fiona Strawbridge, on 30 October 2016

IMG_7849I heard about the Campus Computing survey for the first time at Educause 2016 – but this survey has been around since 1990 – before, I suspect, the term e-learning had even been coined. This is a survey of CIOs’ (IT Directors’) perspectives on e-learning, amongst other things and I was intrigued to find out what they thought, so went to hear about it from Casey (Kenneth) Green, the Founding Director of CampusComputing.net. I haven’t managed to find the actual survey report, so what follows is a bit patchy, but in essence, CIOs’ have ‘great faith in the benefits of e-learning’, but Learning Analytics keeps them up at night.

Their top five priorities are:

  1. hiring and retaining skilled staff;
  2. assisting academics with e-learning;
  3. the network and data security;
  4. providing adequate user support;
  5. leveraging IT resources to support student success.

The trouble with learning analytics:

CIOs are consistently bothered about their institutions’ ability to deliver learning analytics capabilities and cited concerns with:

  • the infrastructure to deliver them;
  • effectiveness of investment to date;
  • sense of satisfaction with what has been delivered

There was a general sense that their ‘reach exceeded their grasp’ in this area.

What we do vs what we buy:

An interesting observation was that CIOs’ rating of services and facilities that are bought in or outsourced was higher than of those that are developed in house. ‘What we buy works better than what we do’.  Which is perhaps unsurprising, but rather depressing. The service that CIOs were happiest about was wifi!

If I manage to get a link to the report or presentation I will link to it here.

What (American) Students Want

By Fiona Strawbridge, on 30 October 2016

Infographic of ECAR Survey - https://library.educause.edu/resources/2016/6/~/media/files/library/2016/10/eig1605.pdf

ECAR infographic

One motivation for enduring the jet lag and culture shock of a long haul conference is the chance to find out what the big issues are in a different HE environment; Educause is a very good opportunity to do that as it reports on a number of surveys in the world’s largest higher education sector.

So, at this year’s Educause in LA, I went to sessions reporting the results of two very different surveys. One – the ECAR (Educause Center for Analysis & Research) Student Survey – asks students themselves about their attitudes to, experiences of and preferences for using technology in HE – a bit like a tech-focused NSS. The second – CampusComputing.net – surveys IT Directors’ views on e-learning; this seemed, to me, to be a rather odd perspective (why ask CIOs and not heads of e-learning who are closer to the area?).  This post looks at the ECAR student view. To find out what the directors want I’ve written a more sketchy post…

The student survey was completed by a staggering 71,641 students from 183 institutions in 37 states and from 12 countries. The survey is a good benchmarking tool for participating institutions – they are able to compare their results against those from other institutions. Christopher Brookes and Jeffrey Pomerania from Educause presented a whistle-stop tour of the main findings. The full report is at the survey hub, and the infographic shown on the right is a nice summary. There weren’t too many surprises; in a nutshell, students own a lot of devices, and they view them as very important for their learning.

Their devices

In terms of devices, 93% own laptops and a further 3% plan to purchase one, and almost all say they are very or extremely important for their studies. 96 % own smartphones. Tablet ownership is much lower at 57%, and students rated them as less important to their studies than their smartphones. 61% of students have two or three devices, and 33% own four or more. Challenging for wifi, as we know…

Techiness

ECAR looked at techiness (sic) as measured by students’

  1. disposition to technology (sceptic vs cheerleader, technophobe vs technophile etc);
  2. their attitude (distraction vs enhancement, discontented vs contented etc) and
  3. their actual usage of technology (peripheral vs central, never vs alway connected etc).

Since 2014 all three measures have increased – so students are more techie now, and men are more techie than women. As I said, no great surprises.

Students’ experiences of technology

We were told that there was good news about students’ experiences of technology – 80% rated their overall technology experience as good or excellent. Now, it strikes me that if our scores for question 17 in the National Student Survey which asks about technology had been this low (we score 87%) we’d be very seriously concerned – but of course the questions are different so a direct comparison isn’t valid. But a good question is what is actually meant by “students’ experience of technology”. We were told that the main determinants were wifi in halls of residence and on campus, ease of login, having skilled academics, students’ own attitudes to technology, and it helped if technology used in class was perceived as relevant to their career.

Technology in teaching

Around 69% of students said that their teachers had adequate technical skills. More than half reported that technology was being used to share materials (61%) and collaborate (57%). There was less use which encouraged critical thinking (49%) and only a 34% of students said they were encouraged to use their own technology in the classroom.

82% of students reported preferring a blended learning environment over a fully online or fully offline one. Since 2013, the percentage of students who don’t want any online education has halved from around 22% to 11%. The number wanting a fully online experience has dropped slightly, but the number wanting a ‘nearly fully online’ experience has increased; the number wanted a more traditionally blended approach is stable at around 60%. Those who have previous experience of fully online courses are more likely to want a more fully online experience, and women were more likely than men to want to learn online – it was suggested due to a reluctance to speak up in a face-to-face environment.

Students found technology helped them with engagement with academics, with one another, and with content. There were some other interesting demographic effects. Women, first generation students, and non-white students were more likely to say that technology had a positive impact on the efficacy of their learning – it empowered them; it was helpful for communication, for helping them with basic terminology, and for getting swift feedback from others. It was found to enrich the learning experience in many ways.

And finally, students want more:

  • Lecture capture – this mirrors experience at UCL
  • Free, supplemental online content
  • Search tools to find references – this has digital literacy implications as tools exist so perhaps students are unaware.

But, I guess, not more engaging or challenging online learning experiences. Ah well…

Reflections before UCL’s first Mooc

By Matt Jenner, on 26 February 2016

Why We Post: Anthropology of Social Media

Why We Post: Anthropology of Social Media

UCL’s first Mooc – Why We Post: The Anthropology of Social Media launches on Monday on FutureLearn. It’s not actually our first Mooc – it’s not even one Mooc, it’s 9! Eight other versions are simultaneously launching on UCLeXtend in the following languages: Chinese, English, Italian, Hindi, Portuguese, Spanish, Tamil and Turkish. If that’s not enough  we seem to have quite a few under the banner of UCL:

(quite a few of these deserve title of ‘first’ – but who’s counting…)

Extended Learning Landscape - UCL 2015

Extended Learning Landscape – UCL 2015

UCL is quite unique for some of these – we have multiple platforms which form a part of our Extended Learning Landscape. This maps out areas of activity such as CPD, short courses, Moocs, Public Engagement, Summer Schools (and many more) and tries to understand how we can utilise digital education / e-learning with these (and what happens when we do).

 

Justification for Moocs

We’ve not launched our first Mooc (apparently) but we also need to develop a mid term plan too – so we can do more. Can we justify the ones we’ve done so far? Well a strong evaluation will certainly help but we also need an answer to the most pertinent pending question:

How much did all this cost and was it worth it? 

It’s a really good question, one we started asking a while ago, and still the answer feels no better than educated guesswork. Internally we’re working on merging a Costing and Pricing tool (not published, sorry) and the IoE / UCL Knowledge Lab Course Resource Appraisal Modeller (CRAM) tool. The goal is to have a tool which takes the design of a Mooc and outputs a realistic cost. It’s pretty close already – but we need to feed in some localisations from our internal Costing and Pricing tool such as Estates cost, staff wages, Full Economic Costings, digital infrastructure, support etc. The real cost of all this is important. But the value? Well…

Evaluation

We’ve had a lot of ideas and thoughts about evaluation; what is the value of running Moocs for the university? It feels right to mention public engagement, the spirit of giving back and developing really good resources that people can enjoy. There’s the golden carrot being dangled of student recruitment but I can’t see that balancing any Profit/Loss sheets. I do not think it’s about pedagogical innovation, let’s get real here: most Moocs are still a bulk of organised expert videos and text. I don’t think this does a disservice to our Moocs, or those of others, I’d wager that people really like organised expert videos and text (YouTube and Wikipedia being stable Top 10 Global Websites hints at this). But there are other reasons – building Moocs is an new way to engage a lot of people with your topic of interest. Dilution of the common corpus of subjects is a good thing; they are open to anyone who can access them. The next logical step is subjects of fascination, niche, specialist, bespoke – all apply to the future of Moocs.

For evaluation, some obvious things to measure are:

  • Time from people spend on developing the Mooc – we’ve got a breakdown document which tries to list each part of making / running a Mooc so we can estimate time spent.
  • Money spent on media production – this one tends to be easy
  • Registration, survey, quiz, platform usage and associated learner data
  • Feedback from course teams on their experience
  • Outcomes from running a Mooc (book chapters, conference talks, awards won, research instigated)
  • Teaching and learning augmentation (i.e. using the Mooc in a course/module/programme)
  • Developing digital learning objects which can be shared / re-used
  • Student recruitment from the Mooc
  • Pathways to impact – for research-informed Moocs (and we’re working on refining what this means)
  • How much we enjoyed the process – this does matter!

Developing a Mooc – lessons learned

Communication

Designing a course for FutureLearn involves a lot of communication; both internally and to external Partners, mostly our partner manager at FutureLearn but there are others too. This is mostly a serious number of emails – 1503 (so far) to be exact. How? If I knew I’d be rich or loaded with oodles of time. It’s another new years resolution: Stop: Think: Do you really need to send / read / keep that email? Likely not! I tried to get us on Trello early, as to avoid this but I didn’t do so well and as the number of people involved grew adding all these people to a humungous Trello board just seemed, well, unlikely. Email; I shall understand you one day, but for now, I surrender.

Making videos

From a bystander’s viewpoint I think the course teams all enjoyed making their videos (see final evaluation point). The Why We Post team had years to make their videos in-situ from their research across the world. This is a great opportunity to capture real people in the own context; I don’t think video gets much better than this. They had permission from the outset to use the video for educational purposes (good call) and wove them right into the fabric of the course – and you can tell. Making Babies in the 21st Century has captured some of the best minds in the field of reproduction; Dan Reisel (lead educator) knows the people he wants, he’s well connected and has captured and collated experts in the field – a unique and challenging achievement. Tim Shakespeare, The Many Faces of Dementia, was keener to capture three core groups for his course: people with Dementia, their carers / family and the experts who are working to improve the lives for people with Dementia. This triangle of people makes it a rounded experience for any learner, you’ll connect with at least one of these groups. Genius.

Also:

  • Audio matters the most – bad audio = not watching
  • Explain and show concepts – use the visual element of video to show what you mean, not a chin waggling around
  • Keep it short – it’s not an attention span issue, it’s an ideal course structuring exercise.
  • Show your face – people still want to see who’s talking at some point
  • Do not record what can be read – it’s slower to listen than it is to read, if your video cam be replaced with an article, you may want to.
  • Captions and transcripts are important – do as many as you can. Bonus: videos can then be translated.

Using third party works

Remains as tricky as it ever has been. Moocs are murky (commercial? educational? for-profit?) but you’ll need to ask permission for every single third-party piece of work you want to use. Best advice: try not to or be prepared to have no response! Images are the worst, it’s a challenge to find lots of great images that you’re allowed to use, and a course without images isn’t very visually compelling. Set aside some time for this.

Designing social courses that can also be skim-read

FutureLearn, in particular, is a socially-oriented learning platform – you’ll need to design a course around peer-to-peer discussion. Some is breaking thresholds – you’re trying to teach them something important, enabling rich discussion will help. You’re also trying to keep them engaged – so you can’t ask for a deep, thoughtful, intervention every 2 minutes. Find the balance between asking important questions – raising provocative points – and enjoying the fruits of the discussion with the reality of ‘respond if you want’ type discussion prompts.

Connect course teams together

While they might not hold one another’s hair when things get rough – the course teams will benefit from sharing their experiences with one another. We’ve held monthly meetings since the beginning, encouraging each team to attend and share their updates, challenges, show content, see examples from other courses and generally make it a more social experience. Some did share their dropboxes with one another – which I hadn’t expected but am enjoying the level of transparency. I am guilty of thinking at scale at the moment, so while I was guiding and pseudo ‘project-managing’ the courses, I was keen to promote independence and agency within the course teams. It’s their course, they’ll be the ones working into the night on it, I can’t have them relying on me and my dreaded inbox. The outcome is they build their own ideas and shape them in their own style; maybe we’re lucky but this is important. We do intervene at critical stages, recommending approaches and methods as appropriate.

Plan, design and then build

Few online learning environments make good drafting tools. We encouraged a three-stage development process:

  1. Proposals, expanded into Excel-based documents. Outlines each week, the headline for each step/component and critical elements like discussion starters.
  2. Design in documents – Word/Google Docs (whatever) – expand each week; what’s in each step. Great for editorial and refinement.
  3. Build in the platform.

The reason for this is the outlines are usually quick to fix when there’s a glaring structural omission or error. The document-based design then means content can be written, refined and steps planned out in a loose, familiar tool. Finally the platform needs to be played with, understood and then the documents translated into real courses. It’s not a solid process and some courses had an ABC (Arena Blended Connected) Curriculum Design stage, just to be sure a storyboard of the course made sense.

Overall

  • It’s hard work – for the course teams – you can just see they’ll underestimate the amount of time needed.
  • The value shows once you go live and people start registering, sharing early comments on the Week 0 discussion areas.
  • These courses look good and work well as examples for others, Mooc or credit-bearing blended/online courses
  • Courses don’t need to be big – 1/2 hours a week, 2-4 weeks is enough. I’d like to see more smaller Moocs
  • Integrating your Moocs into taught programmes, modules, CPD courses makes a lot of sense

As a final observation before we go live with the first course: Why We Post: The Anthropology of Social Media, on Monday there was one thing that caught my eye early:

Every course team leader for our Moocs is primarily a researcher and their Moocs are produced, largely, from their research activity. UCL is research intensive, so this isn’t too crazy, but we’re also running an institutional initiative the Connected Curriculum which is designed to fully integrate research and teaching. The Digital Education team is keen to see how we build e-learning into research from the outset. This leads us to a new project in UCL entitled: Pathways to Impact: Research Outputs as Digital Education (ROADE) where we’re exploring research dissemination and e-learning objects and courses origins and value. More soon on that one – but our Mooc activity has really initiated this activity.

Coming soon – I hope – Reflections after UCL’s first Mooc 🙂 

 

When does a technology no longer become a technology?

By Matt Jenner, on 15 February 2013

My answer is, ‘when it works’

A question I like to ask when introducing people to new technologies is “Would you consider your chair a technology?” (It works best when they’re sat on an old chair). The most common answer is either ‘no’ or that disgruntled-where’s-the-door-type-face. Regardless, your chair was once designed, the person who made it sat on 100 failures before his posterior was being cushioned by another round of chair developments. Those failures, were in part, technology.

chairs are awesome

When that chair went to the public it might have been as an icon of modern times. Think of plastic injection (chairs dominated the plastics industry for a while) and how many chairs could be made and how quickly. You would only want to do this once they are really, really good. At some point that technology, the new concept, went mainstream. It’s not easy to say when, where or even why it happened – but your bum is grateful for it. To you, it’s just a chair, and that’s if you even bother to register such a thing. Most of the things around you were at some point in their lives a technology.

So what’s technology? Stuff that does not work? Yes, kind of.

Moodle is a technology, we know it works well enough, but it’s got a lot of things to improve. A pencil is not technology, they work, they’re good, maybe a little dated but you can’t alienate someone with a pencil, not quite yet (kids still love them). A teenager might shoot you a wonky face but you’re otherwise OK. What about chalk? What does chalk need to work? A blackboard? Would a youth want to use chalk? Maybe, maybe not.

Where this is heading is to say that technology are the things we’re experimenting with, before they go mainstream. People have been needed to act as technology filters for many years, starting with ‘this stone good, this stone bad’ & recommendations/trial & error/experience/developments from that point on. This filter helped reduce the information about the world to provide selections for people. This is what we do in E-Learning Environments, to some extent. But we also need to be informed of which rock to recommend.

One area we can choose to follow is the NMC Horizon Reports, described as “an unbiased source of information that helps education leaders, trustees, policy makers, and others easily understand the impact of key emerging technologies on education, and when they are likely to enter mainstream use.” These reports have been in print for 11 years to analyse technologies and predict short, medium and long-term technologies or disruptive innovations that educators and other such folk should be aware of. It’s worth noting that when you’re always looking forward at new technologies (aka shiny things) it’s sometimes harder to look back. When in fact, a review of the past often bring a clearer view of the future – or at least lessons to learn from past experiences.

For one reason or another, I decided to compile a list of the technologies featured in the Horizon reports and see if there were any trends in predicted technologies that are no longer ‘technology’. I found six out of 32 technologies that could be considered either no longer technologies, or have arrived to a point where a majority of people may consider them as ‘normal things’, not necessarily ‘new shiny things’ and they’re used in education. I think with the six, I may have been generous – comments welcome.

Here’s a breakdown:

  • Grassroots video (2008, short-term)
  • User-created content (2007, short-term)
  • Social networking (2007, short-term, also 2005 as long-term)
  • Online learning (2005, short-term)
  • Wireless (2005, short-term)
  • Searching (2005, short-term)

These six items have graduated from the Technology Academy, made it in the real world and are embedded into [UCL’s] education. When looking at the year of prediction and the time-frame they were estimated to ‘arrive’ in the world it gives a boost to the predictions from the Horizon reports. Since then they have all disappeared from the annual reports. In my view (don’t expect much more) this means they have landed, delivered their payload and blended into education as fairly normal things. Between 2005 to 2008 the predictions might have been easier to do or there was a point in technology where what came out become mainstream.

Here’s the rest:

Disrupter / Year 2013 2012 2011 2010 2009 2008 2007 2006 2005 2004
Massive open online courses Short
Wearable technology Long
Mobile computing & apps Short Short Short Short Medium Medium
Open content Short
Augmented reality Medium Medium Long Long Long
Electronic books Short Medium
Gesture-based computing Long Long Long Medium
Data analysis and learning analytics Medium Medium Long Long
Game-based learning Medium Medium Medium Long Medium Medium
Tablet computing Short Short
Internet of Things Long Long Long
Cloud computing Short
Geo-everything Medium
Personal web Medium
Semantic aware applications Long Long
Colloration webs Short
Mobile broadband Medium
Data mashups Medium
Collective intelligence Long
Social operating systems Long
Virtual worlds Medium
Publication platforms Long
Social computing Short
Personal broadcasting Short
Scalable vector graphics Short
Rapid prototyping Long Medium

 

As you can see, it’s a bit messy. Many of these technologies, regardless of prediction, never really took off in mainstream education. The most interesting ones were either mega-hyped or echo throughout the years. It’s hard to invest in something like game-based learning as it appears to remain an unpredictable technology for many years. The same can be said for mobile computing and apps, augmented reality, gesture-based computing, internet of things and rapid prototyping. Others seem too brief to know how to respond. I suspect technologies such as open content, cloud computing, collaboration webs, social computing, personal broadcasting and scalable vector graphics may have ignited some as ‘the next best thing’ and then fizzled out before hitting mainstream. Before I get flamed, it’s obvious that some of these things fit into niche areas; and many would argue they are all deployed in some state or another – but I am going for a general view here.

Predicting the unpredictable

Back to my original point is to say that in education, technologies remain as such until they’re used so naturally people see them as a way of life. They’ve come and stayed, maybe no-one even noticed. If you found a student posted a video to a Facebook group then you’ve just seen a few million years of embedded technology all sloshed together in a 1 minute capture. After this all you’re wondering is why they have a weird haircut. That’s embedded technology, working as ‘things’ to you and I.

What’s next?

Probably a lot more new things that don’t work. This is OK, everyone adopts new things at different rates, and this is one of the most fascinating parts. I’ll have to pick that up another time, but it’s all very splendid. I hope MNC keep writing their reports, I want to keep analysing them, I hope they don’t mind…