Digital Education team blog
  • We support Staff and Students using technology to enhance education at UCL.

    Here you'll find updates on institutional developments, projects we're involved in, updates on educational technology, events, case studies and personal experiences (or views!).

    Subscribe to our elearning newsletters.

  • Subscribe to this blog

  • Meta

  • Tags

  • A A A

    Archive for the 'External Learning Technology events' Category

    Assessment in Higher Education conference, an account

    By Mira Vogel, on 25 July 2017

    Assessment in Higher Education is a biennial conference which this year was held in Manchester on June 28th and 29th. It is attended by a mix of educators, researchers and educational developers, along with a small number of people with a specific digital education remit of one kind or another (hello Tim Hunt). Here is a summary – it’s organised it around the speakers so there are some counter-currents. The abstracts are linked from each paragraphy, and for more conversation see the Twitter hashtag .

    Jill Barber presented on adaptive comparative judgement – assessment by comparing different algorithmically-generated pairs of submissions until saturation is reached. This is found to be easier than judging on a scale, allows peer assessment and its reliability bears up favourably against expert judgement.  I can throw in a link to a fairly recent presentation on ACJ by Richard Kimbell (Goldsmiths), including a useful Q&A part which considers matters of extrapolating grades, finding grade boundaries, and giving feedback. The question of whether it helps students understand the criteria is an interesting one. At UCL we could deploy this for formative, but not credit-bearing, assessment – here’s a platform which I think is still free. Jill helpfully made a demonstration of the platform she used available – username: PharmEd19 p/ wd: Pharmacy17.

    Paul Collins presented on assessing a student-group-authored wiki textbook using Moodle wiki. His assessment design anticipated many pitfalls of wiki work, such as tendency to fall back on task specialisation, leading to cooperation rather than collaboration (where members influence each other – and he explained at length why collaboration was desirable in his context), and reluctance to edit others’ work (which leads to additions which are not woven in). His evaluation asked many interesting questions which you can read more about in this paper to last year’s International Conference on Engaging Pedagogy. He learned that delegating induction entirely to a learning technologist led students to approach her with queries – this meant that the responses took on a learning technology perspective rather than a subject-oriented one. She also encouraged students to keep a word processed copy, which led them to draft in Word and paste into Moodle Wiki, losing a lot of the drafting process which the wiki history could have revealed. He recommends lettings students know whether you are more interested in the product, or the process, or both.

    Jan McArthur began her keynote presentation (for slides see the AHE site) on assessment for social justice by arguing that SMART (specific, measurable, agreed-on, realistic, and time-bound) objectives in assessment overlook precisely the kinds of knowledge which are ‘higher’ – that is, reached through inquiry; dynamic, contested or not easily known. She cautioned about over-confidence in rubrics and other procedures. In particular she criticised Turnitin, calling it “instrumentalisation\ industrialisation of a pedagogic relationship” which could lead students to change something they were happy with because “Turnitin wasn’t happy with it”, and calling its support for academic writing “a mirage”. I don’t like Turnitin, but felt it was mischaracterised here. I wanted to point out that Turnitin has pivoted away from ‘plagiarism detection’ in recent years, to the extent that it is barely mentioned in the promotional material. The problems are where it is deployed for policing plagiarism – it doesn’t work well for that. Meanwhile its Feedback Studio is often appreciated by students, especially where assessors give feedback specific to their own work, and comments which link to the assessment criteria. In this respect it has developed in parallel with Moodle Assignment.

    Paul Orsmond and Stephen Merry summarised the past 40 years of peer assessment research as ’80s focus on reliability and validity, ’90s focus on the nature of the learning, and a more recent focus on the inseparability of identity development and learning – a socio-cultural approach. Here they discussed their interview research, excerpting quotations and interpreting them with reference to peer assessment research. There were so many ideas in the presentation I am currently awaiting their speaker notes.

    David Boud presented his and Philip Dawson’s work on developing students’ evaluative judgement. Their premise is that the world is all about evaluative judgement and understanding ‘good’ is a premise to producing ‘good’, so it follows that assessment should be oriented to informing students’ judgments rather “making unilateral decisions about students”. They perceived two aspects of this approach: calibrating quality through exemplars, and using criteria to give feedback, and urged more use of self-assessment, especially for high-stakes work. They also urged starting early, and cautioned against waiting until “students know more”.

    Teresa McConlogue, Clare Goudy and Helen Matthews presented on UCL’s review of assessment in a research intensive university. Large, collegiate, multidisciplinary institutions tend to have very diverse data corresponding to diverse practices, so reviewing is a dual challenge of finding out what is going on and designing interventions to bring about improvements. Over-assessment is widespread, and often students have to undertake the same form of assessment. The principles of the review included focusing on structural factors and groups, rather than individuals, and aiming for flexible, workload-neutral interventions. The work will generate improved digital platforms, raised awareness of pedagogy of assessment design and feedback, and equitable management of workloads.

    David Boud presented his and others’ interim findings from a survey to investigate effective feedback practices at Deakin and Monash. They discovered that by half way through a semester nearly 90% of students had not had an assessment activity. 70% received no staff feedback on their work before submitting – more were getting it from friends or peers. They also discovered skepticism about feedback – 17% of staff responded they could not judge whether feedback improved students’ performance, while students tended to be less positive about feedback the closer they were to completion – this has implications for how feedback is given to more advanced undergraduate students. 80% of students recognised that feedback was effective when it changed them. They perceived differences between indvidualised and personalised feedback. When this project makes its recommendations they will be found on its website.

    Head of School of Physical Science at the OU Sally Jordan explained that for many in the assessment community, learning analytics is a dirty word, because if you go in for analytics, why would you need separate assessment points? Yet analytics and assessment are likely to paint very different pictures – which is right? She suggested that, having taken a view of assessment as ‘of’, ‘for’ and ‘as’ learning, the assessment community might consider the imminent possibility of ‘learning as assessment’. This is already happening as ‘stealth assessment‘ when students learn with adaptable games.

    Denise Whitelock gave the final keynote (slides on the AHE site) asking whether assessment technology is a sheep in wolf’s clothing. She surveyed a career working at the Open University on meaningful automated feedback which contributes to a growth mindset in students (rather than consolidating a fixed mindset). The LISC project aimed to give language learners feedback on sentence translation – immediacy is particularly important in language learning to avoid fossilisation of errors. Another project, Open Mentor, aimed to imbue automated feedback with emotional support using Bales’ interaction process categories to code feedback comments. The SAFeSEA project generated Open Essayist which aims to interpret the structure and content of draft essays, identifies key words, phrases and sentences, identifies summary, conclusion and discussion, and presents these to the author. If Open Essayist has misinterpreted the ideas in the essay, the onus is on the author to make amendments. How it would handle some more avant-garde essay forms I am not sure – and this also recalls Sally Jordan’s question about how to resolve inevitable differences between machine and  human judgement. The second part of the talk set out and gave examples of the qualities of feedback which contributes to a growth mindset.

    I presented Elodie Douarin’s and my work on enacting assessment principles with assessment technologies – a project to compare the feedback capabilities of Moodle Assignment and Turnitin Assignment for engaging students with assessment criteria.

    More blogging on the conference from Liz Austen, Richard Nelson, and a related webinar on feedback.

    Fake news and Wikidata

    By Mira Vogel, on 20 February 2017

    James Martin Charlton, Head of the Media Department at Middlesex University and co-host of today’s Wikimedia Education Summit, framed Wikimedia as a defence against the fake news currently spread and popularised by dominant search engine algorithms. Fake news undermines knowledge as power and renders societies easily manipulable. This is one reason several programme leaders I work with – one of whom was at the event – have expressed interest in incorporating Wikimedia into their curricula. (Wikimedia is the collection of projects of which Wikipedia is the best known, but which also includes Wikivoyage, Wikisource and Wikimedia Commons).

    Broadly there are two aspects to Wikimedia in education. One is the content – for example, the articles in Wikipedia, the media in Wikimedia Commons, the textbooks in Wikisource. All of this content is in the public domain, available to use freely in our projects and subject to correction and improvement by that public. The other aspect is process. Contributing to Wikimedia can qualify as higher education when students are tasked with, say, digesting complex or technical information for a non-expert Wikipedia readership, or negotiating changes to an article which has an existing community of editors, or contributing an audio-recording which they later use in a project they publish under an open licence. More recently, Wikidata has emerged as a major presence on the linked and open data scene. I want to focus on Wikidata because it seems very promising as an approach to engaging students in the structured data which is increasingly shaping our world.

    Wikidata is conceived as the central data storage for the aforementioned Wikimedia projects. Unlike Wikipedia, Wikidata can be read by machines as well as humans, which means it can be queried. So if you – as we did today – wish to see at a glance the notable alumni from a given university, you can. Today we gave a little back to our hosts by contributing an ‘Educated at’ value to a number of alumni which lacked it on Wikidata. This enabled those people to be picked up by a Wikidata query and visualised. But institutions tend to merge or change their names, so I added a ‘Followed by’ attribute to the Wikidata entry for Hornsey College of Art (which merged into Middlesex Polytechnic), allowing the query to be refine to include Hornsey alumni too. I also visualised UCL’s notable alumni as a timeline (crowded – zoom out!) and a map. The timeline platform is called Histropedia and is the work of Navino Evans. It is available to all and – thinking public engagement – is reputedly a very good way to visualise research data without needing to hire somebody in.

    So far so good. But is it correct? I dare say it’s at least slightly incorrect, and more than slightly incomplete. Yes, I’d have to mend it, or get it mended, at source. But that state of affairs is pretty normal, as anyone involved in learning analytics understands. And can’t Wikidata be sabotaged? Yes – and because the data is linked, any sabotage would have potentially far reaching effects – so there will need to be defences such as limiting the ability to make mass edits, or edit entries which are both disputed and ‘hot’. But the point is, if I can grasp the SPARQL query language (which is said to be pretty straightforward and, being related to SQL, a transferable skill) then – without an intermediary – I can generate information which I can check, and triangulate against other information to reach a judgement. How does this play out in practice? Here’s Oxford University Wikimedian in Residence Martin Poulter with an account of how he queried Wikidata’s biographical data about UK MPs and US Senators to find out – and, importantly, visualise – where they were educated, and what occupation they’ve had (153 cricketers!?).

    So, say I want to master the SPARQL query language? Thanks to Ewan McAndrew, Wikimedian in Residence at the University of Edinburgh, there’s a SPARQL query video featuring Navino Evans on Edinburgh’s Wikimedia in Residence media channel.

    Which brings me to the beginning, when Melissa Highton set out the benefits Wikimedians have brought to Edinburgh University, where she is Assistant Principal. These benefits include building digital capabilities, public engagement for researchers, and addressing the gender gap in Wikimedia representation, demonstrating to Athena Swann assessors that the institution is addressing structural barriers to women contributing in science and technology. Here’s Melissa’s talk in full. Bodleian Library Web and Digital Media Manager Liz McCarthy made a similarly strong case – they have had to stop advertising their Wikimedian in Residence’s services since so many Oxford University researchers have woken up to Wikimedia’s public engagement potential.

    We also heard from Wikimedians with educational ideas, tutor Stefan Lutschinger on designing Wikimedia assignments, and the students who presented on their work in his Publishing Cultures module – and there were parallel sessions. You can follow the Wikimedia Education Summit tweets at .

    Social Media in Higher Education 2016 (#SocMedHE16)

    By Domi C Sinclair, on 23 December 2016

    Last week was the Social Media in Higher Education 2016 conference, which I was fortunate enough to attend. This was the second year of the conference which took place at Sheffield Hallam University, as it did last year also. The first thing that struck me about this conference was both the variety of different skills and usage levels that the various attendees and presenters had with social media.  Some people where slightly more advanced in their usage of social media, whereas others were just beginning.

    During the conference itself there were 3 key themes that struck me, which I will talk about in more detail in this post. The things that struck me most where;

    1. The importance of students as co-creators
    2. An interesting debate about ‘lurkers’
    3. Discussions about Professionalism

    Let’s start by looking at the first key theme, students as co-creators in more detail.

    1.The importance of students as co-creators

    One of the benefits of social media is that it’s interactive and so anyone can be a content creator. This makes it a powerful tool when used with learners, as they can learn by doing, creating content and even sharing it with the wider world and external subject experts. By engaging in this way student are raising their personal professional profile in their chosen industry and gaining experience. There is also another interesting side-effect of this, especially in relation to projects that are co-run with students, and that is the equalising of staff and students.

    Although there may still be a slight hierarchy on social media, it does tend to place all those using it on a more even platform. Connecting celebrities with fans, and experts with learners whilst enabling them to ask questions or engage in conversation they could not normally have. This idea of students as ‘co’-creators can be really empowering for learners and help develop a confidence and passion for enhancing their learning. Social media as a platform for learning has been seen to encourage heutagogy as it put the learner in a more powerful position of control over their connections and output.

    2. An interesting debate about ‘lurkers’

    One of the discussions I found more interesting was a discussion about ‘lurkers’ on social media platforms, that is learners/ participants who do not actively participate in discussion or other activities but instead only view content. There were three main talking points around this topic; terminology, definition and impact.

    The first discussion is one of terminology, should we use the term lurkers or does this have pejorative connotations? Is a better term, ‘silent participants’ or ‘passive learners’? Does it really matter what we call them? Personally, although I don’t mind the term ‘lurkers’ I do see why some would see it in a negative light and I think maybe ‘silent participant’ is a better term. Although the words we use do have significance, it is also important not to become too distracted by talking semantics at the determent of promoting good pedagogy.

    The next point to consider is what counts as lurking? This seemed obvious to me, but as we moved into a group discussion on the topic it seemed that there are varying opinions on this. Some consider complete inactivity to be lurking, as in someone who reads conversations and consumes other content but does not themselves produce anything to share. This was more my view of what silent participation was before the session, and remains so after. However, I was slightly surprised to hear some proposing that those who ‘like’ content but do not offer content are lurkers. The level of engagement that is required to be shown for someone to be considered active, seemed to be something that everyone did not agree on, but it is a valuable conversation to continue having.

    Finally, it is important to reflect on whether lurking is a bad thing. Do we need to consider ways to ‘lure’ those who lurk into the conversation and encourage them to actively engage? Would this enhance their learning, or are there some people who are happier and just as effective when they are consuming content, rather than producing it. If everyone where producing content, then is there a limit on how many people can be in a class? Surely at a certain class size not everyone can talk at once without diluting the conversation. How do we strike a balance in this case? Personally, I think that all participants should feel they have the opportunity to contribute and engage. For those who are hesitant or resistant we should investigate more closely what is holding them back.

    3.Discussions about Professionalism

    The final thing I want to talk about is the many discussions and presentations that focused on professionalism in the use of social media. This is a very natural topic to be considering in this sort of setting as social media puts learners in the public eye, and what they post could have effects long past their degree.

    The main takeaway here was to avoid simply scaremongering. There are potential risks, and plenty of horror stories but if these are focused on too much in guidelines or workshops it puts social media in a very negative light and can understandably make students resistant or hesitant about using online tools.

    Instead of focusing on the risk, it is good to present students with a realistic balance between the potential risk, so they are aware, and the positive impact social media can have. There are many success stories of students getting job offers and securing careers through their use of social media to share examples of work and connect with employers.

    Overall it was an interesting conference, although it did not add a great deal to my personal understanding of social media, it did prompt me to reconsider some topics that I had not been as actively thinking about (such as the ‘lurker’ debate). If anyone is interested in exploring the use of social media in education then I would recommend looking one of the many excellent books produced on the subject, or contacting the Digital Education team who may be able to offer some advice.

    Wikipedia Course Leaders’ event

    By Mira Vogel, on 15 August 2016

    Wikimedia UK held a Wikipedia Course Leaders event on the afternoon of July 19th. The meeting brought together academics who use Wikipedia in their modules, Wikipedians in Residence, and other Wikipedia and higher education enthusiasts (like me) to exchange their practice and think about some of the challenges of working for assessment in an environment which is very much alive and out in the world.

    As you can imagine, we were all in agreement about the potential of Wikipedia in our respective disciplines, which included Applied Human Geography, Psychology, Law, World Christianity, and Research Methods for Film. As you can see from the notes we took, we discussed colleagues’ and students’ reservations, tensions and intersections between Wikimedia and institutional agendas, relationships between students and other Wikipedians, assessment which is fair and well-supported, and Wikipedia tools for keeping track of students. There are plenty of ideas, solutions, and examples of good and interesting practice. There is a new and developing Wikimedia page for UK universities.

    If you are interested in using Wikipedia to give your students the experience of public writing on the Web and contributing within a global community of interest, there is plenty of support.

    Understanding the essence(s) of portfolio-based learning

    By Domi C Sinclair, on 15 June 2016

    Last week saw the first ever joint AAEEBL and CRA conference, hosted in Edinburgh between 6th – 8th June 2016 whioch was titled, ‘Understanding the essence(s) of portfolio-based learning’. For those who don’t  know AAEEBL is a US based global portfolio organisation, it stands for the Association for Authentic, Experiential and Evidence Based Learning. CRA is a very similar UK based organisation, with it’s name standing for the Centre for Recording Achievement. So, as you can imagine this was a portfolio conference.

    There were 3 key themes that emerged from the conference. These themes kept popping up in presentations and discussion :

    Scaffolding
    Process not product
    Cultural shift/ change

    Let’s look briefly at these themes below, but if you would like a more detailed look them please see the AAEEBL/ CRA Conference 2016 on my personal blog.

    The first theme,  scaffolding, refers to the importance of having structure around portfolio activities. This predominately broke down into conversations about templates and frameworks for guiding staff and students without restricting them. Templates can be useful for giving students a little bit of direction without restricting their creative freedom (depending on the content and detail of the template). They are also useful because, anecdotally, students can find it overwhelming to simply be given a blank space to do with as they please. A template gives students a starting place. In relation to frameworks this was mostly a discussion about their usefulness for staff, to help give them some scaffolding from which to build a portfolio activity into their module or course, either as a single assessment or as an on-going activity to support learning via reflective practice. It was thought that this framework should be fairly high level, meaning it was not too prescriptive and not software dependant.

    This actually leads quite nicely into the next theme, process not product. There was a strong emphasis on focusing on the process and pedagogy of portfolios and not the product (either meaning the final output or the technological product used to facilitate them). It is easy to become distracted by debating whether you are using the best online portfolio system. At the moment UCL use MyPortfolio, which is based on the Mahara platform. As good practice we will be reviewing the use of this platform in the near future, however whether we use Mahara, WordPress or Office 365 the process of running a successful portfolio is the same and the buttons are not as important as strong pedagogy.

    The final theme is perhaps the one that has the biggest impact for portfolio, especially online portfolio adoption at institutions, and that is the need for a cultural shift/ change. This is perhaps best summarised by an analogy that was used by Trent Batson (President/CEO of AAEEBL) at the conference. He was talking about the American automobile and how it took 35 years to become fully part of US culture. First they invented the automobile and it opened up a lot of possibilities, such as people being able to commute more easily for work. But even after this it still took time to build all the roads, parking spaces and petrol stations needed. The idea was proven but it took a lot longer for the infrastructure to become part of daily culture. It is fairly easy to see how this relates to portfolios. There are a number of case studies out there to prove their potential, however the infrastructure to support them is not fully part of the culture of universities. Portfolios tend to expose the learning process which can be an intimidating prospect for both students and staff a like. However, portfolios can offer a very useful reflective space where you can use journals to do written reflections, and also reflect whilst curating examples of work you have produced that you are going to include in your portfolio. Reflection gives us the ability to stop and think about our thinking, and to understand how we can do better moving forward.

    ABC reaches Glasgow… and Santiago!

    By Clive Young, on 23 May 2016

    (For latest news about ABC LD, visit ABC LD blog)

    Nataša Perović and I took UCL’s popular ABC learning design workshop on the road last week, on Friday running a session for the first time outside UCL. We were invited to the University of Glasgow by ex-UCL colleague Dr Vicki Dale, now with their Learning Technology Unit. Vicki had seen the workshop running in London and was keen to try it with her colleagues. 32 participants came from all four of Glasgow’s colleges and the energy in the room was remarkable and  indicative of the huge interest generated. We were pleased to see the method was as “really useful” for Glasgow participants (see below) as we have found it with UCL colleagues.

    glasgow

    In a curious coincidence on Friday the ABC method was also used for the first time abroad, this time in Santiago, Chile. Robert Pardo, Director of the Centro de Aprendizaje, Universidad Adolfo Ibáñez, had taken our materials, translated them into Spanish and after a couple of Skype sessions with us ran the workshop very successfully with a group of his colleagues. His conclusion? ” It works!! “

    chile