X Close

Digital Education team blog

Home

Ideas and reflections from UCL's Digital Education team

Menu

Archive for the 'Learning analytics' Category

Lecturecast: what analytics should you use?

Samantha Ahern22 October 2020

There are two types of analytics available from within Lecturecast (Echo360), these are:

The course analytics will provide information about student use of the ALP tools, in addition to interactions with the inidividual media resources. The individual media analytics only provides analytics about the specific media item.

Which set of analytics you use, will depend on how you have added Lecturecast materials to your Moodle course.

If you have embedded recordings in your Moodle course using the Atto editor:

Atto editor toolbar highlighting Echo360 plugin

 

 

Then to obtain accurate viewing data for the recording you should use the Individual media analytics.

Please note that students will not have access to the addiitonal tools such as confusion flags and note taking if recordings are added in this way.

Whereas, if you have linked to recordings ore presentations via the Lecturecast activity in Moodle, you would use the Course and student analytics.

Understanding student activity in Moodle

Steve Rowett7 August 2020

The reduction of on-campus teaching and students studying remotely provides a greater emphasis on understanding how they are engaging with the learning activities within their course.

Moodle does provide some tools for this, and anecdotally they are less well known than they might be. The tools range from a quick check on whether students are accessing a particular course, to a much more detailed view of who has completed which activities within a course. They provide a window into how students are doing but of course are only crude proxies for engagement and learning, and should also be supplemented by other information to understand and support our students as best we can.

Here’s a quick guide to three options based on questions you might want to ask.


When did students last access this course?

You can get a quick report of the last time each of your students accessed your Moodle course. This is particularly useful at the start of term for highlighting students who have never accessed your course and might be having access difficulties and need further support.

To view this report, go to Course Administration -> Users -> Enrolled Users. The Last Access to Course header of this table is clickable so you can sort by this field (clicking once will show those students who have never accessed this course, or have not accessed this course for the longest times, at the top of the table.

Table showing students enrolled on a Moodle course, including their most recent access to that course


Who has or has not viewed or participated in a given activity?

To understand engagement on a specific activity, use Course Participation reports. Find these at Course Administration -> Reports -> Course Participation. You will need to select an activity. In this case, we’re choosing an early activity where students say hello to each other. This is a forum called ‘Getting to know each other’. I select this from the Activity list, and select Students from the ‘Show only’ list, then click Go to get the following report:

A Moodle course participation report, showing that some students have been much more active in a discussion forum than others

This shows your students and indicates whether they have engaged with this activity with a Yes or No. The number in brackets shows the number of engagements, and gives a rough and ready guide, but I wouldn’t take it too literally as different people will normally use websites in different ways. Again the table headers are clickable, so clicking on All actions will order based on that column.

You can drill down a little further by choosing one of the options from the Show actions menu. In this case the options are View and Post. These terms are however slightly misleading, as they count other actions too, but give a broad measure of level of active contribution as compared to reading the work of others.

Finally you can quickly send a message to all students in a No in the actions column, but clicking on Select all ‘No’ and choosing Send a message from the dropdown.


How can I see if students have completed the activities across my course?

Activity completion is a tool in Moodle that lets you get an overview of student participation in all activities in your course. This does need to be set up in advance and is very flexible and configurable.

Once activity completion is turned on for your course, each activity in your course is marked either complete or incomplete for each student. Each activity with then be marked as complete based on student activity in a number of different ways:

  • The student themselves mark is as complete, using a checkbox next to the activity;
  • A simple ‘view’ of the activity marks it as complete;
  • A more complex set of criteria is established to mark the activity is complete. This depends on the type of activity but might be getting a certain mark on a quiz, or submitting a document to an assignment, or posting a message in a forum. These criteria are defined by the teacher on an activity-by-activity basis.

An illustrative screenshot of the report available is shown below. Here you can see that one student has gone ahead of the others, two are up to date, one is a little behind, and one has not completed any activities at all.

Activity completion report in Moodle showing which students have completed which activity according to criteria set by the teacherThe video below explains more about activity completion:

There is a Miniguide on activity completion and also a case study from Jane Burns of using it in UCL teaching.

Reporting across Moodle courses

A limitation is that these tools apply at the level of a Moodle course, which is normally a module, and this limits the ability to get an holistic view of the activities of any particular student. We are very aware of this limitation and are rapidly looking at options for providing a more holistic and student-centred reporting ability.

Compassionate Pedagogy in Practice

Samantha Ahern3 July 2019

Abstract

Compassion can be defined as “a sensitivity to suffering in self and others with a commitment to try to alleviate and prevent it”(Gilbert, 2017). Compassionate pedagogy could be viewed as a response to a growing sense of zombification of the academy. A universal design for education approach to learning design and resource selection, informed in part by learning analytics, could be considered as components of a compassionate pedagogy. However, as compassion requires an innate motivation, it is this motivation rather than a formal framework or policy requirement that makes these activities the actions of a compassionate pedagogue.

Introduction

The development of massified Higher Education and growing concerns around the increasing use of data in both the ranking and management of Higher Education Institutions (HEIs) has led to a growing body of scholarly work around the notion of the Zombie Academy (Brabazon, 2016)(Moore, Walker, & Whelan, 2013).  Neo-liberal discourse and approaches to governance and accountability are increasingly commoditizing education and reducing the role of the student to consumers whilst simultaneously stripping the function and roles of our HEIs of their social, cultural and political meanings (Moore et al., 2013).

Simultaneously, there is a growing rise in literature around and a move towards compassionate pedagogy. Compassion can be defined as “a sensitivity to suffering in self and others with a commitment to try to alleviate and prevent it”(Gilbert, 2017). Teachers are said to show compassion towards students if they endeavour to see things from the students’ perspective (Waghid, 2014), however this omits the need for motivation to act in a way that is of benefit for students. This is encapsulated in (Hao, 2011)’s definition of Critical Compassionate Pedagogy: “a pedagogical commitment that allows educators to criticize institutional and classroom practices that ideologically underserve students at disadvantaged positions, while at the same time be self-reflexive of their actions through compassion as a daily commitment”.

Being a compassion pedagogue and developing compassionate pedagogy can therefore be said to be about the day-to-day choices made by educators. These choices will include decisions about learning design, selection of learning materials and the use of data to inform learning design and student feedback.

Compassionate Pedagogy in Practice

The increase in the proportion of young adults attending Higher Education Institutions has led to an increasingly diverse student intake (‘Who’s studying in HE?: Personal characteristics | HESA’, n.d.), however this is not always represented in the curricula or in how the curricula are presented to students.

In recent years there has been growing dissatisfaction with what some students describe as ‘pale, male and stale’ curricula. This has resulted in some high profile student campaigns to decolonise the curriculum at a number of leading UK universities including UCL (‘Why is My Curriculum White?’, n.d.) and Cambridge University (https://www.theguardian.com/education/2017/oct/25/cambridge-academics-seek-to-decolonise-english-syllabus), becoming a point of discussion and debate across the sector.

Selecting learning resources and situating learning in a manner that reflects the differing voices, perspectives and experiences of those generating and consuming knowledge are a fundamental part of compassionate pedagogy.

Even if our curricula are representative, how do we ensure an equity of experience for our students? Ableism in academia is endemic and so the concern for equality and equitability is on the increase (Brown & Leigh, 2018).  In 2016/17 12% of students were known to have a disability, many of whom may not have a visible disability (‘Who’s studying in HE?: Personal characteristics | HESA’, n.d.).  Therefore, learning design and design choices made when creating learning resources are also key components of an inclusive, compassionate learning environment. Examples of these choices may include automatically adding closed captions to all videos created by an instructor, avoiding the use of colour to infer meaning, ensuring resources are created in formats that are compatible with institutionally supported accessibility tools or selecting an open textbook as the main course text.

These can both be considered as examples of universal design in education (UDE), where UDE is defined as “the design of educational products and environments to be useable by all people, to the greatest extent possible, without the need for adaptation or specialised design” (Burgstahler, 2015).  This requires the acknowledgement and consideration of the diverse characteristics of all eligible students, these may include ability, language, race, ethnicity, culture, gender, sexual orientation and age. Therefore, the application of universal design principles can be considered an act of compassion.

For a course at a HEI, the products and environment would include the curriculum, facilities and technology used in the course.  At a macro level this may be choosing teaching strategies, and at the micro, facilitating small group discussions.  For example, when using a learning method such as UCL’s ABC method, the products and environments will include considering the variety of learning types selected, the blend of online and offline activity and the assessment load, both formative and summative. The Learning Designer tool enables you to see how much time is spent on tasks and what percentage of directed time is spent on each learning type (‘Learning Designer’, n.d.). Additionally, tools such as the Exclusion Calculator created by the University of Cambridge enables the quantification of accessibility of resources and helps to prioritise improvements.

The role of data

Learning analytics is an ongoing trend and has been identified as one of the ‘Important Developments in Technology for Higher Education’ for 2018/19 (Becker et al., n.d.). Learning analytics has been defined as ‘the measurement, collection, analysis and reporting of data about learners and their contexts, for purposes of understanding and optimising learning and the environments in which it occurs’(Siemens & Gasevic, 2012).

Higher Education Institutions store and generate a plethora of data about students and their interactions with the institution’s IT services and systems. Some of this data can be leveraged by educators to inform their practice and tailor student support. For example, the Echo 360 Active Learning Platform system enables students viewing recordings to flag content that they find confusing.  This data could then be used by the instructor to inform planning for forthcoming lectures or tutorials.  Demographic data could be used to identify students who may need additional support as they may have a specific learning difficulty or be first in family to attend university. It is also possible to identify students who may be over-using resources in an institution’s Virtual Learning Environment, e.g. repeatedly completing the same formative quiz, that may indicate support is required.

This data can be collated for different purposes; automated actions (e.g. email triggers) or as data for humans (e.g. tutors or students themselves) to interpret. An example of automated actions is Newcastle University’s Postgraduate Research Student attendance monitoring process undertaken by the Research Student Support Team (RSST) and the Medical Sciences Graduate School (MSGS). Of the three emails that can be sent to a student, the Level 1 email is an informal automated reminder sent to a student if there has been no recorded and confirmed meetings within 6 weeks (‘Attendance Monitoring’, n.d.).

However, this does not mean that actionable insights will necessarily be drawn or that action will take place. Motivation is required at institutional and practitioner level to make meaningful use of the data, returning us back to our notion of compassionate pedagogy and a motivation to criticize institutional and classroom practices for the benefit of students. An added complication are concerns around HEIs’ obligation to act on any data analyses, in particular providing adequate resources to ensure appropriate and effective interventions (Prinsloo & Slade, 2017).

Conclusion

In this paper we have discussed how accessible learning design and moves to liberate curricula can be perceived as acts of compassion, however these may be undertaken by non-compassionate pedagogues in response to mandated requirements from institutional management, for example UCL’s Inclusive Curriculum Health Check (UCL, 2018), potentially becoming another part of the zombie academy.

Likewise, we have identified that learning analytics can have a role to play. However, it too needs appropriately motivated institutions and staff to utilise this technology in a compassionate manner.

The key notion that separates compassion from empathy or sympathy is the desire to help, or in some definitions motivation to act.  It is this combination of awareness of others and motivation to act in a meaningful way, that determines whether a pedagogue is compassionate or not. These are not things that can be embedded in a formal framework or policy document, but are a culture and mindset that need to be cultivated.

References

Attendance Monitoring. (n.d.). Retrieved 18 September 2018, from https://www.ncl.ac.uk/students/progress/student-resources/PGR/keyactivities/AttendanceMonitoring.htm

Becker, S. A., Brown, M., Dahlstrom, E., Davis, A., DePaul, K., Diaz, V., & Pomerantz, J. (n.d.). Horizon Report: 2018 Higher Education Edition, 60.

Brabazon, T. (2016). Don’t Fear the Reaper? The Zombie University and Eating Braaaains. KOME, 4(2). https://doi.org/10.17646/KOME.2016.21

Brown, N., & Leigh, J. (2018). Ableism in academia: where are the disabled and ill academics? Disability & Society, 33(6), 985–989. https://doi.org/10.1080/09687599.2018.1455627

Burgstahler, S. (2015). Universal design in higher education : from principles to practice / edited by Sheryl E. Burgstahler (2nd ed.). Cambridge, Mass. : Harvard Education Press.

Gilbert, P. (Ed.). (2017). Compassion: Concepts, Research and Applications (1 edition). London ; New York: Routledge.

Hao, R. N. (2011). Critical compassionate pedagogy and the teacher’s role in first‐generation student success. New Directions for Teaching and Learning, 2011(127), 91–98. https://doi.org/10.1002/tl.460

Learning Designer. (n.d.). Retrieved 17 September 2018, from https://www.ucl.ac.uk/learning-designer/index.php

Moore, C., editor of compilation, Walker, R., editor of compilation, & Whelan, A., editor of compilation. (2013). Zombies in the academy : living death in higher education / [edited by] Andrew Whelan, Ruth Walker and Christopher Moore. Bristol : Intellect.

Prinsloo, P., & Slade, S. (2017). An elephant in the learning analytics room: the obligation to act (pp. 46–55). ACM Press. https://doi.org/10.1145/3027385.3027406

Siemens, G., & Gasevic, D. (2012). Guest editorial-Learning and knowledge analytics. Educational Technology & Society, 15(3), 1–2.

UCL. (2018, May 11). New checklist helps staff rate inclusivity of their programmes. Retrieved 17 September 2018, from https://www.ucl.ac.uk/teaching-learning/news/2018/may/new-checklist-helps-staff-rate-inclusivity-their-programmes

Waghid, Y. (2014). Pedagogy Out of Bounds: Untamed Variations of Democratic Education. Sense Publishers. Retrieved from //www.springer.com/la/book/9789462096165

Who’s studying in HE?: Personal characteristics | HESA. (n.d.). Retrieved 9 September 2018, from https://www.hesa.ac.uk/data-and-analysis/students/whos-in-he/characteristics

Why is My Curriculum White? – Decolonising the Academy @ NUS connect. (n.d.). Retrieved 10 September 2018, from https://www.nusconnect.org.uk/articles/why-is-my-curriculum-white-decolonising-the-academy

Call for Participants

Samantha Ahern19 November 2018

Participants required for the following study:

What synergies or conflicts exist between current Higher Education Institution Learning Analytics and student wellbeing polices?

As part of an ongoing response to increasing concerns around student wellbeing and mental health UUK, in their September 2017 #StepChange report, recommended the alignment of learning analytics with student wellbeing. However, is it currently possible for these to be aligned?

The aim of this study is to identify the key characteristics of existing policies relating to student wellbeing and learning analytics across the UK Higher Education sector, and the synergies or conflicts that exist between them. This will help to establish whether, at present, learning analytics and student wellbeing initiatives are sufficiently aligned, and if amendments are required to aid alignment.

The study is looking to recruit HEIs who would be willing to share their institutional policies related to student support and wellbeing, and where applicable learning analytics.

For details of the study please view the study’s Information Sheet.

If you would like your institution to participate in the study please complete and return the Registration Form by Monday 21st January 2019.

Participating institutions will be requested to share their policies by Monday 21st January 2019.

Please return competed registration forms either via email (s.ahern@ucl.ac.uk) or by post to the address below:

Ms S. Ahern

ISD –  Digital Education

UCL, Gower Street, London WC1E 6BT

This project is registered under, reference No Z6364106/2018/11/55 social research in line with UCL’s Data Protection Policy.

Loader Loading...
EAD Logo Taking too long?

Reload Reload document
| Open Open in new tab

Download

Loader Loading...
EAD Logo Taking too long?

Reload Reload document
| Open Open in new tab

Download

TPCK, data and learning design

Samantha Ahern13 February 2018

Samantha is an experienced educator, technologist and creator.

This is my standard biog text. Technology is both what I have studied and what I have taught others. The use of technology in learning activities was authentic and integrated into the learning design. Technology, pedagogy and curricula are therefore intrinsically intertwinned.

For meaningful use of technology in teaching and learning these three elements should form a braid.

The 2007 paper What is Technical Pedagogical Content Knowledge? is a good discussion of this interplay and is pretty much how I view the relationship between technology and pedagogy.

When talking about learning and the use of technology in learning I often used the phrase and advocate for ‘pedagogic intent’.

Its a great phrase, but what does it mean?

Lecture capture is very popular with students, and increasing numbers of lectures are recorded.  However, there can be a quite passive use of the technology.

However, it can be used create engagement in the classroom.  The technology becomes part of the pedagogy of the classroom experience.  Our UCL colleague Parama Chaudhury presented a great webinar for the Echo 360 EMEA community on ‘Engaging students with active learning: lessons from University College London’.

This technology can also be used post session to identify content that is that is either difficult, identified by a flag, or of particular interest to students, that could inform future session planning.

Additionally, many taught modules have corresponding Moodle courses.  Although the e-Learning baseline introduces a degree of consistency, these vary immensely in their purpose and content types.

A move towards blended learning designs provides data points that could support post-course review or, perhaps most interestingly, to flag ‘critical-path’ activities (quizzes, forum posts, downloads etc) for intervention in real time. In this case ‘blending’ in online activities becomes an essential part of the student experience.

This identification of course elements of pedagogic interest of existing learning designs and how resulting questions could be answered by the identification of corresponding data points and analysis can be embedded into the learning design process.

The upcoming JISC Data informed blended learning design workshop aims to help participants ensure that their blended learning designs are purposeful. It will seek to make explicit the pedagogic intent in a learning design and explore how data can enable us to understand whether or not learner behaviour is corresponding to those expectations.

Thus returning us to the intertwinned relationship between technology, pedagogy and curricula.

 

Learning Analytics as a tool for supporting student wellbeing – Learning Analytics and Student Mental Health & Wellbeing

Samantha Ahern20 November 2017

Learning analytics is defined as ‘the measurement, collection, analysis and reporting of data about learners and their contexts, for purposes of understanding and optimising learning and the environments in which it occurs’(“Guest Editorial – Learning and Knowledge Analytics. Educational Technology & Society, 15 (3), 1–2. Siemens, G., & Gašević, D. (2012)”, n.d.).

Applications of learning analytics include Early Alert and Student Success, Course Recommendation, Adaptive Learning and Curriculum Design(Sclater, 2017).

Can and should this learning analytics be extended to identify normative behaviours of students and recognise changes to those behaviours, aiding pastoral support?

Although much of the data for informing pastoral support is the same as that for Early Alert and Student Success the aims and implications are different. The data needs may also be more demanding. For example, there will be additional considerations around data sharing and protection as mental ill-health is classified as a protected characteristic.

For inclusion of engagement data from virtual learning environments, this would involve understanding the seasonality of student interactions with online course content, cohort interactions, how a student’s interactions are differing from both their cohort and their own normative behaviour with respect to the seasonality.

Prinsloo and Slade (Prinsloo and Slade, 2017) note that ‘Not  only  do various  stakeholders  in  the  institution  work  in  silos,  responding independently  of  each  other and resulting  in  overlap and inconsistencies, institutional  sense-making  of  students  at  risk is  also  fragmented’, which may hinder student well-being support.

Evidence on the effectiveness of learning analytics based interventions in unclear. A systematic review and quality assessment of studies on learning analytics in higher education by the University of Exeter(Sonderlund and Smith, 2017) was only able to include 20 of 560 papers identified due to the methods employed in the studies, only 4 studies evaluated the effectiveness of interventions based on learning analytics. The key recommendation from the review is that more research into the implementation and evaluation of scientifically-driven learning analytics to build a solid evidence base.

The combination of the lack of evidence of the effectiveness of learning analytics based interventions and the potential negative consequences for both our students and institutions, therefore causes us to question whether learning analytics should be used to support student mental well-being.

Conclusion

Student mental wellbeing and in particular student mental ill-health is of major concern, with 48% of UK HEIs having appropriate policies in place(Universities, 2015), and continually needs to be addressed.

An “unhelpful divide” of distinguishing intellectual needs from emotional needs, then students mental health may suffer if the emotional needs are ignored(“What Happened to Pastoral Care? | HuffPost UK”, n.d.). Therefore, pastoral care in addition to academic support is crucial student mental wellbeing.

The Higher Education Academy UK Professional Standards Framework (“UK Professional Standards Framework (UKPSF) | Higher Education Academy”, n.d.) dimension A4, Developing effective learning environments and approaches to student guidance,  indicates that student support is an area of activity in which those teaching and supporting learning in higher education should be involved. Additionally, the Universities UK #stepchange(“#stepchange”, n.d.) guidance states that HEIs should seek to promote a diverse, inclusive and compassionate culture as part of their preventive actions.

Unfortunately, there are a number of inadequacies with the current provision of pastoral care in UK Higher Education Institutions.  I propose that learning analytics can be used to help to address some of these inadequacies by providing timely and meaningful data to personal tutors about their tutees., this is in alignment with the Univerisites UK guidance(“#stepchange”, n.d.) to align learning analytics with student wellbeing. However, this will require action on behalf of tutors, and there are legal questions still to be answered around negligence and failing to act on or engage with information provided via learning analytics.

Despite the potential ethical and legal issues around using learning analytics to support pastoral care and student mental wellbeing, I believe that this application area that should be explored.

References:

Loader Loading...
EAD Logo Taking too long?

Reload Reload document
| Open Open in new tab

Download