Participants required for the following study: Uses of learning analytics dashboards / visualisations in student advising

This study has been approved by the UCL Research Ethics Committee, study: 8673/006 and is registered under reference No Z6364106/2020/11/19 social research in line with UCL’s Data Protection Policy.

Learning analytics implementations are predominantly designed for use by those supporting students there is a need to connect the literature on advising and tutoring with any research into the impacts of learning analytics on student behaviour. In terms of both learning behaviours and welfare (wellbeing and mental health).

This research aims to provide an overview of what is currently perceived as best practice in advising and tutoring. In this context, we will critically review the current literature on dashboard/visualisation design and investigate their use by student advisors with the aim of identifying any synergies and conflicts that exist. With the aim of providing recommendations on how to improve dashboard / visualisation design.
The study is looking to recruit HEI Student Advisors (incl. Personal tutors) to share their experiences of using Learning Analytics dashboards/ visualisations. This will initially be via an online survey.

For details of the study please view the study’s Information Sheet.

If you would like to participate in the study please visit the online survey.

The project lead is: Samantha Ahern, Digital Education – Information Services Division

 

 

There are two types of analytics available from within Lecturecast (Echo360), these are:

• Course analytics
• Individual media analytics

The course analytics will provide information about student use of the ALP tools, in addition to interactions with the inidividual media resources. The individual media analytics only provides analytics about the specific media item.

Which set of analytics you use, will depend on how you have added Lecturecast materials to your Moodle course.

If you have embedded recordings in your Moodle course using the Atto editor:

 

 

Then to obtain accurate viewing data for the recording you should use the Individual media analytics.

Please note that students will not have access to the addiitonal tools such as confusion flags and note taking if recordings are added in this way.

Whereas, if you have linked to recordings ore presentations via the Lecturecast activity in Moodle, you would use the Course and student analytics.

The reduction of on-campus teaching and students studying remotely provides a greater emphasis on understanding how they are engaging with the learning activities within their course.

Moodle does provide some tools for this, and anecdotally they are less well known than they might be. The tools range from a quick check on whether students are accessing a particular course, to a much more detailed view of who has completed which activities within a course. They provide a window into how students are doing but of course are only crude proxies for engagement and learning, and should also be supplemented by other information to understand and support our students as best we can.

Here’s a quick guide to three options based on questions you might want to ask.

---

When did students last access this course?

You can get a quick report of the last time each of your students accessed your Moodle course. This is particularly useful at the start of term for highlighting students who have never accessed your course and might be having access difficulties and need further support.

To view this report, go to Course Administration -> Users -> Enrolled Users. The Last Access to Course header of this table is clickable so you can sort by this field (clicking once will show those students who have never accessed this course, or have not accessed this course for the longest times, at the top of the table.

---

Who has or has not viewed or participated in a given activity?

To understand engagement on a specific activity, use Course Participation reports. Find these at Course Administration -> Reports -> Course Participation. You will need to select an activity. In this case, we’re choosing an early activity where students say hello to each other. This is a forum called ‘Getting to know each other’. I select this from the Activity list, and select Students from the ‘Show only’ list, then click Go to get the following report:

This shows your students and indicates whether they have engaged with this activity with a Yes or No. The number in brackets shows the number of engagements, and gives a rough and ready guide, but I wouldn’t take it too literally as different people will normally use websites in different ways. Again the table headers are clickable, so clicking on All actions will order based on that column.

You can drill down a little further by choosing one of the options from the Show actions menu. In this case the options are View and Post. These terms are however slightly misleading, as they count other actions too, but give a broad measure of level of active contribution as compared to reading the work of others.

Finally you can quickly send a message to all students in a No in the actions column, but clicking on Select all ‘No’ and choosing Send a message from the dropdown.

---

How can I see if students have completed the activities across my course?

Activity completion is a tool in Moodle that lets you get an overview of student participation in all activities in your course. This does need to be set up in advance and is very flexible and configurable.

Once activity completion is turned on for your course, each activity in your course is marked either complete or incomplete for each student. Each activity with then be marked as complete based on student activity in a number of different ways:

• The student themselves mark is as complete, using a checkbox next to the activity;
• A simple ‘view’ of the activity marks it as complete;
• A more complex set of criteria is established to mark the activity is complete. This depends on the type of activity but might be getting a certain mark on a quiz, or submitting a document to an assignment, or posting a message in a forum. These criteria are defined by the teacher on an activity-by-activity basis.

An illustrative screenshot of the report available is shown below. Here you can see that one student has gone ahead of the others, two are up to date, one is a little behind, and one has not completed any activities at all.

The video below explains more about activity completion:

There is a Miniguide on activity completion and also a case study from Jane Burns of using it in UCL teaching.

Reporting across Moodle courses

A limitation is that these tools apply at the level of a Moodle course, which is normally a module, and this limits the ability to get an holistic view of the activities of any particular student. We are very aware of this limitation and are rapidly looking at options for providing a more holistic and student-centred reporting ability.

Abstract

Compassion can be defined as “a sensitivity to suffering in self and others with a commitment to try to alleviate and prevent it”(Gilbert, 2017). Compassionate pedagogy could be viewed as a response to a growing sense of zombification of the academy. A universal design for education approach to learning design and resource selection, informed in part by learning analytics, could be considered as components of a compassionate pedagogy. However, as compassion requires an innate motivation, it is this motivation rather than a formal framework or policy requirement that makes these activities the actions of a compassionate pedagogue.

Introduction

The development of massified Higher Education and growing concerns around the increasing use of data in both the ranking and management of Higher Education Institutions (HEIs) has led to a growing body of scholarly work around the notion of the Zombie Academy (Brabazon, 2016)(Moore, Walker, & Whelan, 2013).  Neo-liberal discourse and approaches to governance and accountability are increasingly commoditizing education and reducing the role of the student to consumers whilst simultaneously stripping the function and roles of our HEIs of their social, cultural and political meanings (Moore et al., 2013).

Simultaneously, there is a growing rise in literature around and a move towards compassionate pedagogy. Compassion can be defined as “a sensitivity to suffering in self and others with a commitment to try to alleviate and prevent it”(Gilbert, 2017). Teachers are said to show compassion towards students if they endeavour to see things from the students’ perspective (Waghid, 2014), however this omits the need for motivation to act in a way that is of benefit for students. This is encapsulated in (Hao, 2011)’s definition of Critical Compassionate Pedagogy: “a pedagogical commitment that allows educators to criticize institutional and classroom practices that ideologically underserve students at disadvantaged positions, while at the same time be self-reflexive of their actions through compassion as a daily commitment”.

Being a compassion pedagogue and developing compassionate pedagogy can therefore be said to be about the day-to-day choices made by educators. These choices will include decisions about learning design, selection of learning materials and the use of data to inform learning design and student feedback.

Compassionate Pedagogy in Practice

The increase in the proportion of young adults attending Higher Education Institutions has led to an increasingly diverse student intake (‘Who’s studying in HE?: Personal characteristics | HESA’, n.d.), however this is not always represented in the curricula or in how the curricula are presented to students.

In recent years there has been growing dissatisfaction with what some students describe as ‘pale, male and stale’ curricula. This has resulted in some high profile student campaigns to decolonise the curriculum at a number of leading UK universities including UCL (‘Why is My Curriculum White?’, n.d.) and Cambridge University (https://www.theguardian.com/education/2017/oct/25/cambridge-academics-seek-to-decolonise-english-syllabus), becoming a point of discussion and debate across the sector.

Selecting learning resources and situating learning in a manner that reflects the differing voices, perspectives and experiences of those generating and consuming knowledge are a fundamental part of compassionate pedagogy.

Even if our curricula are representative, how do we ensure an equity of experience for our students? Ableism in academia is endemic and so the concern for equality and equitability is on the increase (Brown & Leigh, 2018).  In 2016/17 12% of students were known to have a disability, many of whom may not have a visible disability (‘Who’s studying in HE?: Personal characteristics | HESA’, n.d.).  Therefore, learning design and design choices made when creating learning resources are also key components of an inclusive, compassionate learning environment. Examples of these choices may include automatically adding closed captions to all videos created by an instructor, avoiding the use of colour to infer meaning, ensuring resources are created in formats that are compatible with institutionally supported accessibility tools or selecting an open textbook as the main course text.

These can both be considered as examples of universal design in education (UDE), where UDE is defined as “the design of educational products and environments to be useable by all people, to the greatest extent possible, without the need for adaptation or specialised design” (Burgstahler, 2015).  This requires the acknowledgement and consideration of the diverse characteristics of all eligible students, these may include ability, language, race, ethnicity, culture, gender, sexual orientation and age. Therefore, the application of universal design principles can be considered an act of compassion.

For a course at a HEI, the products and environment would include the curriculum, facilities and technology used in the course.  At a macro level this may be choosing teaching strategies, and at the micro, facilitating small group discussions.  For example, when using a learning method such as UCL’s ABC method, the products and environments will include considering the variety of learning types selected, the blend of online and offline activity and the assessment load, both formative and summative. The Learning Designer tool enables you to see how much time is spent on tasks and what percentage of directed time is spent on each learning type (‘Learning Designer’, n.d.). Additionally, tools such as the Exclusion Calculator created by the University of Cambridge enables the quantification of accessibility of resources and helps to prioritise improvements.

The role of data

Learning analytics is an ongoing trend and has been identified as one of the ‘Important Developments in Technology for Higher Education’ for 2018/19 (Becker et al., n.d.). Learning analytics has been defined as ‘the measurement, collection, analysis and reporting of data about learners and their contexts, for purposes of understanding and optimising learning and the environments in which it occurs’(Siemens & Gasevic, 2012).

Higher Education Institutions store and generate a plethora of data about students and their interactions with the institution’s IT services and systems. Some of this data can be leveraged by educators to inform their practice and tailor student support. For example, the Echo 360 Active Learning Platform system enables students viewing recordings to flag content that they find confusing.  This data could then be used by the instructor to inform planning for forthcoming lectures or tutorials.  Demographic data could be used to identify students who may need additional support as they may have a specific learning difficulty or be first in family to attend university. It is also possible to identify students who may be over-using resources in an institution’s Virtual Learning Environment, e.g. repeatedly completing the same formative quiz, that may indicate support is required.

This data can be collated for different purposes; automated actions (e.g. email triggers) or as data for humans (e.g. tutors or students themselves) to interpret. An example of automated actions is Newcastle University’s Postgraduate Research Student attendance monitoring process undertaken by the Research Student Support Team (RSST) and the Medical Sciences Graduate School (MSGS). Of the three emails that can be sent to a student, the Level 1 email is an informal automated reminder sent to a student if there has been no recorded and confirmed meetings within 6 weeks (‘Attendance Monitoring’, n.d.).

However, this does not mean that actionable insights will necessarily be drawn or that action will take place. Motivation is required at institutional and practitioner level to make meaningful use of the data, returning us back to our notion of compassionate pedagogy and a motivation to criticize institutional and classroom practices for the benefit of students. An added complication are concerns around HEIs’ obligation to act on any data analyses, in particular providing adequate resources to ensure appropriate and effective interventions (Prinsloo & Slade, 2017).

Conclusion

In this paper we have discussed how accessible learning design and moves to liberate curricula can be perceived as acts of compassion, however these may be undertaken by non-compassionate pedagogues in response to mandated requirements from institutional management, for example UCL’s Inclusive Curriculum Health Check (UCL, 2018), potentially becoming another part of the zombie academy.

Likewise, we have identified that learning analytics can have a role to play. However, it too needs appropriately motivated institutions and staff to utilise this technology in a compassionate manner.

The key notion that separates compassion from empathy or sympathy is the desire to help, or in some definitions motivation to act.  It is this combination of awareness of others and motivation to act in a meaningful way, that determines whether a pedagogue is compassionate or not. These are not things that can be embedded in a formal framework or policy document, but are a culture and mindset that need to be cultivated.

References

Attendance Monitoring. (n.d.). Retrieved 18 September 2018, from https://www.ncl.ac.uk/students/progress/student-resources/PGR/keyactivities/AttendanceMonitoring.htm

Becker, S. A., Brown, M., Dahlstrom, E., Davis, A., DePaul, K., Diaz, V., & Pomerantz, J. (n.d.). Horizon Report: 2018 Higher Education Edition, 60.

Brabazon, T. (2016). Don’t Fear the Reaper? The Zombie University and Eating Braaaains. KOME, 4(2). https://doi.org/10.17646/KOME.2016.21

Brown, N., & Leigh, J. (2018). Ableism in academia: where are the disabled and ill academics? Disability & Society, 33(6), 985–989. https://doi.org/10.1080/09687599.2018.1455627

Burgstahler, S. (2015). Universal design in higher education : from principles to practice / edited by Sheryl E. Burgstahler (2nd ed.). Cambridge, Mass. : Harvard Education Press.

Gilbert, P. (Ed.). (2017). Compassion: Concepts, Research and Applications (1 edition). London ; New York: Routledge.

Hao, R. N. (2011). Critical compassionate pedagogy and the teacher’s role in first‐generation student success. New Directions for Teaching and Learning, 2011(127), 91–98. https://doi.org/10.1002/tl.460

Learning Designer. (n.d.). Retrieved 17 September 2018, from https://www.ucl.ac.uk/learning-designer/index.php

Moore, C., editor of compilation, Walker, R., editor of compilation, & Whelan, A., editor of compilation. (2013). Zombies in the academy : living death in higher education / [edited by] Andrew Whelan, Ruth Walker and Christopher Moore. Bristol : Intellect.

Prinsloo, P., & Slade, S. (2017). An elephant in the learning analytics room: the obligation to act (pp. 46–55). ACM Press. https://doi.org/10.1145/3027385.3027406

Siemens, G., & Gasevic, D. (2012). Guest editorial-Learning and knowledge analytics. Educational Technology & Society, 15(3), 1–2.

UCL. (2018, May 11). New checklist helps staff rate inclusivity of their programmes. Retrieved 17 September 2018, from https://www.ucl.ac.uk/teaching-learning/news/2018/may/new-checklist-helps-staff-rate-inclusivity-their-programmes

Waghid, Y. (2014). Pedagogy Out of Bounds: Untamed Variations of Democratic Education. Sense Publishers. Retrieved from //www.springer.com/la/book/9789462096165

Who’s studying in HE?: Personal characteristics | HESA. (n.d.). Retrieved 9 September 2018, from https://www.hesa.ac.uk/data-and-analysis/students/whos-in-he/characteristics

Why is My Curriculum White? – Decolonising the Academy @ NUS connect. (n.d.). Retrieved 10 September 2018, from https://www.nusconnect.org.uk/articles/why-is-my-curriculum-white-decolonising-the-academy

Participants required for the following study:

What synergies or conflicts exist between current Higher Education Institution Learning Analytics and student wellbeing polices?

As part of an ongoing response to increasing concerns around student wellbeing and mental health UUK, in their September 2017 #StepChange report, recommended the alignment of learning analytics with student wellbeing. However, is it currently possible for these to be aligned?

The aim of this study is to identify the key characteristics of existing policies relating to student wellbeing and learning analytics across the UK Higher Education sector, and the synergies or conflicts that exist between them. This will help to establish whether, at present, learning analytics and student wellbeing initiatives are sufficiently aligned, and if amendments are required to aid alignment.

The study is looking to recruit HEIs who would be willing to share their institutional policies related to student support and wellbeing, and where applicable learning analytics.

For details of the study please view the study’s Information Sheet.

If you would like your institution to participate in the study please complete and return the Registration Form by Monday 21st January 2019.

Participating institutions will be requested to share their policies by Monday 21^st January 2019.

Please return competed registration forms either via email (s.ahern@ucl.ac.uk) or by post to the address below:

Ms S. Ahern

ISD –  Digital Education

UCL, Gower Street, London WC1E 6BT

This project is registered under, reference No Z6364106/2018/11/55 social research in line with UCL’s Data Protection Policy.

‘Samantha is an experienced educator, technologist and creator.‘

This is my standard biog text. Technology is both what I have studied and what I have taught others. The use of technology in learning activities was authentic and integrated into the learning design. Technology, pedagogy and curricula are therefore intrinsically intertwinned.

For meaningful use of technology in teaching and learning these three elements should form a braid.

The 2007 paper What is Technical Pedagogical Content Knowledge? is a good discussion of this interplay and is pretty much how I view the relationship between technology and pedagogy.

When talking about learning and the use of technology in learning I often used the phrase and advocate for ‘pedagogic intent’.

Its a great phrase, but what does it mean?

Lecture capture is very popular with students, and increasing numbers of lectures are recorded.  However, there can be a quite passive use of the technology.

However, it can be used create engagement in the classroom.  The technology becomes part of the pedagogy of the classroom experience.  Our UCL colleague Parama Chaudhury presented a great webinar for the Echo 360 EMEA community on ‘Engaging students with active learning: lessons from University College London’.

This technology can also be used post session to identify content that is that is either difficult, identified by a flag, or of particular interest to students, that could inform future session planning.

Additionally, many taught modules have corresponding Moodle courses.  Although the e-Learning baseline introduces a degree of consistency, these vary immensely in their purpose and content types.

A move towards blended learning designs provides data points that could support post-course review or, perhaps most interestingly, to flag ‘critical-path’ activities (quizzes, forum posts, downloads etc) for intervention in real time. In this case ‘blending’ in online activities becomes an essential part of the student experience.

This identification of course elements of pedagogic interest of existing learning designs and how resulting questions could be answered by the identification of corresponding data points and analysis can be embedded into the learning design process.

The upcoming JISC Data informed blended learning design workshop aims to help participants ensure that their blended learning designs are purposeful. It will seek to make explicit the pedagogic intent in a learning design and explore how data can enable us to understand whether or not learner behaviour is corresponding to those expectations.

Thus returning us to the intertwinned relationship between technology, pedagogy and curricula.