Authors: Michelle Lai, Luke Dickens, Bonnie Buyuklieva, Janina Dewitz, Karen Stepanyan

Introduction

Accommodating neurodiversity in higher education often requires reasonable adjustments that take the form of additional time for completing the work, alternative assessment or examination settings, and the use of assistive technology among other options. An alternative approach to accommodating neurodiversity is to ensure that the material given to all students is universally accessible. We here examine AI based tools for improving the language used in tests to better support those with neurodiverse characteristics.

There is a significant proportion of the student body with neurodiverse characteristics. The statistics reported by HESA on self-disclosed learning disabilities such as dyslexia, dyspraxia, or AD(H)D show an increase of 26% between 2014-15 and 2021-22 (Tang et al., 2024). The total number of students with such learning differences in 2022-23 stands at 140,745 corresponding to roughly 6.5% of the total student population (HESA, 2025). Moreover, others have argued that this number is likely to underestimate the issue (Devine, J., 2024).

Neurodiverse characteristics, such as those associated with AD(H)D, autism, and dyslexia, have been linked to learning challenges associated with differences in cognitive function including: executive function (Southon, C., 2022), test anxiety (Mahak, S., 2024), and sensory sensitivities resulting in susceptibility to distraction (Irvine, B. et al., 2024.). This had led to calls for a revised approach to learning and assessment to include considerations of neurodiversity, e.g. (Sewell, A. and Park, J., 2021). Beyond academia, industry recruitment practices have shifted towards specifically taking into account neurodiversity of graduates as a way of obtaining greater competitive advantage (Borrett, 2024). Many universities are now leaning into offering greater support. In practice this takes the form of technology-based interventions, comprehensive support programmes, and transition into university and then employment (McDowall & Kiseleva, 2024). As a result of these changes the education sector is undergoing a paradigm shift that moves away from the traditional teacher-centred education to inclusive environments in Higher Education (Tang, Griffiths & Welch, 2024).

One approach to making education environments more inclusive is applying Universal Design principles to teaching materials. The principles of Universal Design advocate for developing environments that can be accessed, understood, and used to the greatest extent possible by all people, regardless of their ability or disability. Designs suitable for all people suggest that learning environments should strive to be as inclusive as possible, benefiting all stakeholders to the greatest degree possible, and reducing specific adjustments as much as possible (UCL-ARENA, 2024; Center for Universal Design, n.d.). This framework prioritises making the language of assessments more ‘neurodiversity-friendly’. We specifically focus on exploring the relevance of generative AI approaches for reformulating questions that are identified as problematic for neurodiverse cohorts. This project was funded by the UCL Centre for Humanities Education, which enabled us to conduct a case-study that looked at current examples of assessment, more specifically a sample of Multiple Choice Questions (MCQs), and the potential for making these more accessible to all students. In this case study, we look at two AI products that market themselves as generating textual outputs in neurodiversity-friendly language. We investigate how their suggested changes compared with the original questions set by two lecturers of technical subjects at UCL. The study was carried out by a funded postgraduate student and supported by academic staff at the Department of Information Studies, UCL, who co-authored this report.

Can AI Tools Make Assessment More Accessible?

Informed by the current body of research and by surveying available AI tools for diverse classroom applications (see Appendix 1), we shortlisted three tools based around two AI products (see Table 1). We then assessed whether these tools could make multiple-choice and short-answer questions written for assessment in technical modules more neurodiversity-friendly.

Our focus was on the AI tools that purport to be useful for making reading and language easier for neurodivergent people. We adopted Hemingway Editor and Goblin Tools Formaliser in our study. The latter product was tested under two modes: ‘More Accessible’ and ‘More to the point’ (see Table 1).

AI Tool	AI Technology Type	Short Description
Hemingway Editor	Rule based readability & style analyser. Light AI proofreading.	Live colour‑coding of “hard‑to‑read” sentences, passive voice, over‑used adverbs. Grade‑level calculation. AI pass for spelling/grammar and optional paraphrases (paid tier only).
Goblin: More accessible	LLM‑powered text simplification.	Rewrites any passage into plainer language at a lower reading level while preserving meaning.
Goblin: To the point	LLM‑based concision / summarisation.	Strips filler, tangents and redundancies, returning essence‑only version of the text.

Table 1: AI tools examined in this research, which were selected from the list of tools with the greatest potential to support language processing needs of neurodiverse students. Note: the full list of AI Tools considered for examination in this research is presented in Appendix 1.

Evaluation and Results

We combined the questions into question sets based on the topic and word length to accommodate the limitations of the tools. There were 7 sets of questions:

• Web Tech. 1, comprises 7 theoretical and introductory multiple-choice questions (MCQs) on the topic of the Internet and Web technologies;
• Web Tech. 2, comprises 7 MCQs containing HTML code;
• Web Tech. 3, comprises 7 theoretical MCQs on accessibility;
• Stats. 1, comprises a single MCQ in Statistics, which included a longer scenario and a longer text for the MCQ options;
• Stats. 2, comprises 4 MCQs on fundamental concepts in Statistics;
• Stats. 3, comprises on 2 MCQs that required simple statistical calculations;
• Stats. 4, comprises 3 MCQs that can be attributed to set theory – a foundational area of Mathematics.

The question bank comprises 31 questions, and was taken from an introductory web technologies modules taught at an undergraduate level, and an introductory statistics module taught to masters students, both at UCL.

Set	Name	Count	Length	Hemingway Editor				Goblin: More accessible				Goblin: To the point
				Red	Orange	Green	No change	Red	Orange	Green	No change	Red	Orange	Green	No change
1	Web Tech. 1	7	12.7	1	4	2	0	1	0	4	2	0	0	2	5
2	Web Tech. 2	7	19.4	1	1	4	1	0	1	5	1	0	1	5	1
3	Web Tech. 3	7	28.4	2	0	4	0	0	2	4	0	1	2	4	0
4	Stats. 1	1	99	0	0	1	0	0	0	1	0	0	0	1	0
5	Stats. 2	4	45.8	1	2	1	0	0	3	1	0	0	0	4	0
6	Stats. 3	2	59.5	0	0	2	0	0	0	2	0	0	0	2	0
7	Stats. 4	3	45	1	1	1	0	0	1	2	0	0	0	3	0

Table 2: Summary of studied question sets, including the number of questions in each set (Count), the average length of text in words (Length), and frequency of accuracy flags (Red, Orange and Green) by adopted AI tool for each question set or frequency of output with no change.

We passed each of the questions into the chosen three AI tools. We, therefore, had upto 93 suggested revisions along with the original set of 31 questions. Table 2 shows the number of questions (count) and average word count (length) for each set, along with the accuracy flags discussed further down.

One indication of how straightforward some text is to read is captured by the FORCAST (FORmula for Forecasting Average Single-Syllable word Count) readability formula (Scott, n.d.). We calculated the FORCAST scores for the original questions as well as the questions modified by the AI tools (Figure 1). A higher FORCAST score indicates a greater difficulty of written text.

While FORCAST scores can be useful, it is not always possible to gauge whether the output is clearer or more concise. However, there is no widely accepted framework for such an assessment, and we had to develop our own method for doing this consistently. Our assessment of clarity and conciseness was inspired by the earlier work of Bischof & Eppler (2011). However, we adopted a single Likert scale score for human judgment of each aspect for simplicity.

1	Language used is unclear, inappropriate, grammatically incorrect, or awkward.
3	Appropriate language and sentence structure is used. Certain words or turn of phrase are potentially awkward or incorrect, but do not impact the overall readability and understanding.
5	Appropriate language and sentence structure is used. Exceptionally easy to understand with no errors.

Table 3: Likert scale, used in evaluating Clarity of Language.

1	Overly wordy and convoluted. Severely impacts / delays the reader’s understanding.
3	Uses words of an appropriate difficulty for the topic at hand, with a standard / acceptable number of words to convey the intended meaning.
5	Uses the simplest and least number of words needed to convey the intended meaning.

Table 4: Likert scale, used in evaluating Conciseness.

All original and AI-generated question sets were assessed by one of the researchers using these scores for clarity and conciseness. For our analysis, a researcher was asked to rate the question for clarity on the given Likert scale (see Table 3). Next a researcher was asked to rate each question for conciseness on a Likert scale (Table 4). Lower scores, therefore, indicate a lack of clarity or conciseness.

The results of the mean scores of the Clarity and Conciseness evaluations are presented for all seven sets of questions in Figure 2 and Figure 3 respectively.

Finally, for each question in the sets the original authors of the test questions were asked if the questions produced by the AI tools retained the intended meaning with three possible judgement flags: green – the question retained the original meaning, orange – there were some issues with the meaning, and red – the question was no longer accurate given the context of the question. A final flag, gray, was given to questions that were unchanged by the tool. As indicated above, Table 2 presents the accuracy flag counts of the evaluated tools for each set. For readability these data are also plotted in Figure 4. It is clear from these results that these tools can perform differently across different types of questions, and no one tool performed without issues. Hemingway performed particularly badly on theoretical questions (Sets 1 and 3). It is also clear that Goblin: To the point performed well on questions containing code (Set 2). Overall accuracy, regardless the types of questions, does appear to be higher for both Goblin tools. However, this result should be viewed with caution due to a very small sample of questions used in the study. Furthermore, it is also important to study individual questions in greater detail to try and elicit why some of the questions were “tricky” for the AI tools.

Preliminary Findings

The study highlighted that the adoption of Generative AI tools for making text accessible to neurodiverse audiences does not necessarily offer improvements. The FORCAST scores for some of the sets increased (see Sets 2 & 4), making the questions potentially more complex as a result of using AI tools. However, occasionally (see Sets 1, 2 and 6, Figure 1) the use of AI-generated content has offered improvements in FORCAST scores. In some cases, the original questions scored worse on clarity and conciseness than the AI-generated alternatives (see Set 4, Figure 2 & 3). Similarly, improvements in clarity were evident in some instances too (see Set 2, 4 & 5 in Figure 2). Most importantly, however, the noted improvements were rather small, with the range of variation in scores remaining minimal for all of the adopted metrics.

Goblin ‘more to the point’ had a higher conciseness score overall (Figure 3), but had a higher FORCAST score as well, meaning that a higher grade / education level was required to read it. The FORCAST formula uses the number of single-syllable words to calculate a score, which may be why a more concise output using less ‘extra’ words will generate a higher FORCAST score.

Tool Specific Findings

Hemingway Editor

Hemingway Editor is a tool that offers readability statistics with the premium membership, including AI rewrite suggestions. This AI tool uses popular AI services, namely OpenAI, Anthropic, and Together. The interface of the Hemingway editor is easy to navigate, and is customisable. When feeding the questions in, the system recognised that the text was meant for a university-level reader and adjusted its suggestions accordingly. However, questions that contained HTML/CSS code were not processed correctly, failing to escape the tags, and changing the meaning of the question as a result.

Goblin Tools Formaliser: More Accessible

Goblin tools are a set of single-task tools that aim to break down and simplify tasks for neurodivergent people, and use “models from different providers, both open and closed source.” (goblin-tools, n.d.). The Formaliser tool converts texts with up to 15 prompts, such as ‘more accessible’ and ‘more to the point (unwaffle)’. There is a ‘spiciness level’ which controls how strongly the generated text will come across; for this study, it was set to the lowest level.

The ‘more accessible’ option often changed words to more commonly used and ‘lower grade’ words, as well as changing sentence structures. While this generally improved the readability and clarity of the outputs, the accuracy was sometimes affected, given the highly technical language of many questions.

Goblin Tools Formaliser: More to the Point

The ‘straight to the point’ Goblin Tools option often made little to no changes to the shorter questions, which suggests that in most cases the original question author had set questions using concise language. There tended to be no changes in wording, but this meant that the accuracy was usually unaffected, while the clarity of the question did not improve. For longer questions, there was a greater change in wording, which resulted in greater variation in clarity, and higher conciseness scores.

In addition, while the use of generative AI tools did not offer consistent performance in improving the FORCAST, clarity and conciseness scores, the review of generated alternative wording suggested by the tool were at times viewed as useful for formulating alternative ways of phrasing the questions.

Additional Reflections and Summary

The literature often refers to the clarity and conciseness as factors that could affect a neurodivergent student’s understanding of a question, relating to processing speed and cognitive load (e.g. Fakhoury et al., 2018).

In summary, our study suggests that generative AI tools can be useful as scaffolding tools for inclusive assessment design, when paired with informed human judgement. The Hemingway and Goblin Tools occasionally improved clarity, conciseness or readability, but introduced new difficulties related to the use of code snippets or altering meaning. It appears that generative AI can support producing neurodiversity‑friendly assessments, but it should only be adopted in an assistive role.

***

Dr Bonnie Boyana Buyuklieva FHEA, FRGS is a Lecturer (research) in Data Science for Society at UCL’s Department of Information Studies. Bonnie’s background is in architecture and computation (Foster and Partners; Bauhaus University Weimar), and she holds a PhD from the Bartlett Centre for Advanced Spatial Analysis (CASA, UCL).

Janina Dewitz has worked for over a decade as an Innovations Officer at UCL, having previously worked as a Learning Technologist and Learning Facilitator at Barking and Dagenham College. Janina earned a Bachelor’s in Humanities from the University of Hertfordshire in 1998 and a Higher National Certificate in Performing Arts from Barking & Dagenham College in 2004.

Dr Luke Dickens is an Associate Professor at UCL’s Department of Information Studies. Luke is also a Founding member and co-lead of the Knowledge Information and Data Science (KIDS) research group at UCL, and a Founding member of the cross-institutional Structured and Probabilistic Intelligent Knowledge Engineering (SPIKE) research group based at Imperial College.

Michelle Lai recently completed a Masters of Arts in Special and Inclusive Education (Autism) at UCL, having previously earned a BSc in Psychology with Education, Educational Psychology. Michelle is interested in how psychology can inform and enhance inclusive practices in special education and currently works as a Specialist Teaching Assistant at Ambitious about Autism.

Dr Karen Stepanyan is an Associate Professor (Teaching) in Computing and Information Systems. Based at the Department of Information Studies, he is leading the delivery of the Web Technologies (INST0007) module and Database Systems (INST0001).  He contributed to the development of the BSc Information in Society programme at UCL East. His research spans the inter-relation of information technologies and the concepts of knowledge.

***

References:

UCL-ARENA. (2024). “Inclusive education: Get started by making small changes to your education practice.” Teaching and Learning Retrieved April 1 2025, from https://www.ucl.ac.uk/teaching-learning/news/2024/nov/inclusive-education-get-started-making-small-changes-your-education-practice.

Bischof, N., & Eppler, M. J. (2011). Caring for Clarity in Knowledge Communication. J. Univers. Comput. Sci., 17(10), 1455-1473.

Borrett, A. (2024, Dec. 19) ‘UK employers eye “competitive advantage” in hiring neurodivergent workers’, Financial Times, accessed July 17 2025, Available at: https://www.ft.com/content/e692c571-b56b-425a-a7a0-3d8ae617080b.

Center for Universal Design. (n.d.). College of Design. https://design.ncsu.edu/research/center-for-universal-design/. Retrieved April 8, 2025 from https://design.ncsu.edu/research/center-for-universal-design/.

Devine, J. (2024). Comment: Neurodiversity and belongingness. Buckinghamshire New University. Retrieved July 18, 2025, from https://www.bucks.ac.uk/news/comment-neurodiversity-and-belongingness

Fakhoury, S., Ma, Y., Arnaoudova, V., & Adesope, O. (2018, May). The effect of poor source code lexicon and readability on developers’ cognitive load. In Proceedings of the 26th conference on program comprehension (pp. 286-296).

goblin-tools. About (n.d.). Retrieved July 18, 2025, from https://goblin.tools/About

HESA (Higher Education Statistics Agency). (2025, April 3). Who’s studying in HE?: Personal characteristics. Retrieved July 18, 2025, from https://www.hesa.ac.uk/data-and-analysis/students/whos-in-he/characteristics#breakdown

McDowall, A., & Kiseleva, M. (2024). A rapid review of supports for neurodivergent students in higher education. Implications for research and practice. Neurodiversity, 2. https://doi.org/10.1177/27546330241291769 (Original work published 2024)

Scott, B. (n.d.). Readability Scoring System PLUS. https://readabilityformulas.com/readability-scoring-system.php.

Tang, E. S. Y., Griffiths, A., & Welch, G. F. (2024). The Impact of Three Key Paradigm Shifts on Disability, Inclusion, and Autism in Higher Education in England: An Integrative Review. Trends in Higher Education, 3(1), 122-141. https://doi.org/10.3390/higheredu3010007

***

Appendix 1:

List of AI Tools considered for this study and classified by category (i.e. EF – Executive Function (planning, prioritizing, time management, task initiation); WM – Working Memory / information overload; LP – Language Processing / writing / reading load; SN – Sensory / modality flexibility (visual, auditory, text, speech); ANX – Reducing Anxiety (performance, math, participation); SOC – Social Communication load (participate without speaking live, review later); MATH – Math Conceptual/step scaffolding; ACC – General accessibility (alt input, captions, screen-reader friendly, etc.))

Tool	Short description	AI Tool Categories
Conker.ai	One‑click generator of differentiated quizzes & question banks	ANX, WM
Consensus AI	Answers research questions by ranking peer‑reviewed evidence	WM
Desmos	Web graphing calculator with interactive sliders & tables	MATH, SN
DreamBox	Adaptive K‑8 maths lessons that adjust every 60 seconds	MATH, ANX
Elicit AI	Semantic‑searches papers and auto‑extracts key findings	WM
Explain Paper	Click‑highlight any PDF sentence to get plain‑English explainer	LP
fireflies.ai	Live meeting recorder that transcribes, timestamps and summarises team calls	WM, SOC, ACC
GeoGebra	Dynamic geometry & graphing suite—visual proofs, 3‑D, AR	MATH, SN
goblin.tools Chef	Generates grocery lists & cooking steps from meal ideas	EF
goblin.tools Compiler	Condenses chat or notes into tidy bullet points	WM
goblin.tools Estimator	Predicts how long a task list will really take	EF
goblin.tools Formaliser	Rewrites text into more formal / academic register	LP
goblin.tools Judge	Rates whether your instructions are “clear enough”	EF
goblin.tools MagicToDo	Turns a vague task into an ordered, timed action plan	EF
goblin.tools Professor	Explains any concept at a chosen complexity level	WM
Grammarly	Real‑time grammar, tone and clarity checker inside browsers & docs	LP
Hemingway Editor	A readability tool that color‑codes dense or passive sentences, flags adverbs, and shows grade level so writers can simplify and clarify their prose.	LP
Heuristica	Generates interactive concept maps you can chat with	WM
IXL	Skills‑driven practice that levels up as students gain accuracy	MATH
Julius AI	Chat‑style data analyst that cleans, queries and plots spreadsheets	WM
Knewton Alta	Mastery‑based, adaptive courseware for math & science in HE	MATH
Litmaps	Builds visual citation maps to reveal research connections	WM
MathGPTPro / Mathos AI	LLM tutor that OCR‑reads handwritten maths and explains steps	MATH
MATHia	Carnegie Learning’s AI tutor that coaches each step of algebra problems	MATH, ANX
MathPapa	Symbolic algebra solver that shows stepwise solutions	MATH
Motion	AI calendar that auto‑schedules tasks against deadlines and reshuffles as priorities change	EF
Nuance Dragon Speech	High‑accuracy speech‑to‑text dictation across OS‑level apps	ACC, LP
Otter.ai	Live captioning & searchable transcripts for meetings and lectures	WM, SOC, ACC
PhET Interactive Simulations	Free, click‑and‑drag science & maths models (HTML5)	SN
SciSpace	All‑in‑one platform to search 200 M papers and ask PDFs questions	WM
Scite.AI	Shows whether later studies support or contradict a cited paper	WM
Scribe (To‑do)	Breaks complex goals into step‑by‑step checklists and sets reminders	EF
Scribe (Writing helper)	Generates SOPs and how‑to docs from screen recordings and prompts	LP, WM
Semantic Scholar (TLDR)	Gives one‑sentence abstract of any research paper	WM
SmartSparrow	Learner‑authored adaptive modules with branching feedback	WM
Socratic (Google)	Mobile camera‑based homework helper with brief video explainers	MATH, WM
Symbolab	Multi‑step calculator covering calculus, series, matrices, proof	MATH
Synthesia	Turns typed scripts into captioned avatar videos in 120+ languages	SN, ACC
Tavily	Real‑time search API built for LLMs & agents, returns source snippets	WM
Topmarks	Curated hub of short interactive literacy & numeracy games	SN, MATH
TXYZ.ai	AI “research OS” that finds, organises and chats over papers	WM
Unity ML‑Agents Toolkit	Open‑source SDK for building reinforcement‑learning–driven 3‑D sims and games	SN
Writer	Enterprise‑grade generative writing assistant with style‑guide enforcement	LP

 

In their latest research project, Abbi Shaw (UCL Arts & Humanities Faculty Learning Technology Lead) and Jesper Hansen (UCL Arts & Humanities Arena Faculty Lead) surveyed staff in UCL Arts & Humanities about their experiences of feedback and assessment. Abbi and Jesper’s research sheds light on what staff see as constituting effective feedback and how they have experienced student engagement, or the lack thereof, with feedback. The report raises important questions for future discussions concerning student engagement, staff workloads, inclusivity, and teaching processes at UCL Arts & Humanities. To find out more about their findings, we asked Abbi and Jesper some key questions about their work:

1. To start, can you tell us a bit about the background and main objectives of your 2024/25 feedback and assessment survey of UCL Arts & Humanities?

Over the last four years, we have put in a lot of work to centre the actual experiences of students and staff in our Faculty. This is important as it links research done in other places with the reality at UCL Arts & Humanities, which is often both similar and different from other universities and faculties. We have found that this focus facilitates discussions about the direction the Faculty should take in terms of education, and it makes the data more pertinent to all relevant stakeholders: we can actually point to what our own students and staff are telling us. Another advantage is that we have been able to significantly shorten the delay between our research and the dissemination of our findings. Normally there would be years between research being done and findings being published in a peer-reviewed journal. By contrast, our research is normally disseminated in the term following data collection.

This year’s topic came about for two reasons. Firstly, feedback and assessment are at the top of UCL’s priorities and, as such, it makes sense to align our work with it. And, secondly, it is an area where we, as a Faculty, see students express concerns in the National Student Survey. While we are doing better than some other Faculties, there is plenty of room for improvement. When coupled with the fact that feedback and assessment are very important for students and affect their overall experience of studying, it just made sense.

2. What does effective feedback look like?

There is plenty of research on effective feedback, and we know that clarity and actionability are key. This means that students have to understand the feedback and be able to use it to improve their future work. Professor David Boud, one of the world’s leading experts on feedback, spoke at UCL’s Education Conference some years ago. He argues that something cannot be considered feedback unless it has an impact on the student. At UCL, we now ask staff to think about feedback from two different perspectives: a mark justification, which helps students understand the mark they received, and developmental feedback, which helps students improve and, ideally, achieve a better mark in the future. The bespoke part of feedback is then relevant because no two students are the same: if we rely too heavily, for instance, on pre-written comments, it risks not being particularly developmental for the student reading it.

In our Faculty, we know that staff spend a lot of time giving feedback, and we know that there is a desire to help students improve. So this is not, we would argue, about us not trying or not wanting to give good and effective feedback, but about a disconnect between what we are doing and what students expect and see as useful.

3. How do workload, time constraints, and other barriers impact staff’s ability to provide effective feedback?

Given what we outlined above, it might sound like staff are being asked to spend more time giving feedback. But that is not the case: more feedback does not equal better and more effective feedback, if anything it’s often the other way around. We know from other research that students generally do not request large amounts of feedback from their tutors. Rather, they want targeted feedback that clearly explains the mark (and here we suggest linking feedback directly to the marking criteria), and some specific ideas. A good example could be a list of 3-4 bullet points which explain how their work can be improved in future, ideally with some signposting of further support. If you find yourself writing longer feedback, sometimes a list of bullet points can be used to sum up your main action points: ‘Based on the feedback I have given you above and in the margins, I suggest you consider the following three points in your next assignment…’

Staff answering our survey discussed two distinct barriers to providing effective feedback. One concerned anonymity: how do we give bespoke feedback when we don’t know who we are talking to? This is a topic that is being debated across the sector. While we certainly are asked to use anonymous assessment where possible, there are some caveats and workarounds. Firstly, UCL’s academic manual clearly states that we can use non-anonymous marking where there are good reasons for doing so. This might be if you are doing continuous assessment, where students work on their assessment over time (this already happens in the Faculty in, for instance, the English department’s tutorial system). Secondly, some tutors use feedback cover sheets to give students a clearer stake in the feedback process. This happens a lot in, for instance, the Institute of Education, and it might be something we can learn from in A&H. Examples of things students might be asked could be:

• Are there specific areas you would like me to comment on (such as referencing, your engagement with sources, your introduction and conclusion)?
• What did you find particularly challenging in this assignment?
• Briefly outline the feedback you have got in your previous assignments and how you have responded to it in this assignment.
• What format would you like your overall feedback comments in: written or recorded audio?
  

  Fountain pen, courtesy of Petar Milošević via Wikimedia Commons.

The other barrier that staff mentioned was the knowledge that students often don’t read the feedback we give. It can be very disillusioning if we believe that students ignore the feedback we invest an enormous amount of time and energy to create. The way we see it, we could choose to blame the students. Or we could try to understand why they are not reading feedback and discuss how we can make it more attractive for them to engage with it. This question of why some students don’t see value in engaging with their feedback is one that we will explore further this term as we survey students in the Faculty. We will report back on our findings, of course, but we also invite staff to consider how we can change our feedback processes to align more with students’ needs.

4. At the day-to-day level of teaching, how might we build a shared understanding between students and staff of what feedback is and how it can help student progress?

When thinking about how others involved in the teaching, learning and assessment of students might support the development of student feedback literacy, we can look at managing their expectations around feedback and demonstrating putting feedback into practice – for instance, by working through a marked demonstration piece of work. We can also look at developing students’ own capacity to give feedback as a method of supporting their understanding of it, and its relevance. This might be through responding to student work such as presentations, or forum posts, or as we see in the Slade School of Fine Art, where students regularly undertake a collective feedback-giving process, by making space throughout the module for students to respond to each other’s work and ideas.

5. Many respondents believed formative assessment was useful yet expressed caveats about the process. What steps can be taken to improve engagement with formative assessment among students?

Many of our students are high achievers who are very strategic about their time. They all have competing demands on their time: studying for classes, preparing for assessments, social life, jobs (part-time or full-time), caring responsibilities and so on. If we consider formative assessment within this matrix, is it really a surprise that many students choose not to do it?

If we want students to engage with formative tasks – and we do want that, as we know it is beneficial for students – we probably need to rethink them and, just like with feedback, consider how we can make them more attractive to students. Firstly, we need to think about how we introduce these tasks to students. Do we tell them they are optional and not that important? Or do we emphasise that they are expected to do them (just like they are expected to prepare for classes)? And do we explain how doing the formative will be helpful when it comes to completing summative tasks?

6. If, in an ideal hypothetical world, you could make one innovative and less conventional change to UCL Arts & Humanities’ assessment processes, what would it be?

We have already made some really positive strides in the Faculty where we now discuss topics like feedback and assessment much more than we did just a handful of years ago. This is important as feedback and assessment cannot be addressed by looking at one or two parameters – they are too complex for that. What we need is a review of how assessment is used in modules, how module assessments come together at the programme level, and how feedback is central to students’ academic journeys. This will, potentially, be effective in ensuring that new students don’t lose faith in the feedback they get and therefore want to engage with it; which will, in turn, shape how staff engage with the feedback. Concrete examples might include how personal tutors support student feedback literacy, or how staff office hours are used more strategically.

7. What’s next for this project?

We intend to run a survey of A&H students in order to improve our understanding of their experiences. As we go through processes of curricular improvement including PEP2, and EASE, we have opportunities to gather and examine the data around assessment in modules and programmes, as described above. We will also continue to discuss how we can best support the Faculty in resolving some of the issues highlighted in our staff survey and developing students’ relationships with feedback.

 

Macarena Jiménez Naranjo promised her student, Nadia Hussain, and the rest of Nadia’s classmates that they would receive full marks for one part of their assessments — simply for turning up to class. Listen to find out how this bold decision lifted the pressure of grades, fostered student-led scholarly exploration, and drew students back into the classroom after the fracturing effects of the COVID-19 pandemic.

Participants: Mazal Oaknín, Macarena Jiménez Naranjo, and Nadia Hussain (all from UCL’s Department of Spanish, Portuguese and Latin American Studies)

by Selena Daly (SELCS) 

The second meeting of the Creative Teaching in the Humanities Network, in November 2023, focused on assessments and featured two speakers: Dr Akil Awan, Associate Professor of Modern History and Political Violence in the Department of History, Royal Holloway, University of London and Dr Eleanor Chiari, Associate Professor (Teaching) at SELCS in UCL.

Developing the ‘Unessay’

Akil presented his use of the ‘Unessay’ as part of the assessment for his module on the history of terrorism from the 19^th century to today. Instead of submitting a traditional written essay, students were asked to complete an ‘Unessay,’ essentially a creative and critical engagement with any theme from the module. Possible formats could include a piece of artwork, a documentary, a graphic novel, a website, or a short story, among many other possibilities.

Among examples of some of the best work students submitted as ‘Unessays’ were the following: a debate between a perpetrator and victim of terrorism in Northern Ireland written as a play; a board game in which you get to play as British colonial forces or the ‘Mau Mau’ or Kenyan Land and Freedom Army; and a musical composition focusing on the immediate public responses to the 9/11 attacks and remembrance of victims of terrorism. Students were also required to submit a 500-word self-reflexive essay worth 25% of the grade.

Engaging with trauma in visual culture

Eleanor presented an assessment that features as part of an undergraduate module entitled ‘Trauma in Visual Culture,’ which had similarities to the Unessay approach presented by Akil but adapted to her particular module’s context. Its aim was to encourage students to reflect more critically on emotive side of visual culture.

Students were required to submit a portfolio of work that responded to the module’s themes and theories examined. Examples of student work included: a graphic novel-style reinterpretation of Art Spiegelman’s Maus to explore the theme of post memory in the context of the Troubles in Northern Ireland; a visual journal exploring the haunting legacy of Nic Ut’s ‘terror of war’ photograph from the war in Vietnam; and a video essay which explored the idea of the ‘illogical spectator’ using family home videos from before the Syrian war. If students opted to submit an entirely abstract piece, they were required to submit a 1,500-word essay on their work.

Both Akil and Eleanor identified similar advantages to adopting these kinds of creative assessments. Both highlighted their value in catering to a diverse student cohort and the way that they foster creativity, imagination, and experimentation. Creative assessments also allow students to make use of skills they may have developed in other aspects of their lives (e.g. music or art), allowing for more holistic learning.

The approach also encourages students to engage more personally with the module content and Eleanor highlighted how, for some students who accessed family stories, the assessment helped them see how the visual could facilitate processes of grief and healing. Another major advantage is the fact that these assessment types are ‘ChatGPT-proof,’ as an AI system would be unable to produce the creative outputs required of the students.

Navigating difficulties as a module convenor

Although both speakers emphasised how rewarding and stimulating these kinds of creative assessments can be, both Akil and Eleanor also highlighted some issues that any colleagues should be aware of when considering an assignment of this type. Both of these modules confront difficult and potentially upsetting topics so sensitivity is required of the module convenor in navigating these issues, especially if students opt to focus on a topic that is related to their personal experience. Both Akil and Eleanor always offered students an ‘escape option,’ in the form of a traditional essay, if they decided they did not want to attempt the creative assignment, although Eleanor said no student had ever requested it.

There is also a significant time commitment involved for the module leader. Each project must be individually approved, often through a number of meetings with students. And finally, but perhaps most importantly, was the issue of how to ensure parity between students. Marking criteria are thus crucial. Akil’s assignments were thus judged on a non-standard set of the criteria, including the following: suitability (use of a medium appropriate to the topic); engaging (the submission is readable/watchable/listenable); and originality (the submission adds something new rather than summarising existing information).

Assignments of this type require us to ask whether it is even possible to measure creativity, or, as Akil said, ‘how can we compare a watercolour and a short story?’ The answer is with careful handling, precise marking criteria and motivated and committed instructors.

The Creative Teaching in the Humanities Network is led by Dr Selena Daly (SELCS). If you have any queries or suggestions for future events and/or speakers, please do get in touch at selena.daly@ucl.ac.uk.

It’s time for the crit. These are words that might send a nervous flutter through the students at the creative arts university Dr. Victoria Odeniyi works at.

A crit is meant to be an open, democratic, and non-hierarchical space where students receive constructive feedback on their artwork. However, for some students, their lived experience of the crit diverged quite a bit from its intended purpose.

Victoria Odeniyi gave a talk titled “Challenges and opportunities of embedding institutional research findings into practice” at Decolonising Language Studies II about her experiences as an applied linguist at University of the Arts London.

Working with the Decolonising Arts Institute, she conducted an ethnographic research project focused on ways of creating more equitable educational outcomes, partly through narrowing persistent funding gaps between home and international students as well as between students of colour and students who identify as white.

The goal was to challenge colonial and imperial legacies and to drive cultural, social, and institutional change by encouraging the institution to critically reflect on their current practices.

This post is the third part of a series where we summarise speakers’ main ideas about decolonising language studies from a series of symposia organised organised Dr Jelena Ćalić and Dr Eszter Tarsoly on behalf of PROLang.

Expectations versus reality: the crit as “un-safe” space

The crit is intended to be an open space for students to display their artwork – such as sculptures and other installations – to their peers and tutors.

Victoria shares her experience about a crit she attended at a location called Safehouse in Peckham, where, over two days, students showcased their work to visiting arts scholars.

She observed tutors commenting on a student’s work in front of about thirty of their peers:

 

 

As Victoria says:

“We can see that there’s a disconnect between these spaces of open, democratic, and supportive peer review, where the tutor […] holds back, and what actually happens during interactions. So this was one example of how I felt […] why some students may have found this space particularly challenging.”

Turning her attention to the space, Victoria also realised that there was no plumbing or places for people to sit. She remarks that, ironically, “the safe house… felt unsafe to me.” Students undergoing the crit “needed a certain amount of stamina” to spend two days in this space.

Assessing multilingual repertoires in students’ art practices

Victoria then shares a story about a design student named Angela, who is both Cantonese- and English- speaking and uses both languages in her work.

Angela experienced an element of frustration in needing to constantly explain her work to her tutor, whom she felt was resistant towards her artistic choices.

As Angela said:

“I had to keep explaining… there’s a lot of hand-holding in terms of explaining it to tutors so they will understand where I am coming from.”

Victoria remarks that multilingual students often have to bear the responsibility of explaining their choices even though they are otherwise encouraged to draw on multiple semiotic resources – texts, colours, fonts, and layouts – in their work.

This also raises issues around how students using multilingual practices for their art can be assessed, especially if tutors are not language specialists or do not have an interest in language:

“If the teacher doesn’t understand Cantonese, how can this work be assessed according to perhaps relatively abstract assessment criteria – which is used to assess other forms of communicative practices?”

Facing institutional resistance to recommendations presented in Victoria’s report

As part of her research report, Victoria suggested initiatives to support awareness raising activities around language, multilingualism, and named languages within the academy.

She also suggested that recognising students’ language backgrounds and repertoires would be a way of cutting across institutional categories and labels. As she says in the talk: “We no longer [have to] speak about international students and home students; we can talk about the repertoire of semiotic resources for meaning-making.”

These recommendations did not seem controversial to her, but when she completed and published the report, she was asked to pause dissemination.

Victoria acknowledges that part of decolonising the curriculum and university concerns navigating institutional spaces and practices like those described above. Noting that universities are often sites of struggle and inequity, she observed institutional resistance to suggested changes.

Some of the responses she received included:

• Competing priorities other than a focus on multilingual repertoires, such as sustainable fashion and climate change, were listed: “Not now, staff may become confused”
• The predominance of language ideologies and Anglonormativity: “if they choose to come here, they need to do it our way”; “it has to be in English”
• Questions of whether there would be a “safeguarding issue” if students are using a number of different languages
• Concerns around assessment: “How can we assess multilingual practices?”

Victoria also notes that her position as a linguist (instead of an artist) rendered her an outsider.

She was often asked, “but Victoria, what is your practice? […] [which] implies: ‘you’re not one of us, so please explain what you’re doing here.’”

Victoria’s talk reflects on both the challenges and possibilities of turning the decolonial gaze back on to the university; on students’ frustrations and experience; and on how difficult it might be to actually enact institutional change.

You can watch her full talk here.

You can also read earlier posts in this blog series:

• Let’s re-centre multilingual communities in our classrooms and research: a summary of Prof. Li Wei’s talk
• How students learn German through charity shop raids: a summary of Prof. Alison Phipps’s talk

This symposium was organised by Dr Jelena Ćalić and Dr Eszter Tarsoly on behalf of the PROLang (Policy, Research and Outreach for Language-based area studies) Research Group in collaboration with UCL Institute of Advanced Studies, with the support of CHE’s Education Enhancement Grant. This post was written by Kellynn Wee.

Interview with Joana Jacob Ramalho, Lecturer (Teaching), SELCS, UCL.

CHE: Hi Joana, a BIG congratulations on being awarded a Student Choice Award for Exceptional Feedback at the annual UCL Education Awards. Feedback is an area of education that is receiving increasing attention from our students – they expect feedback to be detailed and timely, and they rightly expect feedback will help them improve their academic performance. So, naturally, we would love to learn more about how you do it. But first, tell us about what you teach and how you would describe yourself as an educator.

Joana: I teach Gothic Literature, Spanish Film, Musical Satire, Intermedial Comparison, Lusophone culture and Portuguese language. As an educator, my goal is to guide students to reach their potential, which sounds cliché (I know!), but in fact requires training, experience, patience and, above all, creativity. It means constantly tailoring your modules and materials to your students’ different learning styles, combining inclusive techniques that cater for diversity. I want my students to be curious and remain curious throughout their studies (if not their lives!). I feel it is my duty to empower them to ask questions and be comfortable when addressing their concerns. I teach them about culture, politics, history, the arts, and work with topics that are relevant to them – even when they do not immediately realise why. I help them gain transferable skills they might need for further study and future employment, but they also help me make me a better educator. The students are not empty vessels, waiting to receive knowledge; learning is a dialogue, a conversation.

CHE: What methods or strategies do you use for providing feedback?

Joana: I use a combination of numerical, qualitative (written and oral), and peer feedback to teach mostly small to medium groups (~22 students). Whenever possible, feedback should strive to offer students the possibility to develop their ideas or reorient them, suggesting either complementary or alternative avenues. To accomplish this, some form of qualitative feedback should always accompany a numerical mark, whether that means written feedback or a brief chat where the student can ask questions. On the advice of a colleague from ARENA, I have recently introduced peer feedback into my teaching and the students have welcomed it enthusiastically. In language modules, the students experience, in pairs, what it is like to mark and grade a composition or translation. In content modules, there is a peer-to-peer discussion (with minor input from the tutor) in the seminar half of each lecture.

My department encourages formative feedback and I find it essential to guide students in their learning, while giving us a chance to check in with them and adapt our pedagogical strategies (PhD supervision, for instance, is all about formative feedback). A mix of in-class and at-home tasks has worked best for my students. Moodle offers a wide range of activities, from fora to quizzes and interactive videos, that have become familiar tools in my modules. The type of formative tasks varies, but overall these consist in exam-type assignments for language modules and essay plans, sequence analyses, close readings and annotated tables of contents for film and literature modules.

As for summative assessment, I tend to overdo it on the feedback front… I want students to benefit from the same high-quality guidance I enjoyed when I was a student at UCL and I write… a lot, often managing to mark only one essay a day. This is of course not ideal or ultimately sustainable if I want to still have time to do research! In the last couple of years, I have therefore developed templates for each of my modules that allow me to continue offering comprehensive feedback without spending so much time on marking.

Another way to implement change is to develop a staff-student partnership. I led a ChangeMakers project on feedback and assessment in 2020-21, which resulted in a new set of marking criteria designed with a group of first and second-year undergraduates. The project team emphasised how this initiative made them feel like they were actively contributing to the restructuring of the curriculum.

Whichever strategies or methods I use, timeliness is a core aspect of giving feedback. I want my students to be able to read through the comments and have time to act on them. In the first week of term, I explain when and what type of feedback the students can expect. As an example, my film and comparative literature students know they will have the opportunity to submit an essay plan. I set the submission deadline towards the end of term, to give students the chance to write about any of the texts mentioned in the essay questions. An earlier deadline would mean excluding some of those texts or having the students prepare a plan on a topic or text we have not yet explored, which would be counterproductive. Importantly, I make it a point of always handing back the plans before the end of term, so that students can come to me with any questions. This means marking dozens of plans for different modules in the space of a week, but it is one of the aspects the students feel most grateful about. Returning feedback in a timely fashion is key.

CHE: Why do you think students respond so well to your way of providing feedback?

Joana: The students tell me they understand what they have done well and how to improve. They stress the fact that I use in-text comments along with a detailed overall commentary especially helpful. I cover a little bit of everything in the in-text comments, from formatting issues and written expression to reasoning, validity of arguments and structure. I created a series of labels on turnitin for this purpose, which I can reuse and add further comments to. Positive feedback is important as well, so I have lots of labels ranging from ‘good’ to ‘great’ and ‘praise’.

In addition, I provide examples of how to address the issues I flag. For instance, instead of simply pointing out that students should ‘expand’ or ‘engage with the quotation’, I offer a precise suggestion on how to do that. My goal is not only to help students get a higher grade, but help them to think. When I advise them to add more nuance or avoid rushing from one argument to the next without properly supporting their point, my hope is that this exercise encourages them to reflect and use their critical judgement as they engage with the world around them, questioning that which might appear a given, and refrain from jumping to conclusions without checking the facts.

CHE: Where and how did you learn to provide effective feedback?

Joana: With my parents and at UCL. My parents are both teachers and much loved by their students. They unfailingly go above and beyond their duties, staying longer after class and using different approaches to feedback that cater to a diverse range of students. I’ve got a lot of tips from them over the years.

During my Master’s in Film Studies at UCL, the feedback I received was extraordinary. By that I mean, it was detailed and build me up. I remember receiving my first assignment (a formative 500-word sequence analysis) and all I could see was red. Almost every sentence was underlined and accompanied by a single word scribbled on the margins: expand, detail, rephrase, restructure, good, source?, etc. It was a turning point for me as a student and (little did I know at the time) as an educator. The initial shock quickly subsided, as I realised I now knew exactly how to improve. I still keep that piece of paper!

Another aspect my students comment on is my availability to chat with them and provide additional feedback in a more informal setting (outside the classroom). That is also something I learned as a UCL student. My lecturers, the Film Studies programme director, the Head of the Spanish & Latin American Studies department and, in particular, my Master’s and PhD supervisors always seemed to have time for me. Their generosity was central to shape my pedagogy.

Giving good feedback has been a learning curve. Trying to figure out what works for which students on which platforms is a process of trial and error. In my 15 years working at UCL, the sustained sharing of teaching practices within SELCS-CMII has been crucial: the impromptu brainstorming sessions in the corridors of Foster Court, feedback workshops, second marking, doctoral co-supervision, and programme and Language Coordination meetings have introduced me to innovative methods and creative strategies to produce effective feedback.

CHE: Has your idea of what effective feedback is changed over the course of your career?

Joana: The idea itself has not changed, but the methods and strategies have certainly evolved! My feedback has become more comprehensive and more targeted. In my year-long language modules, I can tailor my comments to each individual student’s needs, which is a privilege of small-group teaching. As for content modules, I have learned to focus on specific areas, depending on whether I am marking undergraduate or postgraduate work.

CHE: What are your top 3 tips for effective feedback?

Joana: Detail – Relevance – Timeliness

Thank you so much for sharing your experiences and expertise with us, Joana.