Exams: changing the rules of the game while you are playing will not rebuild trust

Pete/CC BY 2.0

26 February 2021

By Melanie Ehren

In December last year, Ofqual announced a new expert group to rebuild trust in the exam system. The group is to look into how data on schools’ and students’ performance could be “better and more widely shared”, thereby prising open the box of secrets containing the data and processes that drive the awarding of exam grades.

The group’s appointment could not come at a better time; Education Secretary Gavin Williamson has since announced that teachers’ estimated grades will replace cancelled GCSEs and A-levels in England this summer, saying that he would “trust in teachers rather than algorithms”, a reference to last year’s exams U-turn. Today, Government announced new plans for teacher assessed GCSEs, AS and A levels which will include a series of ‘mini-exams’.

But is this alternative approach the best way forward to rebuild trust in exams? Or do we need a wider set of strategies?

To answer the question, we first look at whose trust needs to be rebuilt.

Exams have an important role to play in our education system: they certify or assess student competence based on a set of agreed standards and award successful candidates with diplomas or certificates. Exams provide a basis for considering candidates for higher levels of education and for employment. The outcomes of exams are also used as performance indicators to hold schools and teachers accountable.

Much can be at stake. If a student fails an exam, or when exams fail to provide accurate information, students may miss out on the college or university of their choice, while universities may select students who not up to the demands of a particular programme. Meanwhile, employers might hire new recruits without the skills level needed for the job.

Trust violation

Students (and their parents), teachers, school leaders, universities and employers all rely on exams to provide accurate information and a fair assessment of students’ competences. It is their trust that was broken last year.

There was much coverage of the statistical model used last year to standardise grades based on teacher judgements, which aimed to predict what students would have been most likely to achieve if the courses and assessments had proceeded as normal. The standardisation was intended to issue results that were as fair as possible across different schools and colleges and to offer ‘robust, good and dependable information for employers’. The problem was that the standardisation resulted in a downgrading of results for many students, particularly those from disadvantaged backgrounds. As a result, the process was abandoned, and students received their original teacher/school-based grades.

This situation highlights the complexity of the problem. Students, teachers, schools, universities and employers all have a different stake in the exams, which are expected to combine many different functions at the same time. Therefore a variety of professions are involved in their design and implementation, awarding of grades and communication of results: teachers, schools, exam boards, Ofqual, the Department for Education and government more widely. And let’s not forget (social) media where messages are filtered and take on a life of their own: a statistical model to standardise grades become a ‘mutant algorithm’.

A strategy to repair trust ideally addresses the ways in which trust was broken and by whom. This set of strategies could form a starting point for further deliberation.

Repairing trust

Studies on trust repair tend to distinguish between whether the breach is associated with integrity or competence. In last year’s exam debacle, integrity was clearly breached when the standardisation disproportionately upgraded the results of private school students, while downgrading those from state schools, especially in the poorest areas. As this was caused by a mistake in the algorithm, there is also a violation of trust in the regulator’s competence. To repair these would need a combination of strategies and particularly:

• an explanation of what went wrong in clear and transparent language and an honest apology to students, teachers and schools who were affected
• a solution which shows that those involved have learned from the mistake. This would include a proposal for this year’s exams which ensures that university places are allocated on the basis of universal merit – but more importantly, treats the efforts, abilities, hopes and expectations of millions of young people with respect.

Unfortunately there is no quick fix solution to repair trust. A good start however is to show a commitment to really understanding what went wrong, to prevent similar chaos in planning for this year’s exams, and listening to, and addressing, concerns of all those involved.

The current plans for teacher assessments do not indicate that lessons were learned.

This year’s arrangements call for teachers to use evidence about students’ performance gathered throughout the course to inform their judgement. This might include completed work, mock exam results, homework, in-class tests, or questions from exam boards. And then school or college heads will need to confirm that the student has been taught enough content to be able to progress to the next stage of education.

Even though students will be told which pieces of their work will count towards their grade, much of this work will have been completed at home during lock-down without exam grades in mind, and when high stress levels could have affected the work’s quality. Some students will have had little face to face interaction with their teacher(s) or may not have a good relationship with their teacher. Yet, students are asked to trust teachers blindly, even if they believe they would have submitted better if they had known it would count towards a final grade.

Changing the rules of the game while you are playing is not a good recipe for rebuilding trust.

Photo by Pete via Creative Commons