GCSE results in English and maths: whatever approach is taken, here is how it should be validated

27 March 2020

By John Jerrim

We found out last week that GCSE grades for the 2019/20 cohort will be based upon judgments made by teachers. It has been announced that Ofqual will be working with the sector to provide guidance on how this should be done, with one possible approach suggested by FFT Education Datalab here.

As many people have pointed out, one of the potential problems with teacher-determined grades is that they could be biased for or against certain groups (e.g. children from lower socio-economic status backgrounds receiving worse grades than their more advantaged peers). It is therefore critical that a. such predictions are underpinned by data wherever possible, and b. that the guidance issued by Ofqual (and the approach taken by teachers in making their predictions) has been validated.

This is how I suggest it could be done in English and maths.

Could the National Reference Test be our “get out of jail free” card?

The approach I suggest draws heavily upon the National Reference Test (NRT). For those who don’t know, this is a test of English and maths skills taken by Year 11 pupils in England. It is designed to assess the same skills as GCSE English and maths, and is nationally representative, with around 600 schools and 14,000 pupils randomly selected to take part.

Critically, in 2020, this test was conducted between February 24^th and March 6^th; just before the current situation blew up here in the UK. Schools and pupils are not provided with any information on how they did on this test, which means it cannot be used by teachers in these schools to inform their view on GCSE grades.

This makes it the best available data source to validate the guidance Ofqual will publish – and the approach teachers will take – on how to award GCSE grades.

How exactly would this be done?

Although the finer details would need to be ironed out, here is how I believe this could be done.

• First, Ofqual would work out their provisional guidance to teachers and schools, as I am sure they are already doing.
• Second, let’s randomly split the NRT sample of 600 schools into half. This means we have about 300 schools to initially work closely with – 150 of whose Year 11 students completed the NRT English test and the other 150 of whose Year 11 students completed the NRT maths test.
• Ofqual would then ask these schools to follow their provisional guidance, and provide grades for these students as soon as possible. They would also ask the schools to provide any additional test data they hold about these students, such as their scores on internal assessments (particularly those from commercial providers).
• Ofqual would then hold a database that includes, for these 150 schools, information on Year 11 pupil’s Key Stage 2 scores, NRT scores and any other test data the school holds. This database, combined with data on pupil characteristics, could be used to investigate the psychometric properties of teacher-determined grades, including if there is any evidence of bias for/against certain groups.
• The guidance to be provided to schools/teachers could then be altered if necessary, if problems of bias do arise – or the information could be used later to inform their moderation of grades.
• Ofqual could then repeat this process with the other 300 schools that participated in the NRT, to test out their revised guidance.

The practicalities

Can this be done in practice?

The honest answer is that no-one really knows – we have never been in this situation before! And there are only four months until GCSE results will be released.

For it to work, it will require a lot of goodwill from the schools that participated in the NRT, while also requiring staff at the exams regulator to work non-stop over the coming months (although I am sure that is likely to be the case anyway).

But, to be credible, it is vital that the approach to awarding GCSE grades is tested and to some extent validated first.