Seven principles for a fair and relevant assessment system

Ruth Dann.
In my contribution to last week’s IOE Debate asking ‘What if… we designed our school testing and assessment system from scratch?‘ I distilled what I think are 7 key principles that might help us shape our examination and assessment system differently.
Principle 1. That all tests and examinations can only ever be a proxy measurement, sampling what someone knows. All exam results will have measurement error. Exam boards try to minimise such error, by giving careful attention to issues of validity and reliability. However, in England, for GCSE and A levels, we do not know how questions will affect different subgroups of the candidate cohort, as questions are not trialled in advance because they might be leaked.
Of further concern are issues of bias and fairness which extend beyond the test paper and should include consideration of the opportunities that pupils have had to learn, as well as the quality of the teaching they have received. Therefore, there is a strong imperative to interpret examination results from a position of understanding possible fairness and bias. This means that any new system must ensure that other sources of evidence are used alongside test scores, and for test items to be trialled in advance, as they are for KS 2 tests.
Principle 2. Any test or examination needs to have a clear purpose; only then can it be considered in terms of its fitness for purpose. Multiple and competing purposes may result in inferences that are not valid. Currently examination results are used for many purposes: to predict future scores, to measure the quality of schools or the quality of teaching, as well as for pupil certification. If the purpose is accountability, not every student is needed to sit such exams; they can sample the student population, and this may not necessarily need to be done annually. Data need only be considered in aggregate form and not on an individual pupil level. We need to decouple our current examinations for pupil certification from our accountability system.
Principle 3. The curriculum we want pupils to receive has to be understood and planned in relation to what we will examine. We need to reconsider what we value in education so that our examination system does not drive out what we value. Research repeatedly shows us that in high stakes examination cultures there will be teaching to the test. So we need to make sure what we examine is broad and we do not unnecessary narrow pupils’ learning. This will mean we need to develop an assessment system that can examine and record a broader range of knowledge and skills. Records of achievement and portfolios offer ways of capturing a broader range of pupil achievements. However, issues of reliability, in terms of the quality of judgments, would require a new moderation infrastructure.
We also need to recognise that examining pupils in isolation from each other does not tell us much about the way they can work with others. Research already established, assessing collaboration skill, needs to be continued and extended.
Principle 4. We need a broader system of examinations and assessments along different pathways (including vocational), that are valued equally, rather than hierarchically, which help to promote rather than hinder social mobility. Currently we have designed and accepted what Joseph Fishkin describes as an ‘assessment bottleneck’. Only a proportion of candidates will get the necessary grades to take future pathways that open up beyond the bottleneck (others fall back down into the bottle). We know that groups of children can be more or less privileged in our examination system. For example, children with summer birthdays are more likely to have compounded disadvantage from the point at which they enter the school system.
Principle 5. We need a system of examinations which offers challenge, but each candidate’s outcomes should not be dependent on the outcomes of another. Our current GCSE and A level examination system in England is both competitive and positional. To do well means others have to do less well. In our GCSE and A level systems grade boundaries are now largely statistically imposed through the regulators Ofqual’s intention to stop grade inflation. Therefore, we have designed into the system winners and losers.  We need to make sure that our examinations and assessments are criteria referenced and that we have robust ways of assessing pupils’ achievements in relation to criteria.
Principle 6. We need to better consider how the United Nations Convention on the Rights of the Child relates to our examination and assessment system- particularly article 3 about the child’s best interest, and article 12 the child’s right to participate. Now that our children do not leave school or training until the age of 18 we need to ask why we have a high stakes examination system at age 16. At the very least we should be asking: is this in the child’s best interest? and whose best interest is it in? In terms of a child’s right to participate in decisions made about them, we need to review the extent to which pupils have any participation in the way that their examination results are used.
Principle 7. The importance of assessment literacy for all stake holders in the examination system: teachers, pupils, parents and policy makers. We need better ways to ensure teachers are trained more extensively in the principles and practices of assessment. We need better ways of improving pupils’ understanding of how examinations work. We need better ways to convey to other stake holders what examinations can and cannot do, and how to interpret the outcomes of assessments in meaningful ways. This will be a significant challenge which demands that the whole of society considers how we want our assessment system to both construct and define our children.
More information on IOE events and debates can be found here