Grammar schools: can Artificial Intelligence create a fairer way to assess children?
By Blog Editor, IOE Digital, on 16 September 2016
Theresa May’s plans for new or expanded grammar schools in England have brought a torrent of comment, debate, criticism and rhetoric since these plans were inadvertently revealed last week. Most of the discussions seem to have focused on whether or not grammar schools are the right mechanism to aid social mobility. This is an extremely important issue, but let’s put the rights and wrongs of selection and grammar schools to one side for a moment and look at the eleven-plus examination itself.
The eleven-plus is the key to the door of one of the 164 grammar schools in England, or one of the 69 grammar schools in Northern Ireland. The exam is sat by children in their last year of primary school and it varies depending upon where in the country it is taken. In fact, the situation is very complicated, with a wide range of approaches even within the same county. For example, in Yorkshire there are three Local Authorities with grammar schools: Calderdale has 2, Kirklees has 1 and North Yorkshire has 3. The 2 grammar schools in Calderdale use verbal reasoning tests as well maths and English tests examined by GL Assessment, University of Edinburgh and the school themselves. However, the 1 school in Kirklees uses tests in verbal reasoning and non-verbal reasoning, plus an English examination and a numerical reasoning test. These are all examined by the University of Durham. In North Yorkshire the approach is different yet again, with 2 schools using verbal reasoning and non-verbal reasoning tests examined by NFER and the 1 remaining school administering and examining its own selection tests.
The complexity in the selection process is not helpful to poorer parents, who do not have the time, and possibly not always the middle-class-style nous, to navigate the process. In addition, the examination approach is traditional and outdated. The need to look deeper than the selection process to the eleven-plus examination itself was highlighted in an interesting discussion on the Radio 4 Today programme last week. The discussion was between Laura McInerney, the editor of Schools Week, and Sean Worth, from Policy Exchange. Sean pointed out that the current mechanism for selecting children for grammar schools can be gamed and that we therefore need to change the examination if we are to ensure that the poorest children are not disadvantaged. Laura said the major problem for poorer children accessing grammar schools is that we “put a test in the way”, especially divisive when the parents of poorer children can’t pay for tutoring to get their offspring through the eleven plus examination.
The Guardian published a depressing article on the problems inherent in the eleven-plus test: “‘Tutor-proof’ 11-plus professor admits grammar school test doesn’t work”. The article reports the failure of a ‘coaching resistant’ test developed by CEM at the University of Durham for use in Buckinghamshire. CEM has now withdrawn the claim that the test could assess “natural” ability. Prof Coe, director of CEM is reported as saying: “Whatever system you use it is imprecise, there are false positives and negatives and probably more of those than people realise.” He goes on to reflect that whilst he would not support the expansion of grammar schools on the grounds of social mobility, if we are to have more then we need to try and make the system fairer. I couldn’t agree more – and the need for a radical rethink is echoed in what the IOE’s Tina Isaacs says about the problems of coming up with any test that can assess future potential.
So, let’s take the test away and develop a radically different, more socially equal eleven-plus. We are lucky enough to be in a very different position today from that which existed when the original test was introduced in 1944. There is now a realistic and economically attractive alternative at our fingertips. We have the Artificial Intelligence (AI) technology to build a superior assessment system should the proposed reforms become a reality. AI provides a powerful tool to open up the ‘black box of learning,’ to provide a deep, fine-grained understanding of when and how learning actually happens. Intelligent algorithms can process information about each learner and reach a view about their progress, knowledge and understanding of a subject or skill over a ‘period of time’. Unlike the eleven plus examination, this ‘period of time’ could be a whole school semester, a year, several years and beyond.
We know that well designed AI systems that assess learning, as well as providing tutoring, are accurate in their assessment. AI assessment can tackle more than subject specific knowledge and reasoning, it can also evaluate skills such as planning and knowing what we know. AI assessment would also provide a fairer assessment system that would evaluate students across a longer period of time and from an evidence-based, value added perspective. We also know how to prevent a large amount of the tactics that people use to game AI software, and in addition AI Assessment systems would also offer support and formative feedback to help students learn and improve.
Of course there are serious ethical questions around AI being used in education and these must be explored. But the over-riding and uncontested fact in this debate is that education is the key to changing people’s lives. We trust AI with our personal, medical and financial data without a thought, so let’s trust it with the assessment of our children’s knowledge and understanding. Let’s open our minds and explore the challenges to build a new generation of assessment that genuinely irons out the inequalities and gives all children a chance to shine.
 Hill, P. & Barber, M. (2014). Preparing for a renaissance in assessment. London: Pearson., DiCerbo, K. E. & Behrens, J. T. (2014). Impacts of the digital ocean on education. London: Pearson.