X Close

IOE Blog


Expert opinion from IOE, UCL's Faculty of Education and Society


Grammar schools: can Artificial Intelligence create a fairer way to assess children?

By Blog Editor, IOE Digital, on 16 September 2016

Rose Luckin.
Theresa May’s plans for new or expanded grammar schools in England have brought a torrent of comment, debate, criticism and rhetoric since these plans were inadvertently revealed last week. Most of the discussions seem to have focused on whether or not grammar schools are the right mechanism to aid social mobility. This is an extremely important issue, but let’s put the rights and wrongs of selection and grammar schools to one side for a moment and look at the eleven-plus examination itself.
The eleven-plus is the key to the door of one of the 164 grammar schools in England, or one of the 69 grammar schools in Northern Ireland. The exam is sat by children in their last year of primary school and it varies depending upon where in the country it is taken. In fact, the situation is very complicated, with a wide range of approaches even within the same county. For example, in Yorkshire there are three Local Authorities with grammar schools: Calderdale has 2, Kirklees has 1 and North Yorkshire has 3. The 2 grammar schools in Calderdale use verbal reasoning tests as well maths and English tests examined by GL Assessment, University of Edinburgh and the school themselves. However, the 1 school in Kirklees uses tests in verbal reasoning and non-verbal reasoning, plus an English examination and a numerical reasoning test. These are all examined by the University of Durham. In North Yorkshire the approach is different yet again, with 2 schools using verbal reasoning and non-verbal reasoning tests examined by NFER and the 1 remaining school administering and examining its own selection tests.
The complexity in the selection process is not helpful to poorer parents, who do not have the time, and possibly not always the middle-class-style nous, to navigate the process. In addition, the examination approach is traditional and outdated. The need to look deeper than the selection process to the eleven-plus examination itself was highlighted in an interesting discussion on the Radio 4 Today programme last week. The discussion was between Laura McInerney, the editor of Schools Week, and Sean Worth, from Policy Exchange. Sean pointed out that the current mechanism for selecting children for grammar schools can be gamed and that we therefore need to change the examination if we are to ensure that the poorest children are not disadvantaged. Laura said the major problem for poorer children accessing grammar schools is that we “put a test in the way”, especially divisive when the parents of poorer children can’t pay for tutoring to get their offspring through the eleven plus examination.
The Guardian published a depressing article on the problems inherent in the eleven-plus test: “‘Tutor-proof’ 11-plus professor admits grammar school test doesn’t work”. The article reports the failure of a ‘coaching resistant’ test developed by CEM at the University of Durham for use in Buckinghamshire. CEM has now withdrawn the claim that the test could assess “natural” ability. Prof Coe, director of CEM is reported as saying: “Whatever system you use it is imprecise, there are false positives and negatives and probably more of those than people realise.” He goes on to reflect that whilst he would not support the expansion of grammar schools on the grounds of social mobility, if we are to have more then we need to try and make the system fairer. I couldn’t agree more – and the need for a radical rethink is echoed in what the IOE’s Tina Isaacs says about the problems of coming up with any test that can assess future potential.
So, let’s take the test away and develop a radically different, more socially equal eleven-plus. We are lucky enough to be in a very different position today from that which existed when the original test was introduced in 1944. There is now a realistic and economically attractive alternative at our fingertips. We have the Artificial Intelligence (AI) technology to build a superior assessment system should the proposed reforms become a reality. AI provides a powerful tool to open up the ‘black box of learning,’ to provide a deep, fine-grained understanding of when and how learning actually happens. Intelligent algorithms can process information about each learner and reach a view about their progress, knowledge and understanding of a subject or skill over a ‘period of time’. Unlike the eleven plus examination, this ‘period of time’ could be a whole school semester, a year, several years and beyond.
We know that well designed AI systems that assess learning, as well as providing tutoring, are accurate in their assessment.[1] AI assessment can tackle more than subject specific knowledge and reasoning, it can also evaluate skills such as planning and knowing what we know. AI assessment would also provide a fairer assessment system that would evaluate students across a longer period of time and from an evidence-based, value added perspective. We also know how to prevent a large amount of the tactics that people use to game AI software, and in addition AI Assessment systems would also offer support and formative feedback to help students learn and improve.
Of course there are serious ethical questions around AI being used in education and these must be explored. But the over-riding and uncontested fact in this debate is that education is the key to changing people’s lives. We trust AI with our personal, medical and financial data without a thought, so let’s trust it with the assessment of our children’s knowledge and understanding. Let’s open our minds and explore the challenges to build a new generation of assessment that genuinely irons out the inequalities and gives all children a chance to shine.
[1] Hill, P. & Barber, M. (2014). Preparing for a renaissance in assessment. London: Pearson., DiCerbo, K. E. & Behrens, J. T. (2014). Impacts of the digital ocean on education. London: Pearson.

6 Responses to “Grammar schools: can Artificial Intelligence create a fairer way to assess children?”

  • 1
    gideonsappor wrote on 16 September 2016:

    I will support grammar schools policy if children’s normal school performance is used as basis for selection rather than tests that favour those with the ability to buy tutoring.

  • 2
    Robin Ravi (@robinravi) wrote on 16 September 2016:

    yes, great point

  • 3
    brendanhoare wrote on 16 September 2016:

    Does it really matter whether we can devise a more impartial selection system? Would it not continue to be a method for rationing access to a good education? It’s rather like saying we now have the technology to build a much more humane electric chair.

  • 4
    rose wrote on 16 September 2016:

    Many thanks for your comment. I agree that we must have good schools for all and will always fight for that. However, if we are to have more Grammar schools then the least we can do is to make the assessment used to decide on schools for children fairer and more accurate. Rose

  • 5
    L Patel wrote on 16 September 2016:

    “We know that well designed AI systems that assess learning, as well as providing tutoring, are accurate in their assessment.[1] ”
    Do we ? Sounds interesting. Tried to follow the reference but it loops on this same article.
    I would be interested to find out more about such AI systems. Could the author of this article share some more information about what AI systems are and how they are used for assessment of learning.
    If one knows how to design such AI systems that assess learning, could this be used by “Tutor-proof’ 11-plus professor” who is researching something scientific to create a fairer way to assess children ?

  • 6
    rose wrote on 16 September 2016:

    Many thanks for your comment. Apologies that the reference is not working properly. Here are some references that you might find useful:
    Dodds, P., Fletcher, J.D.: Opportunities for new ‘smart’ learning environments enabled by next-generation web capabilities. Journal of Educational Multimedia and Hypermedia 13(4), 391–404 (2004)
    Kulik, C.-L., Kulik, J.A.: E↵ectiveness of computer-based instruction: an updated analysis. Computers in human behavior 7(1), 75–94 (1991)
    Durlach, P.J., Ray, J.M.: Designing adaptive instructional environments: Insights from empirical evidence. Army Research Institute Report (2011)
    Ritter, S., Anderson, J.R., Koedinger, K.R., Corbett, A.: Cognitive tutor: applied research in mathematics education. Psychonomic Bulletin and Review 14(2), 249–254 (2007)
    VanLehn, K.: The relative e↵ectiveness of human tutoring, intelligent tutoring systems, and other tutoring systems. Educational Psychology 46(4), 197–221 (2011)