‘PISA has shifted from being a measure to a target, and in so doing it has lost its value’

Paul Morris.

A recent IOE Blog asks whether England should continue its involvement with the triennial PISA tests and concludes that we should, as it provides a wealth of unexplored data for analysis.

The question is timely as the outcomes of the 2018 PISA exercise have just been released. They show once again that England’s scores are fairly stable and around the average – although the they do show improved scores in Reading and Maths and a decline in Science and Life Satisfaction.

The important question in deciding whether to continue with PISA is: what have been the major benefits over the last 19 years?

A number of nations (e.g. Germany) have used poor performance on PISA as a catalyst to promote significant long term reform programmes. The PISA results engendered a sense of urgency and allowed policy makers to overcome local resistance. The extent to which those reforms were linked to the PISA outcomes and their longer term benefits are contested, but they did involve an extended process of reflection and deliberation.

However, the response in England – where media and politicians have previously tended to portray the results as somewhere between poor and apocalyptic – has been more expedient and selective. PISA data has been raided to promote serial reforms designed to legitimate policy-makers’ desire for a more academic curriculum and whole class teaching. 

What has been presented to the public as ‘evidence-based policy making’ in the pursuit of ‘world class schooling’ and ‘global best practices’ has been more akin to constructing a façade for promoting ideologically driven initiatives. The result is that education in England has been viewed through the prism of PISA, with its focus on Maths, Science and Language, and this has distorted the purpose and nature of schooling. I elaborate below.

1. School Autonomy: The most intense use of PISA was in 2010 when then Education Secretary Michael Gove described England’s results as ‘stagnant’, claiming that the top-performing East Asian nations were successful largely because they provided head-teachers with a high degree of autonomy. He argued that massive increases in the number of academies and free schools would boost autonomy and performance levels in England. These claims were false. As I noted in 2015 Head-teachers in England had long had higher levels of autonomy than in East Asian nations and earlier analyses had argued that the success of East Asian nations was facilitated by high levels of central control. 

2. Mathematics Teaching: After the 2012 results, the UK Government promoted the adoption of Shanghai’s curriculum in order to emulate their high PISA scores. Significant sums were spent importing teachers from Shanghai to train Maths teachers and establish pilot schools in England. In 2019 the DfE evaluation of the project, released with little media coverage, concluded that it had had no measurable benefit and had not met its objectives. Notwithstanding, media coverage has already attributed the recent improved maths scores to the adoption of the Chinese curriculum.

3. Improving PISA Scores: As politicians have harnessed their careers to improving the nation’s PISA outcomes, so initiatives have been enacted to this purpose. In England domestic systems of accountability closely mirror what is measured in PISA, through structures such as OFSTED and the Ebacc. The knock-on effect is that schools have diverted time and resources to the subjects for which they are held accountable, and others, such as art, music and PE, have been marginalised. In effect, the PISA test has confirmed Goodhart’s Law; PISA has shifted from being a measure of performance to a target and in so doing it has lost its value. As Biesta notes, we have ended up valuing what we measure; not measuring what we value.

4.  Broader Issues: The tendency to view schooling through the prism of PISA has reinforced the acceptance of the fundamental assumptions which underpin that enterprise: that PISA is a valid measure of educational quality across nations; improving PISA scores will generate economic growth; the causes of high performance are located in schools; and these can be identified and readily transferred between nations. Notwithstanding that each of those assumptions have been extensively challenged in the literature, they continue to drive the OECD and educational policy in England.

5. The Future: Has PISA outlasted its usefulness? Andreas Schleicher, Head of the OECD Education Section, now states that PISA is not measuring the important skills required in the future. Success, he argues, will go to those nations that promote 21^st Century skills, especially creativity, global competencies and well-being. The OECD has included a measure, albeit a problematic one, of Global Competencies in the 2018 PISA test and will measure creativity in 2021. That vision does not accord with the UK Government’s focus on improving ‘academic standards’; accordingly it decided that pupils from England will not take those tests.