Close
Should we eat more fish or more ice-cream to boost PISA scores?
By Blog Editor, IOE Digital, on 12 November 2019
12 November 2019
By John Jerrim
If anyone has ever read one of the international PISA reports or seen Andreas Schleicher present they will know that the OECD is rather fond of cross-national scatterplots. These illustrate the relationship between two variables measured at the country level.
Take, for instance, the chart below. This has been taken from one of Mr Schleicher’s blogposts, and illustrates the relationship between a country’s test scores and its rate of economic growth. It has been interpreted by the OECD as showing “that the quality of schooling in a country is a powerful predictor of the wealth that countries will produce in the long run”.
Source: Research by Hanushek and Woessmann, via the OECD Education and Skills Today blog
Sounds convincing, right?
The trouble is, correlation does not equal causation. And, despite the OECD’s obsession with such cross-country relationships, they can often deceive.
Should we eat more fish?
Of course, it is not only the OECD that produces such analyses. Academics do as well.
One of the more unusual examples I have found is this study by a group of epidemiologists. They look (I kid you not) at the relationship between average fish consumption and average PISA test scores.
Somehow, the study manages to find a strong positive correlation of around 0.6! It then goes on to conclude that a “linear association between fish consumption and PISA scores is likely” and that “it is plausible to assume that a higher level of fish consumption drives improvement at school level”.
As a nation of fish-and-chip lovers, this should be good news for our PISA scores…
But isn’t eating ice-cream just as important?
Getting kids to eat more fish is difficult. My little boy hates the stuff! He does, however, like ice-cream – and lots of it (who doesn’t?). If only eating ice-cream were as good for PISA scores as eating fish.
Well, apparently it is.
The chart below provides one of my all-time favourite PISA graphs. It plots the level of ice-cream consumption of a country against average PISA reading scores. There is actually quite a strong positive relationship, with around half the variation in PISA scores explained.
It also seems that the relationship is exponential (i.e. the line of best fit is curved upwards). Not only should we eat more ice-cream, but we should be eating lots of it.
The take-away message (no pun intended)
Hopefully, the point of this blogpost has become clear.
When the PISA results get released at the start of December, the international report and presentations given by the OECD are bound to include this kind of graph, along with stories about how ‘high-performing countries’ all do X, Y or Z.
Clearly, we should be treating any such interpretation of the PISA results with caution. There are likely to be hundreds, if not thousands, of reasons why some countries do well on PISA and others don’t. In reality, it is almost impossible to separate these competing reasons out.
What we do know is that overly simplistic “explanations” for the PISA results must be avoided. Organisations like the OECD have their own agenda, and it is just too easy for such groups to use the results to promote their own hobby-horses.
Other posts in this series can be found here, here, here, here, here, here, here and here.
Photos: fish by tkcrash123, ice cream by J Lippold via Creative Commons
8 Responses to “Should we eat more fish or more ice-cream to boost PISA scores?”
- 1
-
2
rogertitcombe wrote on 12 November 2019:
“If anyone has ever read one of the international PISA reports or seen Andreas Schleicher present they will know that the OECD is rather fond of cross-national scatterplots. These illustrate the relationship between two variables measured at the country level.”
This is true, but the really important link with national IQ is ignored. I did the statistics myself for PISA maths scores and you can find it here.
There is a correlation of 0.89, which is very similar to that which results when the GCSE scores of schools within an LA area are plotted against mean intake Cognitive Ability Test (CATs) scores. Whereas causation looks unlikely in the case of fish & ships and ice-cream this is not the case in respect of IQ and it is indeed clear from national CATs data going back decades that CATs scores are the strongest predictor of GCSE results. Indeed when CATs scores are taken into account social and economic disadvantage effects largely disappear.
My study has been updated with better national IQ data and can be found here.
https://rogertitcombelearningmatters.wordpress.com/2016/12/18/national-iqs-and-pisa-update/
Surely there should be no great surprise that OECD’s plot of national economic growth against national IQs follows the same pattern as PISA scores.
Virtually every article on my website hammers away at these links, the obvious point being that this being the case schools should be focused primarily on exploiting the proven plasticity of cognitive ability/IQ to prioritise its development over jumping the arbitrary GCSE school attainment thresholds need to drive school league tables and the marketisation ideology produced perverse incentives that are actually making our students dimmer rather than brighter.
-
3
africanschoolclapham wrote on 16 November 2019:
Great article thanks!
I wrote a paper on PISA while at the IoE. I visited the offices of the OECD in Paris and had to plead and go through 3 levels of security just to go to the bookshop pick up the 20 pound PISA report.
Also what Schleicher et al put forward from the report is a mini-sample from the research. If you read more deeply you will see a large part of the findings contradict what the OECD is promoting. So for ex the OECD promotes the publication of student results in league tables but its own research shows that countries that don’t do this have the highest scores (and generally best regarded educational systems). Hmmmm -
4
Fish, Ice Cream, & Lightbulbs – Diving into the Black Box of Research | Beth R. Holland wrote on 18 November 2019:
[…] form of relationship. However, as Edward Tufte has famously illustrated, a a recent article from John Jerrim at University College London beautifully illustrates, correlations can be somewhat absurd. As a warning about the upcoming release of PISA scores, […]
-
5
Is Canada really an education ‘superpower’? The evidence is not as clear-cut as you might think | IOE LONDON BLOG wrote on 19 November 2019:
[…] posts in this series can be found here, […]
-
6
The Buildup to PISA Results with Highlights from the IOE Blog | International Education News wrote on 21 November 2019:
[…] For the full blog post, see the IOE blog. […]
-
7
Should England continue participating in PISA? | IOE LONDON BLOG wrote on 26 November 2019:
[…] Other posts in this series can be found here, here, here, here, here, here, here, here, here and here. […]
-
8
Five things to remember when the PISA 2018 results are released | IOE LONDON BLOG wrote on 2 December 2019:
[…] Other posts in this series can be found here, here, here, here, here, here, here, here, here, here and here. […]
John, super post, as are all in this series!
I am just intrigued to know that despite us eating so much fish, whether or not there is a negative correlation with chips….. so much research still to do (sigh!).