Too Much, Too Little? Finding the ‘Goldilocks’ Level of Assessment to Advance Personalized Approaches to Education for Everyone

By Blog editor, on 24 October 2023

By Dr Dominic Kelly

This article was first published by UNESCO MGIEP as part of The Blue Dot 17: Reimagining Assessments.

I like to think my teachers would say I was a relatively good child, but I am not sure they would say I was the most consistent one. In school, concentration on my studies was too often distracted by Pokémon cards, Arsenal F.C., and elaborate daydreams. As inconsistent as I was, from my experience, I knew that some of my classmates could be even less consistent – how they behaved yesterday could be radically different to how they behaved today, regardless of how clever they could be at their best. Given the challenges that many children face at home, there were many reasons for these inconsistencies. Did they get a good night’s sleep, despite a noisy, overcrowded house (Hershner, 2020)? Did they even have breakfast that morning (Hoyland et al., 2009)? Therefore, if you had entered our classroom with a clipboard and a page of arithmetic on a random Wednesday afternoon, I am unsure that you would have caught all of us at our best – or even at our most typical. Likewise, whether our typical selves happened to be present on the same day as standardized tests were administered, was certainly not a given. Perhaps this all seems obvious to you but, despite this, why are single assessments of children often assumed to be representative or reliable?

In recent years, there have been understandable worries that we assess children too much (Hopkinson, 2022). Students and parents have reported that schools put too much focus on ‘high stakes’ testing, potentially to the detriment of children’s ‘love of learning’ (More Than a Score, 2020) and to their mental health (Newton, 2021) – although it should be noted that recent empirical research in a British sample found no relation between children’s wellbeing or happiness in school and participating in standardized testing (Jerrim, 2021). Either way, there is a distinct possibility that high- stakes standardized assessments are not the most representative way of assessing children’s educational capabilities (e.g., Morgan, 2016). Furthermore, I would also suggest that the most vulnerable children from the least consistent home settings are often those assessed the least fairly. Increasing evidence suggests that our cognitive performance in any given moment is affected by many contextual factors (e.g., Chaku et al., 2021). If so, given the variability that we know all children but especially the most disadvantaged show, conclusions about academic behaviours which are drawn from single measurements may not be as representative of a student’s capabilities as once thought because these measurements are affected by external factors such as sleep, stress or nutrition. Given this, there should be a real concern that single assessments, whether standardized or not, could be a format that works to the advantage of children from affluent backgrounds, while being particularly unfair to children from disadvantaged ones. For this reason, I would argue that education experts and developmental psychologists typically assess children too little. Instead of having occasional high-stakes assignments which potentially disrupt learning and increase tension in the classroom, I argue that there is a need for more frequent, low-stress assessments that occur in the background of the learning environment without disrupting instruction, which are not only more representative of achievement but also allow us to really engage with what makes a child’s classroom experience so variable from day to day.

Technological advances in educational technology (EdTech) offer us the potential to fundamentally change how interventions are developed for students, which can represent their variability in a manner which is much closer to “real time”, especially in high-income countries where many classrooms might have these technologies already available. Largely because of the substantial amount of labour and expenditure required to administer assessments, longitudinal educational studies have traditionally had long measurement intervals – for example, years or months apart. But what might appear to be relative stability in educational behaviours when assessed infrequently may in fact be a highly dynamic process with substantial fluctuations between days. Modern technology in the classroom setting provides the opportunity to dramatically reduce costs and both increase the number of assessments and decrease the intervals between assessments – for example, intervals of days, hours or even minutes. This latter approach to assessment can be considered prototypical of intensive longitudinal designs, which involve the collection of many repeated observations per person (also known as micro-longitudinal designs). Data for these studies are often collected by measuring individuals’ thoughts and behaviours, typically in familiar environments (e.g., the classroom, at home), instead of unfamiliar laboratory environments, with relatively non-intrusive smartphones, tablets, wearable technology, and so on. These assessments go beyond traditional continuous assessments as contextual, non-cognitive factors can be collected too. These studies may also be more accurate due to the decreased intervals between when thoughts and behaviours occurred and when they are reported (Trull & Ebner-Priemer, 2014).

One of the most important benefits of collecting intensive longitudinal data is the potential to adapt instruction to the needs and variability of each child. Instead of generalising broad conclusions across students, we have the potential to utilize previously unfathomable amounts of data collected from EdTech to create highly personalized models for every child. To date, intensive longitudinal studies have disproportionately featured adults (e.g., Kelly & Beltz, 2021) and have rarely been set in the classroom. Yet, compared to data collected much less frequently, intensively collected data on the variability of student’s’ experience can be sought regularly in the classroom – learning behaviours and outcomes, wellbeing, peer interactions, and so on. Personalized education is a burgeoning field focused on leveraging ‘big data’ to develop complex but parsimonious models based on students’ needs and nuances, which can lead to effective interventions, but there is still relatively little known about what factors in children’s daily lives are important for their academic achievement and wellbeing. Intensive longitudinal studies can inform this and facilitate potentially powerful personalized interventions. This personalization is particularly important given the diversity we see in the classroom. Many intervention efforts for equalizing educational outcomes have been designed for the ‘average student’. Yet, no student is average: students’ learning processes are contextualized by the intersections and interactions of each element of their identity, background, and history, which may not be consistent in how they manifest in the classroom every day. Rather than apply broad educational practices across students, leveraging intensive longitudinal data offers enormous potential for developing highly personalized models and interventions tailored to each student’s unique needs.

Given that personalized approaches to education require a greater number of assessments than other approaches, there is some concern that administering regular assessments could be burdensome for teachers and potentially disrupt learning. The innovative applications of EdTech, so that assessment goes relatively unnoticed while providing the most benefit, are therefore essential. Many classrooms in high-income countries already have some relevant technological infrastructure in place, even if it isn’t intended for that purpose yet. Daily educational data is already being collected: namely, formative assessments which are used at the moment by educational professionals to monitor progress. Although continuous forms of assessment can potentially be useful for reducing the pressure on students on specific occasions, their potential is being underutilized: these data also allow for a more fine-grained understanding of what predicts and what is predicted by students’ daily variability. The thoughtful measurement and modelling of this data could be elucidating, but there is still a lack of suitable methods, leaving the field “data rich but information poor”. If this data could be complemented by other short-form, easy-to-administer surveys about behaviour or cognition, it would be possible to address questions about children’s individual progress and setbacks in the classroom, without placing extra stress on teachers. To ensure this, thoughtful teacher training will need to be developed and provided, which itself will likely need to be tailored to teachers’ existing knowledge of EdTech. An important challenge will be determining the right number of assessments that provide enough fine-grained detail to understand the complexity of a child, but that is not so demanding that it impedes the classroom – in fact, that ’Goldilocks’ number of assessments may itself be unique to each child. Of course, there are continued inequities in access to these opportunities as it is mostly high-income economies that have embedded technology in their classrooms, and there are also notable differences in opportunities within those economies. As EdTech decreases in cost and hopefully spreads to more diverse settings, an important challenge will be designing and administering assessments which are culturally specific to local educational needs and resources.

Another potential limitation of intensive longitudinal designs is that they track fluctuations over short periods of time, but do not alone allow for plotting long-term changes. Therefore, there is a clear need for studies and interventions that combine both traditional and intensive longitudinal assessments together – what are called ‘burst designs’ (Stawski et al., 2015). For example, one could measure children’s academic performance and experience in the classroom every day for two weeks, every year for five years. Such a design would have the potential to address unique questions about how short-term fluctuations become long-term change. Are there specific times in a child’s life where they are the least consistent in their behaviours, and does that matter? Is a child’s lack of consistency in daily assessments indicative of problem behaviours in later life? Only by integrating intensive longitudinal data and traditional longitudinal data can these questions be addressed.

In sum, we have good reason to question whether single assessments can truly represent the variability of a child’s experiences in the classroom. Contextual factors can lead to substantial fluctuations in cognitive performance. Intensive longitudinal studies to measure these fluctuations have previously been used primarily with adults, but this work has generally not yet translated to the classroom or with children, despite the potential that the thoughtful leverage of this type of assessment offers for our understanding of variability and the future of personalized education. Furthermore, there is a distinct need for research that suitably assesses both short-term fluctuations and long-term change together, to determine how the former becomes the latter in ways that are potentially unique to each child. I believe this to be a worthy endeavour – individualized approaches to education which fully engage with the heterogeneity of the unique disparities that students face, have the potential to reduce barriers, equalize outcomes, and improve social mobility. The inconsistency of a child’s cognition or behaviours should not be treated as error, noise or inconvenience but as a vital, and long overlooked, aspect of their development.

I’d like to thank my doctoral dissertation committee – Drs. Adriene Beltz, Pam Davis-Kean, Robin Edelstein, and Ioulia Kovelman – for their insight in developing this line of research with me.

The Class of 2023: (some of) the kids are alright

By Blog editor, on 17 August 2023

By Gill Wyness, Lindsey Macmillan, and Jake Anders

Today marks the first ‘true’ A level and Vocational and Technical Qualifications (VTQs) exam results since the more innocent days of 2019. Although pupils sat exams in 2022, their results were adjusted by Ofqual’s “glide path” which aimed to move results gradually back to pre-pandemic exam grading following the grade inflation of 2020 and 2021. So, a comparison of todays’ results versus 2019 should provide information about the extent of learning loss experienced during the pandemic. And the results present a bleak picture for inequality in England.

Back to the future?

Overall, the proportion of pupils awarded a C or above at A level is more or less back to 2019 levels. This is perhaps grounds for optimism; these students have had a very different learning experience compared to their peers in 2019. They experienced a global pandemic, and severe disruption to their schooling in critical years – as Figure 1 shows. In addition, this cohort did not sit GCSE exams, so lack that crucial experience of performing under pressure that previous cohorts have had.

Figure 1: Timeline of the Class of 2023

However, there is much less cause for optimism when we look at inequalities in the results. At the moment, these are only available across school types, and regions.

Mind the gap

Looking first at inequalities by school type, the gap between academies and independent schools has widened since 2019. The proportion awarded a C or above is down slightly in academies (75.4% in 2023 versus 75.7% in 2019) whilst it is up in independent schools (89% versus 88%). This represents a 1.3 percentage point increase in the state-independent gap, which now stands at 13.6 percentage points. For the top grades (A and above), the independent-state gap has widened slightly more, to 1.4 percentage points.

We already know that students from these different school types had very different schooling and learning experiences during the pandemic. Figure 2, from CEPEO’s COSMO Study, highlights the disparity in learning provision experienced by students from different school types during the pandemic. The disparities in A level performance that we see today are yet more confirmation that these students did not experience the disruption of the pandemic equally.

Figure 2: Provision of live online lessons, by school characteristics, lockdown 1 and 3

There are also notable inequalities across region. In particular, (and as was the case last year), London and the South East continue to pull away from all other regions with the largest increase in the proportion achieving A/A* since 2019, and some of the smallest declines relative to last year’s cohort. Meanwhile, the North East and Yorkshire and Humber are the only regions with a lower proportion achieving A/A* compared to 2019.

Competitor advantage

As discussed, it is harder to make comparisons between today’s results and last years, because 2022 results were adjusted by Ofqual’s glide path. However, for pupils receiving their results today, the 2022 cohort are their closest competitors in terms of higher education (gap year students) and more crucially the labour market, so for them, the comparison really matters.

Overall, results are down from 2022; this isn’t surprising and is part of the glide-path strategy. However, inequalities here are concerning; for C or above, the academy-private gap is up 3.8 ppts compared to 2022.

Among the highest attainers (A/A*), the story is more nuanced. Compared to last year’s cohort the state-private gap has narrowed (1ppt). Remember private schools had far more grade inflation at the top of the distribution when exams were switched to Teacher Assessed Grades in 2021, as figure 3 clearly shows. That they have failed to maintain this stark advantage among top grades suggests some of the record grades awarded over the pandemic were likely down to teacher ‘optimism’. But as mentioned above, the gap among high attainers is up since 2019, therefore also reflecting the better quality of learning experienced by private school students during the lockdowns.

Figure 3: Teacher ‘optimism’ across school types

Subjective scrutiny

Comparing subjects that are more or less objective to assess can also tell us something about how meaningful the results of 2020 and 2021 might be. For example, if we compare results for maths – arguably an easier subject for teachers to grade in their assessments – with drama, we see just how problematic the teacher predicted/assessed grades of 2020 and 2021 were.

Figure 4: Grades by maths and drama

The forgotten 378,000

Of course today isn’t just about A levels – a sizeable proportion of the Class of 2023 – over 378,000 students – took Level 3 vocational and technical qualifications. There is far less data available on these qualifications to provide any real depth of analysis – an issue in itself.

Following a similar pattern to A levels, higher attainment grades for level 3 qualifications are: – up relative to 2019 (at distinction or above), and down relative to 2022 (at merit or above)

Many have pointed out the worrying dropout rates of those taking T levels (the new, technical-level qualification which were launched in 2020) – although it’s important to note that these currently make up only a tiny fraction of vocational and technical qualifications. Of those who did complete, 90% passed with girls outperforming boys at higher levels.

Inequalities persist

In summary then, today’s results present a mixed picture for young people in England. On the one hand, results are back to the pre-Covid heyday, but inequalities are wider, and this again emphasises the inequalities in experiences during the pandemic. These will likely persist well into the labour market – a feature that our COSMO study will track well into the future. Look out for more bleak news to come. This should be a timely reminder to those making education spending decisions in Whitehall for future cohorts.

Post-pandemic schooling challenges: CEPEO’s second annual lecture by Professor Joshua Goodman

By Blog Editor, on 10 August 2023

Lisa Belabed with Jake Anders and Lindsey Macmillan

Each year, CEPEO holds an annual lecture showcasing vital research on education policy and equalising opportunities. For this year’s lecture (held at Central Hall Westminster on Thursday 20 July 2023) we were delighted that Professor Joshua Goodman from Boston University, who recently completed a year in Washington DC on the US President’s Council of Economic Advisors, joined us to talk about his ongoing research on post-pandemic schooling challenges in the United States — which have many parallels with the challenges we are facing in the UK.

Professor Lindsey Macmillan, CEPEO’s Director, began proceedings by recapping highlights from the centre’s past year, including the recently released evidence-based policy priorities, and new research from the COSMO study, as well as reminding us that CEPEO was established just six months before the pandemic started, with its huge implications for educational inequalities and, hence, the direction of our research, including a shift to carry out work to understand post-pandemic schooling challenges.

How it Started

Professor Goodman’s talk was divided into three parts. He started with his research at the beginning of the pandemic, as 50 million children across the US saw their schools close their doors, for what would turn out to be periods ranging from a few months to over a year. At this point, there was virtually no data on school disruption of this scale, and this lack of historical precedent made it difficult to predict learning losses.

However, those unprecedented learning losses were difficult both to grasp and fully anticipate, and there were disparities among parents’ ability to and proactivity in responding to them. Professor Goodman and colleagues used Google Trends to try to quantify the extent of these responses: by April 2020 searches for school- and parent-centred online learning resources doubled compared to pre-Covid levels. Partly because of these differential responses, learning losses were worse for students in high-poverty schools, whose families were less likely to have the resources to compensate for their losses outside of school. Josh pointed out his own ability as a professor and former mathematics public school teacher in helping his children stay on track as exemplifying those disparities.

Beyond learning loss, Professor Goodman highlighted that school enrolment has also been adversely affected by the pandemic, with public school enrolment dropping 3% in autumn 2020, the largest decrease since World War II. Part of this can be explained by an increase in homeschooling, likely driven by health fears, but many of those students effectively vanished.

Josh was also keen to avoid being entirely negative in his assessment of the situation. One silver lining of the pandemic relates to bullying. Turning again to Google Trends data, Goodman and colleagues found that searches about bullying and cyberbullying (perhaps especially surprising for the latter) were reduced as a result of school closures. Suicide rates among children also decreased as schools closed.

How it’s Going

Having described the immediate impacts of COVID-19 disruption, Professor Goodman turned to post-pandemic recovery efforts. Early in the pandemic, the US federal government sent $60 billion to K-12 schools: an unprecedented use of federal funds for school support, as schools in the US are mostly funded at the state- or local-level. This was followed in March 2021 by Elementary and Secondary School Emergency Relief (ESSER), another $120 billion to be spent by 2024. A minimum of just 20% of ESSER was allocated to be spent on programmes to reduce learning loss, and a lot has rather been spent on temporary hires and salary increases for teachers. The extent of this support dwarfs that which we have seen in England, but appears to have been less well targeted.

As our understanding of the effects of the pandemic on learning loss has become clearer, the picture has remained bleak. The US has seen a drop in 8^th grade math scores estimated to lower lifetime income by 1.6% or $1 trillion across all affected students. Chronic absenteeism rates have doubled in many states — more so in schools with high proportions of ethnic minority students — with an average of 6-7 missed days per school year. And enrolment has remained an issue: there was no rebound in enrolment in autumn 2021 to offset the 3% of students who had left public schools at the onset of the pandemic in 2020. While there were widespread concerns about teacher burnout early in the pandemic, there is a lack of direct evidence on this point, but the rate of quitting has jumped 17%.

What now?

Concluding his talk, Professor Goodman was keen to focus on the future and what we should be doing to mitigate the challenges he outlined. Asking “what now?” he set out what he sees as the biggest challenges facing education policymakers at this moment. First, the big federal spending in light of the still terrible learning results is leading to a narrative of wasted funds, which makes it challenging to obtain further funding. But, he pointed out, this ignores the counterfactual: the learning losses without this federal support could very easily have been even worse.

Second, districts (somewhat equivalent to English local authorities, albeit retaining more oversight over schools than many now have here) still do not understand the scope of intervention necessary to tackle the learning loss that has become embedded. On top of this, they are concerned that adding instructional time is not popular, are finding high-quality tutoring hard to scale, and that the teacher workforce is tired, sapping energy for taking the urgent action still needed.

Third, there is a perception that parents do not have the appetite to tackle the embedded learning loss their children are facing. This is largely because they often do not recognise that their child’s academic skills were affected by the disruption of the pandemic, as school grades (“on a curve”) can have tended to obscure learning losses from them.

In the face of these challenges, Professor Goodman stressed the importance of getting attendance back to pre-pandemic levels, as absenteeism makes all other interventions less effective. This mirrors one of CEPEO’s policy priorities of reducing pupil absenteeism. He also highlighted the need to help districts (or, in a UK context, school and academy chain leadership) understand the scope of intervention needed, as well as the importance of choosing solutions that scale. For instance, tutoring has proven difficult and labour-intensive, and Josh argues that other, more widely manageable, teaching solutions would be preferable, notably through technology that supports, rather than replaces, teachers’ work.

Wrapping up

The lecture ended with Professor Goodman reasserting the importance of data, which is foundational in his research on education and economics — the same being true for CEPEO. In the US, the local nature of schooling presents major data challenges. He also stressed the need for timely datasets: federal enrolment data is released with a one-year lag, which he argues is much too long: by the time the data are released, another school year has already begun so policymakers often feel that priorities have ‘moved on’. He also stressed a lack of data on important issues such as teacher workforce, and the difficulty of measuring ESSER funds’ spending and efficiency.

Having had the opportunity to speak with members of CEPEO on their own research, Professor Goodman referred to the datasets being used and developed within the centre. These include pioneering use of administrative data such as the Longitudinal Educational Outcomes (LEO) dataset, in partnership with the UK Government, along with the COVID Social Mobility & Opportunities (COSMO) study supported by UKRI Economic and Social Research Council and in partnership with the UK Department for Education. He called for more partnerships of this kind in building datasets that will allow research that drives forward education policy by helping to identify schooling challenges rapidly and responding to them just as rapidly.

Retain external examination as the primary means of assessment

By Blog editor, on 22 June 2023

By Dr Dominic P. Kelly

Earlier this year, CEPEO launched ‘New Opportunities’, a list of practical priorities for future governments based on the best existing evidence across the social sciences. One of our recommendations regarded calls to abolish high stakes educational assessment by traditional means (i.e., ‘high stakes’ external examinations). Although these calls are not novel, in the wake of disruption to traditional GCSE and A-Level examinations in the past few years, those calls have certainly grown recently. Critics of external examinations argue that they are merely a test of rote memory of impractical knowledge and neither measure the underlying skill they claim to nor improve a child’s educational experiences or later outcomes. There is additional suggestion that these exams lack predictive value for future educational achievement, particularly in higher education, and provide additional stress for students. Based on the existing empirical evidence, we disagree with these assertions. On this basis, our recommendation is to retain external examination as the primary means of assessment and contend that not doing so would harm equity between students in a way that outweighs other concerns around external assessment. Yet, we also advocate for evidence-based improvements to assessments to be made with the intention of holistically improving both the examination system and national curricula and we detail some potential changes below.

External examinations are more resilient to examiners’ bias

One primary concern about external examinations is that they are disadvantageous for students from minority groups compared to coursework or continuous assessment. Although continued effort is needed to modernise the curriculum and provide content and skills that are relevant to diverse backgrounds, the evidence that alternatives reduce disadvantage for minority groups is lacking. It is often suggested that internal assessments by teachers would be an optimal alternative. But evidence suggests that teachers are prone to either showing bias or being inaccurate in their assessments of students from minority groups. Research has shown that some teachers demonstrably show ethnic bias in their assessments such that nearly three times as many Black Caribbean pupils received predicted scores below their actual scores than White students. Additional research also suggests that students from higher socioeconomic status (SES) are more likely to have more favourable internal assessments by teachers than those students from lower SES backgrounds. Although it is sometimes thought that students from low SES backgrounds do comparatively better in coursework than examinations, evidence from the UK suggests that this is not the case. In addition, concerns about examinations causing increasing in student anxiety are contested: although test anxiety is a notable phenomenon among students, it has been suggested that there is minimal effects of test anxiety on performance in GCSE examinations and that children’s wellbeing or happiness was not related to their participation in Key Stage 2 external examinations. Therefore, systems without anonymous external assessments are thought to have substantial biases and this would have deleterious effects for reducing inequalities in the UK education system.

External examinations can be complemented by improvements made elsewhere

Despite our recommendation that external assessments remain the primary means of assessment, that is not to say that these assessments could not be complemented by evidence-based approaches to reform to the current curricula and assessment. First, concerns over the practice of ‘teaching to the test’ could be addressed by changing the content of the external examinations to reward a richer approach to teaching, which would likely benefit students. For example, providing examples with “higher level” items in classroom assessments (which encourage deeper processing of information than items that only require rote learning) may yield comparatively stronger performance on both low- and high-level items in external examinations. Incidentally, reducing the amount of time teachers spend on providing their own assessments of students could potentially provide more time for other activities in the classroom. Second, portfolios and other types of summative assessments can have utility as a complement to traditional exams but they should be marked externally by someone unfamiliar with the student to avoid biases. Third, and most of all, the administration of thoughtful formative assessment has the potential to prepare students for external examinations and potentially providing a greater context for external examinations to be interpreted within. Formative assessment has been shown to have notable positive effects on pupil attainment and can be structured in a way devoid of teacher bias. Given the evidence that single assessments of cognition can be affected by stress, sleep and other extraneous factors, there are benefits to repeatedly assessing smaller and more specific elements of knowledge or cognition in a way that complement single external assessments and benefit learning environments – as long as it can be done objectively and with minimal bias from educators.

Summary

We advocate for external assessments to continue to be the primary means of educational assessment in the UK. Switching to internal teacher-based assessment would set back attempts to reduce inequalities in the UK education system. Furthermore, we would argue that concerns about stress are surpassed by evidence of teacher bias and inaccuracies in internal assessments. There are additional benefits to students and teachers of summative or continuous formative assessment, but these are complements rather than substitutes for external examination. The content and format of external assessments, whether high-stakes or continuous, should continue to be re-evaluated in line with well-founded and methodologically rigorous research.

Reduced engagement in extra-curricular activities post-pandemic: should we be worried?

By Blog Editor, on 19 June 2023

Young people’s engagement with extra-curricular activities following the pandemic

By Blog Editor, on 19 June 2023

The path to a more socially diverse and inclusive workforce

By Blog editor, on 15 June 2023

By Dr Claire Tyler

CEPEO recently launched New Opportunities: our evidence-based policy priorities for equalising opportunities. In this weekly blog, we are highlighting one of our priorities and the reasoning and evidence behind them. This week, we are focusing on socio-economic inequalities in the workplace and how employers can use data to create more socially diverse and inclusive organisations.

Evidence consistently shows that an individual’s social background predicts their chances of accessing a ‘top job’ such as a professional or managerial career – for example, 74% of medical professionals, 64% of journalists and lawyers and 89% of senior financial services professionals originate from professional or managerial backgrounds compared to 33% of the population. Two-fifths of Britain’s ‘leading people’ attended independent schools compared to 7% of the population. Even comparing graduates with similar academic backgrounds, privately educated graduates are still a third more likely to enter top jobs than comparable peers from state school.

But this isn’t just an issue of access: large socioeconomic pay and progression gaps also exist within many occupations. Individuals from working-class backgrounds earn 16% less in top jobs compared to colleagues from more privileged backgrounds. And These issues of access, pay and progression gaps by social background mirror those faced by women and ethnic minorities in the workforce. These gaps could therefore be targeted by a similar data-led policy approach – mandatory gender pay gap reporting was introduced in 2017 in the UK and mandatory ethnicity pay gap reporting is on the horizon after the government released guidance for voluntary reporting last month.

So what do we propose? We suggest the introduction of both entry and pay gap audits by socio-economic background. Employers would report: 1) the proportion of individuals from different socio-economic backgrounds entering occupations (which are easily compared to national benchmarks), and 2) pay inequalities by socio-economic background in a similar way to current gender pay-gap reporting. This policy would shine a light on current disparities in both access and progression of individuals from lower-socioeconomic backgrounds and reveal if career opportunities are being equalised over time.

Why is this a policy priority? Creating a level-playing field in the workplace is not only the ‘right thing to do’; it is also economically advantageous – a win-win strategy. Recent work shows that reduced workplace discrimination improves the allocation of talent in the labour market and drives economic growth. The growing ‘business case’ for diversity argues that more inclusive workforces drive profitability, innovation and better decision making.

It is also the right time for such a policy – the growth of EDI (equality, diversity and inclusion) agendas alongside more responsible and sustainable business practices (such as ESG reporting) has raised the profile of socio-economic diversity in recruitment, promotion and retention. Young people increasingly aspire to work for companies with a strong commitment to diversity – 72% of workers aged 18- 34 recently said they would consider turning down a job offer or leaving a company if they did not think that their manager (or potential manager) supported EDI initiatives. The policy would also crucially provide employers with a baseline for monitoring the impact of their own diversity initiatives and identify ‘what works’ for creating more diverse organisations –particularly vital insight at time when early talent teams are dropping academic credentials, navigating hybrid working for new recruits and designing alternative pathways into careers to widen and diversify their talent pools.

The policy is also likely to have a substantial impact – recent evidence on the effectiveness of UK gender pay gap legislation shows that pay transparency increases the probability of women working in above-median-wage occupations by 5% and closes the gender pay gap by 18%. Evidence on social background shows that firms returning year-on-year to the Social Mobility Employer Index are more likely to demonstrate progress on social mobility – they are four times more likely to be collecting at least three socioeconomic background data points than new entrants to the index, suggesting that transparency and focus can facilitate change.

While most policy levers available to government to equalise opportunities occur before labour market entry, this policy priority highlights one way in which government can hold employers to account for their role in equalising opportunities. If the government is committed to collecting and using data differently to improve social mobility (as stated here), introducing entry and pay gap audits by social background is a cost-effective place to start with benefits for employees, firms and society.

Social mobility scorecards for universities

By Blog Editor, on 6 June 2023

Oliver Cassagneau-Francis

CEPEO recently launched New Opportunities, our evidence-based manifesto for equalising opportunities. In this blog series, we are highlighting one of our policy priorities each week. This post makes the case for why we should introduce an official “Social Mobility Scorecard” for universities. This would both act as an incentive to those universities who are failing to help young people from disadvantaged backgrounds, and to recognise others who are a positive force for social mobility.

Graduates earn more money and enjoy better employment prospects than non-graduates, among other benefits. Governments know this, and so a key policy focus has been to encourage more young people, particularly those from less-advantaged backgrounds, to attend university. This is seen as a way to improve social mobility: put disadvantaged young people through a process that the evidence suggests can elevate people to be among the advantaged in society. And it works for some. However, “university” encompasses a wide range of institutions with wildly varying returns. Recent research in the UK has shown that returns can be as high as 35% for the universities with the highest financial rewards, falling to -5% for those with the lowest returns. So, securing high returns is not just about graduating – from where you graduate matters a lot too.

This issue is particularly relevant for the impact of pro-social mobility policies aimed at higher education. Are disadvantaged students attending the universities whose students get the best returns? Recent evidence suggests they’re not – the “best” universities and subjects are doing pretty badly at admitting less-advantaged young people. Students from the most disadvantaged groups were 100 times less likely to attend Oxford or Cambridge (two of the best-performing universities in terms of returns) than students from private schools.

This is where social mobility scorecards could make a difference. To ensure the “best” universities are not cherry-picking advantaged students (who would likely go on to do well anyway), and to reward those universities helping the most disadvantaged students, we need a public record of how universities are doing with regard to social mobility.

These scorecards are inspired by the Social Mobility Foundation’s “Social Mobility Employer Index” which appears to improve both practice and reporting around social mobility of the employers involved. Going a step further and making university scorecards official government releases will increase their traction among policymakers and other stakeholders, and ensure all universities take them seriously. There is evidence that publishing key information does hold educational institutions to account – for example, abolishing school league tables in Wales led to a 3.3 percentage point fall in the percentage of students achieving at least five GCSEs at A*-C (the key published measure) relative to schools in England (where league tables were not abolished).

Following the lead of researchers in the US, researchers at the IFS have already produced a one-off version of these scorecards – showing that the data required is readily available. They calculate a “mobility rate” for each university, which is the share of students from low-SES backgrounds (“access rate”) multiplied by the share of low-SES graduates who are in the top 20% of earners at age 30 (“success rate”). These calculations led to the damning results for Oxbridge reported earlier. However, there are also some positive findings. London-based universities do particularly well on this metric, mainly driven by their high access rates, with Queen Mary, University of London topping the overall ranking of mobility rates at 6.8%. There is no correlation between the average labour market returns at a university (i.e., wages) and mobility rates, highlighting the need for these scorecards in addition to already available metrics.

Officially publishing these mobility rates regularly (alongside other related statistics) would not only hold the most-selective universities to account regarding their poor performance on this metric, but would also highlight the important contribution of many less prestigious universities to society through their role in improving social mobility. This would be an important step towards ensuring that higher education is a positive force for social mobility, and why we’ve chosen to make it one of CEPEO’s policy priorities.

Persistent absenteeism: Who is missing school since the pandemic?

By Blog Editor, on 1 June 2023

Xin Shao, Jake Anders, and Lindsey Macmillan

Since the COVID-19 pandemic there has been a worrying rise in school absence rates, raising concerns about the detrimental impact on young people’s education. This is a key reason that tackling absence is one of CEPEO’s policy priorities — including better understanding its root causes.

The latest attendance data from the Department for Education (DfE) show that absence rates remain significantly higher than before the pandemic. Missing school can have negative impacts on pupils’ outcomes including educational attainment, wellbeing and wider development. However, as yet there is very little robust evidence about which pupils are most likely to be absent from school since the COVID-19 pandemic, and the potential reasons for this absence.

Improving our understanding of the main challenges to school attendance is important so that effective, targeted policies can be developed to tackle this issue. In this blog post, we provide new evidence on some key drivers of school absences, including providing empirical analysis of qualitative reasons raised in the recent inquiry by House of Commons Education Committee into persistent absence. We use data from the COVID Social Mobility & Opportunities study (COSMO), one of the only studies available to look at this issue, by tracking a cohort of young people currently in Year 13 (or equivalent) whose education has been significantly affected by the disruption of the COVID-19 pandemic.

The DfE’s definition of a persistent absentee is “a pupil having 46 or more sessions of absence (authorised or unauthorised) during the academic year, around 15% of overall absence.” We define persistent absence here as whether pupils met this criterion during their Year 11 in academic year 2020/21. Schools fully reopened at the start of this academic year following the initial disruption of the pandemic, however in-person schooling was suspended again during the third national lockdown, from 4 January 2021 until 8 March.

A picture of persistent absence in COSMO

About 10% of our sample met the definition of persistent absentees. But this overall rate masks important variation associated with the challenges that young people are facing in their lives. There are important differences in persistent absence by socio-economic background and food poverty, demographics, mental health experiences, and SEND status.

Pupils who are eligible for FSM were 14 percentage points more likely than non-FSM eligible pupils to be persistently absent from school (Figure 1), indicating that pupils from disadvantaged family backgrounds are more likely to have school attendance problems.

Figure 1. Percentage of persistent absentees by FSM eligibility

Notes. Analysis is weighted to account for sampling design and non-response. FSM eligibility over the last 6 years. N = 8,774.

Persistent absenteeism also varies significantly by household food poverty (Figure 2). Pupils living in households who have suffered from hunger were 13 percentage points more likely to persistently absent from school, compared to those living in households that did not experience hunger. Pupils from families who used a food bank were 18 percentage points more likely to be persistent absentees (Figure 3), compared to their peers whose families never used a food bank.

Figure 2. Percentage of persistent absentees by food poverty in households

Notes. Analysis is weighted to account for sampling design and non-response. N = 6,297.

Figure 3. Percentages of persistent absentees by food bank usage

Notes. Analysis is weighted to account for sampling design and non-response. N = 6,220.

Boys are slightly more likely than girls to be persistently absent from school (Figure 4), while differences by ethnicity are somewhat larger, with White pupils more likely than other ethnic groups to be persistent absentees (Figure 5).

Figure 4. Percentage of persistent absentees by gender

Notes. Analysis is weighted to account for sampling design and non-response. N = 8,563.

Figure 5. Percentage of persistent absentees by ethnicity

Notes. Analysis is weighted to account for sampling design and non-response. N= 8,686.

Pupils facing challenges with their mental health were also more likely to be persistently absent from school, compared to those do not have mental health problems (Figure 6).

Figure 6. Percentages of persistent absentees by high psychological distress

Notes. Analysis is weighted to account for sampling design and non-response. High psychological distress defined using a score of 4 or above in the General Health Questionnaire, GHQ-12. N = 8,508.

Even starker is the inequality in persistent absence by pupils’ Special Educational Needs (SEN) status. Students with SEN status were 21 percentage points more likely to be persistently absent from school than those without SEN status (Figure 7).

Figure 7. Percentages of persistent absentees by SEN

Notes. Analysis is weighted to account for sampling design and non-response. SEN = Special Educational Needs. N = 8,772.

Key drivers of persistent absenteeism

While there are stark differences in persistent absenteeism across socio-economic background, mental health experiences and SEN status, these factors are likely to be related to each other, so we could be seeing the same underlying difference in multiple ways. We can therefore consider changes in the probability of persistent absence related to each of these factors, while holding the others fixed, and comparing otherwise similar pupils in terms of family background (parental education and occupation) and prior attainment.

Figure 8 shows the probability of being persistent absentees from each particular group, including those pupils living in a household that used a food bank, having SEN status, and being at elevated risk of psychological distress, relative to those pupils not in each of these groups, but with similar prior attainment, and similarly educated parents, working in similar occupations.

Pupils whose family needed to use foodbanks were over 5 percentage points more likely to be persistently absent, even compared to pupils who had similar family backgrounds and prior attainment, but whose family didn’t need to use foodbanks. Similarly, pupils who had SEN status were over 10 percentage points more likely to be persistently absent than pupils without SEN status but otherwise similar. Pupils facing mental health challenges were over 2 percentage points more likely to be persistently absent than otherwise similar pupils not facing high psychological distress levels. Each of these factors feature heavily in qualitative reports of school absenteeism beyond a breakdown of the social contract between parents and schools, and do not fit the caricature of a school truant who is missing school simply because they don’t feel like turning up.

Figure 8. Change in probability of pupils living in households that used a food bank, pupils with SEN status and pupils with high psychological distress being persistently absent from school, relative to their otherwise similar peers

Notes. Reporting change in probability (marginal effects). N = 4,387; Residual Degrees of Freedom: 2456. The model also includes gender, ethnicity, parental education, parental occupational status, and Key Stage 2 prior attainment; SEN = Special Educational Needs

Moving beyond standard approaches to absenteeism?

Understanding rates of persistent absenteeism, beyond a breakdown of the social contact between parents and schools, is crucial for tackling the issue. Building on existing qualitative evidence, we show substantial differences in experiences associated with financial instability, mental health, and pupils’ SEN status, over and above young people’s demographic and socioeconomic backgrounds. This highlights the potential to tackle a number of these factors directly to alleviate possible reasons for young people not attending school.

In particular, reviewing eligibility for free school meals to help lessen the pressure on families struggling to provide food for everyone in the family, or who are having to organise their lives accessing food banks in a way that may be disruptive to children’s education, appears an important factor. Similarly, ensuring that schools have the funding and capacity to support SEN pupils so that they can attend school is likely to be another way to reduce persistent absenteeism. Finally, these findings again highlight the importance of mental health support for young people, both for its own sake, but also because mental health challenges appear to be negatively affecting young people’s ability to engage with their education.

Notes

Aspects of the analysis use administrative data from the Department for Education (DfE)’s National Pupil Database (NPD), where consent was gained for this linkage (73% of young people), with additional weighting carried out to ensure (insofar as is possible) representativeness of analysis using linked administrative data. This work was produced using statistical data from the DfE processed in the Office for National Statistics’ (ONS) Secure Research Service (SRS). The use of the DfE statistical data in this work does not imply the endorsement of the DfE or ONS in relation to the interpretation or analysis of the statistical data. This work uses research datasets, which may not exactly reproduce National Statistics aggregates.

CORRECTION: An earlier version of this blog post incorrectly referred to this absence as being during this cohort’s Year 12 (academic year 2021/22), rather than their Year 11 (academic year 2020/21).

Making Maths Count in Early Childhood

By Blog editor, on 25 May 2023

By Dr Laura Outhwaite

CEPEO recently launched New Opportunities, our evidence-based manifesto for equalising opportunities. In this blog series, we are highlighting one of our policy proposals each week. This post makes the case for why we need to raise standards in maths attainment from early childhood, and how this can be achieved through a new campaign to support parents’ engagement with children’s early maths skills.

Why maths?

Children’s maths attainment has been significantly impacted by the disruptions caused by Covid-19. Only 71% of 11-year-olds met expected standards in mathematics in their end of primary school SATs in 2022, compared to 79% in 2019. This decline in maths attainment was not observed for reading, which showed a small increase from 73% of 11-year-olds reaching expected standards to 74%, over the same period. This also reflects trends seen in longitudinal cohort data prior to the pandemic, where a maths-reading attainment gap emerges in the first years of school, with reading skills significantly exceeding those of maths.

These figures are a great distance from the Levelling Up Mission of 90% of 11-year-olds meeting expected standards in maths, reading, and writing by 2030. It also poses challenges to the Prime Minister’s vision for every young person to study some form of maths up to the age of 18. While there are several shorter-term solutions to these goals, such as the recruitment and retention of specialist maths teachers, we also need to address the bigger picture on factors that impact maths attainment, including the need to emphasise the importance and impact of maths development in early childhood.

Early maths matters

A meta-analysis of six longitudinal datasets shows that early maths skills at the start of primary school are the strongest predictors of later general attainment at ages 10-11, compared to other skills, including reading. How well children do in basic maths skills at ages 4-5 also significantly predicts enrolment in advanced mathematical courses between ages 15-18. This relationship remained significant after socio-economic status was accounted for. Furthermore, children who do well in maths throughout their educational careers are also more likely to have better labour market outcomes, including employment opportunities and earnings in adulthood.

Overall, this is not to say that a focus on maths should replace a focus on reading.  Rather, opportunities for maths need to have an increased presence in children’s early learning environments, including at home.

Maths in the home learning environment

Parents and caregivers typically read with their young children every day, compared to engaging in maths related activities once a week. These engagements in children’s learning at home are shown to be related to their attainment outcomes, including after controlling for socio-economic background. Likewise, parents who report feeling more confident in early maths spend more time engaging with maths activities, which in turn supports their child’s outcomes. Whereas feelings of anxiety about maths from parents can have a negative impact on child’s attainment, as well as their own anxieties about maths.

There are also inequalities in opportunities for active parental engagement with children’s maths development. For example, higher levels of maternal education support higher family incomes, which in turn supports increased parental investments in educational resources at home, and consequently increased maths skills for primary-school aged children. This highlights the need for low-cost solutions and resources that can boost parents’ confidence and engagement in early maths at home.

However, to date, initiatives aimed at encouraging and supporting parents to engage in early learning at home with their children have primarily focused on literacy and language skills. Review evidence suggests these kinds of programmes are beneficial for boosting engagement in the home learning environment. Feedback from the focus groups conducted by Public First about the CEPEO policy priorities also showed this proposal was well received as a way of solving the learning gap, particularly if the resources were online or on-demand. Therefore, we recommend a similar national campaign that targets children’s early maths skills.

Launch a new campaign to support children’s early maths skills

Research shows when involving parents in home learning, simply communicating the need to ‘do more maths’ is not enough. Instead, active parental engagement also needs to be encouraged. This can be achieved in several ways. For example, studies show parent-based educational apps that provide parents and caregivers with resources and ideas for how to engage with their child’s maths development have shown positive and sustained benefits on child outcomes and parent confidence. Mathematical story books and applying maths concepts into everyday life situations, conversations and play have also shown positive benefits.

Encouraging parents to engage with these resources, alongside support from early childhood education and care providers, can be a valuable way forward. Evidence from the Education Endowment Foundation describes the parental engagement on child outcomes as “high impact for low cost based on extensive evidence”. The Centre for Social Justice also calls for an integrated approach for supporting parental participation in children’s education as a way to contribute to reducing the attainment gap.

Overall, in creating this national campaign, it is vital to signpost and promote high-quality, evidence-based resources. This should be combined with working with parents and practitioners to maximise their reach and impact, particularly for those from disadvantaged backgrounds. In doing so, the summarised evidence suggests that this policy priority will make important contributions to raising maths attainment in early childhood with long-term benefits.