Global learning: is ‘Teaching at the Right Level’ a good buy?

23 November 2020

By Angela W Little, republished from UKFIET, the education and development forum

In 1900, the comparative educationist Michael Sadler wrote:

Nowadays, the ‘children’ who wander the garden include philanthropists, NGOs, trade unions, international and comparative educationists, international businessmen and businesswomen, as well as the all important country policymakers and politicians.

Through their recently-released report ‘Cost Effective Approaches to Improve Global Learning’, the Global Education Evidence Advisory Panel (GEEAP) offers some education ‘great buys’, ‘good buys’ , ‘promising buys with low evidence’ and ‘bad buys’. In short, they offer a ‘great buy’ flower here and a ‘bad buy’ leaf there.

Teaching at the Right Level (TaRL)

Among the ‘good buys’ are ‘interventions to target teaching instruction by learning level, not grade (in or out of school). This is known as ‘Teaching at the Right Level’ (TaRL). The essential idea is that students should be grouped for teaching, based on what they know and can do, rather than on their formal age or grade. The cited evidence rests on some positive results from evaluations of TaRL programmes in (i) India, over the period 2001-2014, (ii) Western Kenya, in 2005, and (iii) Botswana, in 2020.

While all employ a randomised trial evaluation design and similar measures of learning outcomes, all have adopted different programme designs. The report authors suggest that TaRL has also been trialled in Ghana and Zambia but no links to these cases are provided. Of 19 treatment effect estimates reported across the Indian studies, 10 are positive and 9 show no effect. To date, there have been no replication studies of any of the TaRL programmes, either within or across countries.

Costs, scalability, long-term impact and independent evaluation

The GEEAP accords most weight to interventions that include information about costs, scalability and long-term impact. However, none of the TaRL research reports cited refer to costs[i], only the India case has ‘gone to scale’ (an intervention in one state has been adapted in other states) and none of the programmes is reported to have had a long-term impact.

The Indian and Botswana cases have involved a close collaboration between programme designers, implementers and evaluators. While this may demonstrate how evaluation information can help designers and policymakers to improve their programmes (i.e. the evaluation is used formatively to improve the next programme iteration), it may also suggest that the evaluation has been conducted by interested parties i.e. the evaluation is non-independent.

TaRL is an old idea

That TaRL is an old idea does not invalidate its value as an idea or practice. But it should encourage those who promote it to be aware of a much larger literature on this ‘buy’ than that cited. Historical research tells us that in the period of Late Roman education, literacy teaching was geared to the learning level of individual students. In the late 18th/early 19th Century, the Monitorial system of education was practised in over 30 countries. Students were grouped according to their progressive mastery of learning tasks, i.e. teaching at the right level, and taught by pupil monitors.

During the 20th Century, many systems of education have employed a range of ability grouping practices. The OECD-PISA and PIRLS cross-national studies show the extent of different types of between-school, within-school and within-class ability grouping practices. Many well-respected researchers based in the US and elsewhere, including economists, have used rigorous evaluation designs to evaluate the impact of ability-grouping practices on learning gains for different groups of learners. Results are mixed, with several researchers pointing to a widening of the learning gap between high and low achievers when students are streamed or tracked by ability.

As an educator, I support student-centred and differentiated teaching that generates learning benefits for all. And I support reviews of curriculum levels and content. I also appreciate reviews of evidence that are systematic and which report null, negative and positive effects in context.

System-wide reform

In recent years the international community of development assistance has been at pains to emphasise the importance of system-wide support to countries. And the Foreign, Commonwealth and Development Office (FCDO), through UK Aid, has invested a considerable amount of money into research on education system reform. The approach adopted in this report  – offering ‘smart buys’ irrespective of a consideration of all the related elements of an education system that might make that ‘buy’ more or less feasible and affordable – runs counter to this broader advocacy.

To be fair, the GEEAP suggests that judgements about the ‘smartness’ of the ‘buy’ should be undertaken alongside context-specific analysis and system diagnostics, without explicitly setting out any protocols for how this should be done. In the case of TaRL, the note on context reads ‘effective where there is a wide variety of learning levels within a class and student learning levels are below grade level curriculum expectations (most LICs and MICs)’. The reference to ‘most’ is tantamount to ignoring context characteristics of individual LICs and MICs altogether.

System characteristics that are likely, a priori, to mediate the interpretation and implementation of TaRL include, I would suggest:

• the history of in-country streaming, tracking and grouping practices;
• the familiarity of teachers, teacher educators and education officers with tracking and grouping practices;
• the availability of funds for innovation;
• the availability of NGOs to organise, support and fund teachers;
• the internal structure of the current primary education curriculum (including curriculum blue prints and levels, textbooks, remedial and enhancement learning materials);
• prevailing summative and formative assessment practices;
• prevailing philosophies and theories of learning;
• prevailing dominant beliefs among teachers, parents and students about the determinants of learning progress (e.g. innate ability, effort, home support, task difficulty, teacher, luck); and
• the readiness of curriculum development agencies and in-country teacher education institutions to interpret and implement the ‘good buy’ message.

I am not averse to the sharing of good education practices. On the contrary, innovations in one country have often been borrowed from another and re-interpreted and implemented very successfully in specific contexts at specific points in time (e.g. the Massachusetts State Board of Education borrowing from Prussia of the grade-wise organisation of education in urban schools in the 19th Century). Policymakers worldwide do need to be aware of the rich emporium of education ideas and practices that have emerged and survived over time and space. They need to include a consideration of some of them in their policy deliberations. They also need to consider the inter-related sub-systems of education which may need to be re-aligned to effect change and have an impact on students’ learning.

What education policy makers need is a simple-to-use and accessible catalogue of succinctly-described ideas and practices, with an indication of, inter alia, their advantages and disadvantages in specific contexts, reasons for their implementation success/failure, simultaneous changes made in related subsystems and the main elements of cost. This would help to guide them through the marketplace of education ideas and practices on offer in the 21st Century. While such a ‘catalogue’ will contain many gaps, the ideas and practices would be available to all. Market stallholders will of course do all they can to promote the practices they are hoping to sell – but shoppers should be allowed to make their own judgements about the best buys, in relation to a wide range of evidence, needs and means. Critically, shoppers should be wary of the money-lenders and pawn-brokers standing at the next stall.

[i] The evidence on costs, as reported, is obscure. The main report offers the reader links to published papers on TaRL that contain no information about TaRL costs. A related technical paper by Angrist et al (2020) refers to Learning Adjusted Years of Schooling (LAYS) per $100 for some TaRL studies, but it unclear which and how many. Significantly, interventions with null effects are omitted. Why?

This blog was written as a response to the GEEAP ‘Cost Effective Approaches to Improve Global Learning’ report, published in October 2020. Other blogs written in response to the report are:

Smart buys, great sales and special offers: cost-effective approaches to improve global learning

Smart buys: system reform, local-level buy-in and reaching the most marginalised