Close
Lesson study: new evaluation did not do this valuable type of teacher development justice
By Blog Editor, IOE Digital, on 18 December 2017
Sarah Seleznyov, with members of the Collaborative Lesson Research Group*.
“If there is no impact on teaching and learning, then that (intervention) is not lesson study”. (Akihiko Takahashi at a workshop in London, 7 December 2017)
Having looked forward to the Education Endowment Foundation’s evaluation of lesson study for some time, we are disappointed by the outcome. This disappointment is not about the low effect size, but because this narrowly-focused review of just one fairly basic model could discourage schools from using a very valuable approach to teacher development.
We would challenge the EEF study’s methodology and argue that the conclusions (e.g. no evidence of improved maths and reading at KS2) are misleading. Our own conclusion is that the version of lesson study used in the project was a ‘first step’ version, which most teachers and schools new to the practice need to go through, but certainly not a thorough version which is known to, and might be expected to, have impact.
In lesson study, a Japanese approach to professional development, a group of teachers collaboratively plans, delivers, observes and discusses lessons that have a particular teaching focus. We represent an emerging community of lesson study practitioners, particularly in mathematics, who have been working closely with Japanese maths educators over the past seven years to develop a deeper understanding of lesson study and its potential.
What was the nature of the intervention studied in the evaluation?
This seems to us the most critical issue. Lesson study as used in Japan is established in a whole-school environment of practitioner inquiry and research. This appears to be missing from the project that was evaluated here. The report recognises that the ‘Lesson Study’ model used in this project is just one of many available to teachers in the UK’ (p.5). The key question is whether it accurately embodied the critical components of Japanese lesson study to ensure high quality outcomes:
Figure 1: Critical components of lesson study
| 1. Identify research focus The research focus will likely require several research lessons, for each of which distinct research questions should be developed. 2. Planning Teachers work in collaborative groups to carry out kyozai kenkyu (study of material relevant to the research theme). This study leads to the production of a collaboratively written plan for a research lesson. This detailed plan, written over several meetings, attempts to anticipate pupil responses, misconceptions and successes for the lesson. 3. Research lesson The research lesson is taught by a focus teacher, who is a member of the collaborative planning group. Other members of the group act as silent observers, collecting any available evidence of pupil learning. 4. Post-lesson discussion The collaborative group meet to formally discuss the evidence gathered, following a set of conversation protocols. Their learning in relation to the research theme is identified and recorded by the discussion chair. It is intended that this learning informs subsequent cycles of research. 5. Repeated cycles of research Subsequent research lessons are planned and taught that draw on the findings from the post-lesson discussions. These are new lessons and not revisions nor re-teachings of previous research lessons. They may involve new focus teachers and new classes. 6. Outside expertise At key junctures, outside experts or knowledgeable others (koshi) are invited into the process to provide inputs into the planning process and the research lesson. 7. Mobilising knowledge Opportunities are created for teachers working in one lesson study group to access and use the knowledge from other groups, through observing other groups’ ‘open house’ research lessons, from the knowledgeable other’s experiences of networking across schools, or through the publication of group findings. (Seleznyov, forthcoming) |
There is no reason to assume that any one of these critical components can be easily dispensed with, without compromising the quality of the lesson study process. Some of the components are trickier to achieve than others, but the schools we support are experimenting with ways to achieve these as faithfully as possible within the cultural and structural restraints of the UK education system.
The EEF project evaluators frequently refer to lesson study as a form of peer-to-peer observation and feedback. But as the list of critical components above makes clear, Japanese lesson study is about so much more than that. If the lesson study conducted under the project was merely a form of peer-to-peer observation and feedback, it is not surprising that there was little impact observed.
The evaluation states that: ‘The goal of this intervention was to assess if this model of Lesson Study develops a superior teacher training programme compared to the status quo’ (p.45), and lists as one of the key conclusions:
‘Teachers felt Lesson Study was useful professional development, valued the opportunity to collaborate with colleagues in a structured way, and reported several changes to their practice as a result of the programme.’ (p.4)
It also describes how lesson study enabled teachers to better understand pupil needs and what constitutes effective pupil learning, and to develop more effective collaborative relationships around the sharing of good practice both within and across schools.
Using a more rounded set of measures – for example ones based on Guskey’s five levels for measuring professional development – these descriptions could be seen to show that lesson study was in fact successful, since it engaged teachers, enabled learning and facilitated positive changes to practice, as well as positive changes to school culture. The evaluation did then measure whether these changes to teachers’ practice made a difference to pupils’ learning. However, it can be argued that the evaluation of lesson study – the professional development approach – as opposed to Talk for Learning – the classroom intervention – should have measured pupils’ outcomes for participating teachers’ own current pupil cohort and not the teachers’ historic pupil cohort, who they no longer teach.
Therefore, we suggest that no one should use this EEF study to argue that lesson study is ineffective. Rather, we should conclude that this particular version of lesson study had little effect and that a different set of outcome measures might have been more appropriate to measure its anticipated outcomes. The next step should be to conduct lesson study working to a more effective model that embodies the important features that ensure long-term professional learning.
*Rosa Archer, Manchester University; Professor Toshiakira Fujii, Tokyo Gakugei University; Dr Sachi Hatakenaka, Bowland Maths; Rachael Horsman, Cambridge Maths; Matt Lewis, NAMA; Professor Keichii Nishimura, Tokyo Gakugei University; Professor Akihiko Takahashi, De Paul University, Chicago; Professor Geoff Wake, Nottingham University
Reference: Seleznyov, S. (forthcoming). Lesson study: an exploration of its translation beyond Japan.
Photo by Donna Cleveland via Creative Commons
One Response to “Lesson study: new evaluation did not do this valuable type of teacher development justice”
- 1
[…] Education Endowment Foundation declared an ineffective tool for CPD. (Read more about this on the IOE blog.) This is the quality of the evidence schools are being asked to base decisions on. When you look […]