Best Practice in Grouping Students: Mixed methods in action

– Dr Becky Taylor

At the heart of BPGS are two interventions: Best Practice in Setting (BPS) and Best Practice in Mixed Attainment (BPMA). Our interventions draw on decades of prior research into attainment grouping. Both are intended to combat some of the unintended, but frequently-observed negative consequences of setting. BPS does this by describing an approach to attainment grouping that does away with these poor practices, while BPMA is designed to provide a framework for teaching students in mixed-attainment groups as an alternative to setting.

As Project Director Professor Becky Francis makes clear here, the project team do not subscribe to easy ‘rights and wrongs’ on grouping practice. The point of the ‘Best Practice in Setting’ trial was to explore whether setting could be done with better fidelity and equity (and consequent better impact on outcomes) than has been found by prior studies.

Our project has investigated the two interventions using ‘mixed methods’ – a pragmatic approach to research that draws on some of the strengths of both traditional quantitative and qualitative methods and allows us to investigate not just ‘what works?’ but as they say ‘why?’, ‘how?’ and ‘in what context?’. We offer a summary of our methods here:

Randomised Controlled Trial

Our two interventions are being independently evaluated by NFER using randomised controlled trial (RCT). The RCT of the BPS intervention, which involves 126 schools, will be able to detect whether the intervention has had any effect on the attainment and self-confidence of the students who participated. The BPMA trial is a ‘feasibility trial’, which means that its main purpose is to establish whether it might be possible to carry out an RCT in this area in the future. As only 13 schools participated, it is underpowered, meaning that the effect of the intervention cannot be detected. We completed data collection in July 2017. The RCT results are expected to be published in May 2018, following a process of peer review. The protocols can be found here.

Questionnaire surveys

Teachers and students in all 139 of the schools participating in our study were asked to complete online questionnaire surveys at the beginning and end of the study period. One important purpose of the student questionnaire was to collect student data, including a measure of self-confidence, for use in the RCT analyses.

It is also important for us to understand how faithfully intervention schools followed our interventions, so the teacher questionnaire asks questions to help us establish this. We also asked teachers and students about other key areas that have been highlighted in previous research into attainment grouping.

Interviews and observations

Alongside the quantitative methods above, we visited a smaller number of schools to conduct interviews with teachers and students, and in BPMA schools to carry out a small number of lesson observations as well. These qualitative approaches help us to check findings from the quantitative data and to understand in richer detail the attitudes of students and teachers toward different grouping practices. They also help us to understand the ways in which grouping practices are enacted in different schools and how they are experienced by teachers and students.

Our analysis so far

While we wait for the results of the RCTs to be published, we have been analysing findings on particular aspects of the data, especially from the baseline survey results. These do not address the success or otherwise of the RCT: they measure aspects from pupils’ opinions and experiences early in the study. The resulting journal articles draw primarily on the quantitative analysis to explore trends, and use qualitative data for explanatory power. We have also published on our findings about various reasons schools found it hard to implement some of the stipulations of our interventions. You can find a list of our publications here.

We will continue to publish our findings as they emerge, and intend a large-scale dissemination event following release of the trial results so that those interested can engage with and interrogate our findings and conclusions.