Best Practice in Grouping Students: Mixed methods in action

By qtnvarl, on 19 March 2018

– Dr Becky Taylor

At the heart of BPGS are two interventions: Best Practice in Setting (BPS) and Best Practice in Mixed Attainment (BPMA). Our interventions draw on decades of prior research into attainment grouping. Both are intended to combat some of the unintended, but frequently-observed negative consequences of setting. BPS does this by describing an approach to attainment grouping that does away with these poor practices, while BPMA is designed to provide a framework for teaching students in mixed-attainment groups as an alternative to setting.

As Project Director Professor Becky Francis makes clear here, the project team do not subscribe to easy ‘rights and wrongs’ on grouping practice. The point of the ‘Best Practice in Setting’ trial was to explore whether setting could be done with better fidelity and equity (and consequent better impact on outcomes) than has been found by prior studies.

Our project has investigated the two interventions using ‘mixed methods’ – a pragmatic approach to research that draws on some of the strengths of both traditional quantitative and qualitative methods and allows us to investigate not just ‘what works?’ but as they say ‘why?’, ‘how?’ and ‘in what context?’. We offer a summary of our methods here:

Randomised Controlled Trial

Our two interventions are being independently evaluated by NFER using randomised controlled trial (RCT). The RCT of the BPS intervention, which involves 126 schools, will be able to detect whether the intervention has had any effect on the attainment and self-confidence of the students who participated. The BPMA trial is a ‘feasibility trial’, which means that its main purpose is to establish whether it might be possible to carry out an RCT in this area in the future. As only 13 schools participated, it is underpowered, meaning that the effect of the intervention cannot be detected. We completed data collection in July 2017. The RCT results are expected to be published in May 2018, following a process of peer review. The protocols can be found here.

Questionnaire surveys

Teachers and students in all 139 of the schools participating in our study were asked to complete online questionnaire surveys at the beginning and end of the study period. One important purpose of the student questionnaire was to collect student data, including a measure of self-confidence, for use in the RCT analyses.

It is also important for us to understand how faithfully intervention schools followed our interventions, so the teacher questionnaire asks questions to help us establish this. We also asked teachers and students about other key areas that have been highlighted in previous research into attainment grouping.

Interviews and observations

Alongside the quantitative methods above, we visited a smaller number of schools to conduct interviews with teachers and students, and in BPMA schools to carry out a small number of lesson observations as well. These qualitative approaches help us to check findings from the quantitative data and to understand in richer detail the attitudes of students and teachers toward different grouping practices. They also help us to understand the ways in which grouping practices are enacted in different schools and how they are experienced by teachers and students.

Our analysis so far

While we wait for the results of the RCTs to be published, we have been analysing findings on particular aspects of the data, especially from the baseline survey results. These do not address the success or otherwise of the RCT: they measure aspects from pupils’ opinions and experiences early in the study. The resulting journal articles draw primarily on the quantitative analysis to explore trends, and use qualitative data for explanatory power. We have also published on our findings about various reasons schools found it hard to implement some of the stipulations of our interventions. You can find a list of our publications here.

We will continue to publish our findings as they emerge, and intend a large-scale dissemination event following release of the trial results so that those interested can engage with and interrogate our findings and conclusions.

New findings on secondary students’ negative views about setting

By qtnvarl, on 23 January 2018

– Dr Antonina Tereshchenko

Setting is a widespread practice of grouping students in UK secondary schools. This is despite little evidence of its efficacy and substantial evidence of its detrimental impact on those allocated to the low sets.

In her recent paper [accessible online here] Professor Louise Archer, along with the ‘Best Practice in Grouping Students’ research team, has proposed that setting represents a process through which the social and cultural reproduction of inequality and dominant power relations are enacted within schools.

The arguments are based on the statistical modeling of the set allocation data from 94 project schools, a survey with 12,178 Year 7 students, and interviews with 33 students from four schools.

What are the characteristics of students in higher and lower sets?

The analysis of the school-reported set levels confirms social justice concerns that underpin our project.  Notably, lower sets are overpopulated by disadvantaged students. The allocation of students by subject also highlights gender and racial inequalities.

• Working-class children and those eligible for free school meals were statistically more likely to be in middle and bottom sets.
• A statistically significantly greater proportion of boys were in the bottom set for English (60% boys vs. 40% girls), but significantly more boys were in the top set for mathematics (56% boys vs. 44% girls).
• White students were significantly more likely to be in top sets for English (81%) and mathematics (77%), whereas Black and mixed-ethnicity children (and Asian students in the case of English) were more likely to be in lower sets for both subjects.
• Variation by whether students spoke English as an additional language (EAL) was only significant in English (not mathematics), where higher proportions of students with EAL in lower sets.

Feelings of students in top sets: ‘proud’, ‘confident’ and ‘superior’

Students in top sets (who were also White and/or middle-class) spoke about how being in their sets made them feel ‘proud’, ‘confident’ and ‘superior’. It is also clear that while those at the top may like their location, they are aware that those lower in the hierarchy may dislike their allocation. The following quote exemplifies their beliefs:

You feel good about yourself when you know that you’re thriving in the top set, not that you’re being dragged along the bottom [ … ] It makes you feel good [ .. . ] I think you must feel superior to the group below you, until you’re at the bottom. I think, yes, I do enjoy it but you also have to be quite careful with what you say and how you act. Like, you don’t want to be going round to people saying, ‘Oh, well, I’m in the top set and you’re in the second set’, because that makes people feel really hard [bad], and so I do enjoy it but you do have to be careful with what you say. (Monica, White British)

The top set students viewed being in top sets as ‘natural’ and ‘deserved’. They defended the legitimacy of setting and set allocations drawing on discourses of talent, ability and meritocracy. In the view of top set students, setting is not an unfair practice that produces unequal outcomes. Rather, it is a ‘common sense’ approach.

Who expresses the most negative views of setting?

The following groups of students have statistically more negative attitudes to setting:

• Students who perceive themselves to be in the bottom sets.
• Students with lower levels of attainment at Key Stage 2 tests (both for English and maths).
• Boys express more negative attitudes than girls do.
• Black students and those (ever) eligible for FSM express significantly more negative views of setting.

Students in low sets on their set position: ‘I almost died’

Students in bottom sets were painfully aware of their inferior position. The interviews highlighted feelings of embarrassment – thus one boys described his feelings on learning that he had been allocated to the bottom mathematics set as ‘I almost died’.

Students in bottom sets were striving to move up. Those who had managed to move out of the most-disparaged bottom sets expressed their relief:

Well, I used to be in Set 5, then I moved up to Set 4, so I’m happy now, because I’ve moved up. (Levon, White and Black Caribbean)

Yeah, I don’t mind. It’s better than being in Set 5. (Emily, White Other)

While students in bottom sets were the most likely to raise questions about the legitimacy of set allocation, the above quotes suggest that those who ‘escape’ the fate of bottom sets are more likely to accept the legitimacy of setting.

This research foregrounds the views of those who are most disadvantaged by the setting practice as a tool for questioning the ideology and popularity of setting.

Project Update: Highlights from 2017

By qtnvarl, on 20 December 2017

– Dr Anna Mazenod

This has been a busy year for the project and we would like to share with you some of our highlights in 2017:

We have completed data collection…

• We have worked with more than 140 schools over the two years of the project to collect quantitative and qualitative data
• Teachers and students from participating schools have completed the final round of questionnaires and we have been busy matching the 2015 baseline questionnaire data from over 14 000 students and over 700 teachers with the final questionnaire data
• We now have qualitative data from over 50 teacher interviews and nearly 50 student focus groups

We have published more findings from our project…

• We published a paper discussing the relationship between student’s self-confidence and setting, ‘Attainment Grouping as self-fulfilling prophesy? A mixed methods exploration of self-confidence and set level among Year 7 students’ in International Journal of Educational Research
• We have written about our project in publications aimed at practitioners and researchers (Mathematics Teaching; Research Intelligence)

We have presented our research at national and international events…

• Academic conferences in the UK (British Sociological Association Conference, British Educational Research Association Conference, and the British Psychological Society’s Psychology of Education Section Conference) and in Denmark (European Conference on Educational Research)
• Practitioner and policy events, including British Society for Research into Learning Mathematics conference, ResearchED, the Westminster Education Forum and Hampshire school leaders’ conference
• Symposia and seminar presentations (International symposium on ‘ability’ grouping at UCL Institute of Education, funded through the UCL Global Engagement Fund and Centre for Research on  Learning and Life Chances seminar)

And there’s still plenty to look forward to in 2018!

• We have many more project papers out soon with topics including students’ negative views of setting, why teachers struggle to equitably allocate students into sets, teacher constructions of students in bottom sets and students’ experiences of mixed attainment, and will notify you as these are published (you can follow us on Twitter @GroupingStudy to get the latest updates on the project)
• The evaluators for our project, NFER, arranged for students to take Progress Tests in English and Mathematics last summer. This will enable us to know what the impact of our interventions has been on students’ progress, and our funders, the Education Endowment Foundation, will be reporting on the outcomes in Spring/Summer 2018
• Look out for new materials developed with English teachers to support mixed attainment English teaching in secondary schools
• And we’ll be inviting you to come and hear more about our project findings at events in 2018

Lastly we would like to wish you a very happy festive period from the Best Practice in Grouping Students team!

Photo by William Warby, CC BY 2.0

Psychology of education conference in Edinburgh: Teachers’ beliefs about student ‘ability’

By qtnvarl, on 8 November 2017

– Dr Becky Taylor

I recently visited Edinburgh for the British Psychological Society Psychology of Education Section Annual Conference 2017. With a focus on ‘Learning, Teaching and Assessment’ it had been difficult to resist submitting an abstract based on our project findings, and it was a great opportunity to discuss the project with fellow psychologists.

For the occasion, I presented some analysis that we hadn’t shared before, looking at teachers attitudes toward and beliefs about student ‘ability’. If you follow our project closely, you will have noticed that we refer to ‘attainment grouping. ’ We believe that the term ‘ability grouping’ confuses educational attainment with a notion of innate potential academic ‘ability’, which in turn legitimises in educational discourse the inequality of outcomes that we observe. Instead, we think it is helpful to consider ‘ability’ as flexible as this can motivate students to overcome failure and remedy mistakes through effort.

Photo by Carlos Delgado, CC BY-SA 3.0 licence.

Findings from our baseline teacher questionnaire reveal that teachers tend to see student ‘ability’ as flexible, scoring a mean of 3.8 on a scale of 1-5, where 5 is the highest score for belief that ‘ability’ is flexible. We examined these findings to see if we could find any differences between groups of teachers, but they appear to be consistent regardless of subject taught (maths or English), length of service or the grouping practices in their school (setting or mixed attainment).

We then explored what teachers said about their students in interviews. We found that many teachers talked about students in their classes as homogenous groups, labelling them according to set level. For example, one English teacher said, ‘Upper band … are avid readers … whereas middle band … watch television and play computer games, they don’t read.’

Furthermore and replicating previous research, we found that teachers’ perceptions of their students dictated the curriculum and pedagogical choices that they made, with lower attaining students getting more repetition and rehearsal, while higher attaining students get more problem-solving and creative opportunities. As we have argued elsewhere, this is limiting for students in low sets, and may contribute to the poorer outcomes for these students.

What is particularly interesting here is that teachers’ classroom practices don’t align with their beliefs about students’ ability. Even though teachers believe that students can improve their outcomes, evidence from our interviews suggests that they still don’t provide the opportunity for low-attaining students to do so.

Attainment grouping as self-fulfilling prophesy?: A new project paper

By qtnvarl, on 25 October 2017

– Dr Anna Mazenod

Our latest project paper ‘Attainment grouping as self-fulfilling prophesy? A mixed methods exploration of self-confidence and set level among Year 7 students’ has just been published in the International Journal of Educational Research. In this blog post we offer a digest of this important paper.  For the full article click here.

Prior research has identified a tentative relationship between attainment grouping and students’ self-confidence, with those in low sets having lower self-confidence than their peers in high sets. However, findings have not always been consistent, and there is much complexity at stake. The relativist nature of students’ sense of self has for example been illustrated by the ‘Big-fish-little-pond’ effect whereby students’ sense of self has been shown to depend on the characteristics of the group they compare themselves against. Drawing on the quantitative and qualitative data we have collected as part of the Best Practice in Setting trial, we can now shed more light on the link between self-confidence and students’ set level. Crucially, we have established significant relationships between students’ self-confidence and their set level. These relationships hold for both self-confidence in maths and English, but also for students’ general self-confidence. Our findings provide evidence of the impact of labelling through attainment grouping and of the self-fulfilling prophecy as one important explanation for low attainers’ poorer results.

Set placement has an impact on student self-confidence in maths and English

From our extensive student survey with 11,546 students aged 11/12 we have found that placement in ‘ability sets’ has an impact on student self-confidence in learning. Analysing the data through a series of multilevel models we can identify that set placement has an impact on students’ self-confidence for the subjects in which they are set. Our study has focused on two subjects; English and maths, and we have identified a similar pattern in both subjects. Students in the top sets for maths, for example, have the highest self-confidence in maths. There is then a step down in student’s self-confidence in maths when comparing top sets with other sets. Lowest levels of self-confidence in maths are reported by students in the bottom set for that subject.

Set placement has an impact on student general self-confidence

It may be argued that findings about self-confidence in a subject correlating with set level are not surprising, given that set level reflects prior attainment in these particular subjects, which might be expected to impact confidence. However, further analysis of the survey data shows that the impact of set placement extends beyond subject self-confidence to general self-confidence. Top set students are more self-confident in their learning across the range of subjects, whilst students in the middle and bottom sets progressively score lower in their general self-confidence. These new findings provide evidence of the impact of labelling through set allocation, and we have been able to explore this further through analysis of our qualitative data.

Self-fulfilling prophecy an explanation for low-attainers’ poorer results

Our survey data is complemented by data from individual student interviews and focus groups with 66 students. The potentially detrimental impact of set placement on student self-confidence is strikingly summarised by one focus group participant in her assertion that ‘sets ruin your self-esteem.’ The wealth of qualitative data we have gathered attests to the impact of labelling on student’s self-confidence, and particularly how set placement can be internalised by students as their ‘ability’ label. Our findings thus lend support to the hypothesis of a self-fulfilling prophecy as one important explanation for low-attaining students’ poorer results.

You will be seeing more results from our data analysis in the coming months as we gear up to the final stages of the project, so remember to save space on your reading list for more research findings from Best Practice in Grouping Students!