Sandy Oliver
What do farmers attending schools in the African fields have in common with women attending maternity clinics in England? Both groups have played a role in rigorous academic research. They have influenced studies evaluating programmes that were designed to improve their lives.
In the mid-1990s Farmer Field Schools were spreading across Africa. These schools use active, hands-on learning and collaboration to improve agricultural productivity. Their strong participatory ethos makes the field schools very relevant to those involved.
Logic tells us that these schools should make a big difference to the farmers’ yields and to their lives. However, a strong theoretical base, enthusiasm and participatory principles don’t guarantee success. A research study seeking to collect, analyse and synthesise a wide range of evaluations of field schools found their success is largely limited to pilot projects. Furthermore, success is less likely with poorer farmers and women farmers.
It would be helpful to know how Farmer Field Schools compared with other approaches to improving agriculture – but the authors found a dearth of such rigorous impact evaluations. They see a need for studies that track potential changes through the whole course of the project — from the preparatory work of training facilitators and identifying potential participating farmers through to the ideas they discuss, try out and share with their neighbours.
They particularly recommend rigorous evaluations assessing impact in broad terms – not just agricultural productivity, but also empowerment, health and the environment.
Carrying out such evaluations is highly skilled work. In fact, knowing how to commission research that will yield really practical information – that will answer the questions and concerns of the people whose lives it is seeking to benefit – is not straightforward either.
Such issues will be part of a short course in Evaluation for Development Programmes offered by the London International Development Centre (LIDC) later this year, on which I will be teaching.
The course will offer opportunities for participants and tutors to all learn from each other, and is designed for:

• development professionals who commission and use evaluation studies
• academics who plan to work in multi-disciplinary teams on future evaluation studies of development programmes and
• postgraduate students who wish to gain a better understanding of the terminology and fundamentals of evaluation methods.

Our vision for the new course is that it will help to achieve effective and appropriate support for better health and wellbeing through training professionals who design social interventions. It will help them to understand, commission, use and interpret evaluation studies, and work with potential beneficiaries such as farmers in Africa or pregnant women in the UK.
Research on anti-smoking support for pregnant women in the UK offers a contrasting example of why rigorous academic evaluation of the impact of social interventions is not enough.
In many high income countries in the 1990s, pregnant women were commonly advised to avoid or give up smoking for the health of their baby. The success of this strategy was assessed by rigorous randomised controlled trials, which reported reduced proportions of women smoking and fewer babies born too soon, too small or sick.
However, these trials took little notice of other criteria considered important by health promotion specialists and pregnant women themselves. What, they wondered, were the effects of encouraging women to give up smoking, if smoking helped them cope with the daily pressures of disadvantaged lives? Might asking midwives to counsel women against smoking interfere with supportive midwife-mother relationships?
Concerned practitioners and women who smoked (some who gave up, and some who did not) discussed their theories about the impact of smoking cessation programmes in pregnancy. At that time these theories had not been tested. Drawing attention to this gap in our collective knowledge encouraged a new generation of randomised controlled trials that took into account the social and emotional consequences, not just biomedical measures, of smoking cessation programmes. Subsequent studies showed that concerns about potential harm, such as stressed mothers and damaged family relationships, were largely unfounded. Now national and international guidelines are based on rigorous evaluations designed with women, not just for them.
These two very different examples raise questions in common about theories of change, research methodology, criteria for success, equity and ethics. They also feature not just individual studies, but whole literatures of similar studies which strengthen the evidence underpinning current recommendations. These key characteristics for evaluating complex social interventions require research approaches that cut across traditional academic disciplines, and draw heavily on the policy, practice and personal knowledge of those directly involved.
 

Sandy Oliver

There is nothing unusual about academics and amateurs sharing and discussing their interests in learning. Professional and amateur stargazers debate the night sky, volunteers dig alongside archaeologists, biographers need readers and museums thrive with interactive exhibits.
Applied research such as medicine, communications or agriculture, elicits opinions about the focus, ethics and governance of research from people interested in the potential benefits and harms of new technologies or ways of working. All this is public engagement with research – where non-researchers are either contributing to the research, or debating or making use of the findings. Citizen science, engaged scholarship, patient involvement, public participation, practitioner research and many other terms describe activities which have overlapping principles and methods.
Sharing lessons between the disciplines and across policy sectors is difficult because we do not have a common language or shared understanding of what public engagement comprises and how it operates.
In the UK universities are supported by Research Councils UK and the National Coordinating Centre for Public Engagement  to encourage a growing culture of public engagement with research – by developing the “the myriad of ways in which the activity and benefits of higher education and research can be shared with the public”. This engagement is taken to be “a two-way process, involving interaction and listening, with the goal of generating mutual benefit”.
Staff at the Institute of Education have been part of this social movement. Universities across the UK and internationally have been opening up their ‘ivory towers’ and finding new ways to work with other people in organisations and networks where knowledge is valued for culture and for policy decisions.
For instance, university students who grew up in local authority care played a prominent role as workshop leaders and spoke movingly about their own challenging experiences at a conference reporting a study of them and 150 of their peers. Ultimately, the By Degrees study (pdf) led to the introduction of a bursary for care leavers who go on to higher education and encouraged many UK universities and local authorities to improve the support they offer care leavers.
On a lighter note, pupils and teachers were involved in pilot testing software (pdf) that would allow young people to make 3D adventure and puzzle games that are as satisfying to play as the ones they buy. Developing a common language was important for cross-disciplinary and cross-generational understanding of game design, and a new quality, commercial product.
Elsewhere, public debate about nanotechnologies (engineering on a molecular scale) illustrated how public engagement can: reveal public concerns and wishes; suggest new lines of enquiry; open science to public scrutiny; provoke reflection on the wider, social implications of scientific developments; and help scientists and the public develop new skills and mutual appreciation.
Ironically, despite holding similar principles, academics who are applying them in various areas for different purposes are often working in isolation, unaware that other enthusiasts are down the corridor or in neighbouring universities. Now, discussions between the eight universities with RCUK public engagement ‘catalyst’ funding, and the NCCPE, have inspired plans for an international journal for academics and others interested in research.
This journal, to be launched by IOE Press, will focus on the role of academic research in society at large, and the role of society at large in academic research. It will publish empirical research and critical analyses of public engagement with research across all academic disciplines; opinion pieces from public perspectives and engagement intermediaries; and reviews of books and events. It is a forum for sharing the learning from research and practice that crosses boundaries between research and the wider world, across academic disciplines and policy sectors.
The journal will consider the questions academics ask about how to choose between different publics and different methods depending on the context of their research projects, and the consequent impact on the research and those involved. It will consider the questions asked by outsiders wishing to engage academics in research – how to read research, news or commentaries with a critical eye, navigate university structures, and inspire academics with new agendas. Lastly, it will consider the systems and cultures that support or block academics and the public learning from each other.
Typical of this area – where choice of language reveals assumptions, cliques and fashions – an appropriate name for such a journal remains elusive. The vision is to bring together the wisdom of academics, practitioners, Science Technology Engineering and Maths (STEM) Ambassadors and all manner of engaged publics. Their task will be to shape a ground-breaking journal – and find a name.