Developing the behavior change intervention in participation with users: Can it be too much?

By Ann DeSmet (1) & Geert Crombez (2)

(1) Postdoctoral Fellow, Physical Activity and Health Team, Ghent University

(2) Professor of Health Psychology, Ghent Health Psychology Lab, Ghent University

Many behavior change interventions involve users in the development of the program. An approach of user participation, user-centered design, formative research or co-creation is believed to increase user empowerment and user commitment, which may eventually lead to more effective interventions with less dropout. It is then not surprising that many studies report the involvement of users in the design of an intervention. Notwithstanding, there is a full panoply of ways to involve users. And it is often less clear which user involvement strategies were used, and which strategies result in better interventions.

Evidently, a theory-driven approach may miss contextual and individual particularities in everyday life.  It is good that developers and experts are aware that they do not know all or know best.  A move away from a paternalistic approach is commendable.  Nevertheless, the pendulum may have swung too far the other way. Target users are often expected to bring in expert knowledge on designing material. The opinion of a few, highly-selected users are taken as representative for the entire population, or, worse, as the absolute truth. We recommend a mindful balancing between a theory-based, person-based and practice based approach (see e.g.).  We need to carefully consider when and why we involve users, which strategies to use, and how to best combine users’ input and professional expertise. In an era of big axioms such as empowerment, participatory approach and co-creation, our view may be met with resistance. However, it is corroborated by evidence, and arose after investigating the effectiveness of participatory design in serious digital games.

Participatory design in serious digital games can involve users as informants (e.g., asking for preferences via qualitative research) or as co-creators of the intervention (e.g., in charge of part of the material design). A key principle of participatory design is that target users are involved throughout most or all stages of the intervention development. It is contrasted with less intensive user involvement, such as users only testing a prototype developed by professionals, or having no users involved at all.

We tested this issue in a meta-analysis of 58 digital serious games for health promotion, expecting to find superior effects for more intensive user involvement. Counter to expectations, we found the opposite: games developed via participatory design were less effective in achieving behavior change than games where users were only involved as testers. They were also less effective in raising self-efficacy than games that did not involve users at all. Within the group that used participatory design, games that involved users as informants were overall more effective than those where users were involved as co-creators. Effectiveness also varied as a function of the topic on which users were consulted. It seemed that games were more effective when users were consulted on parts that related to the cognitive challenge, but less when consulted on artistic aspects such as character or game world looks.

Evidently, one may dismiss the results as due to inappropriate reporting or contextualization of user involvement in studies. We however hope that such reflections do not create blind spots for what we consider crucial questions to address:

• Are we expecting too much from target users when asking them to help design the intervention?
• Should we provide training to target users when involving them in intervention design?
• How should we recruit target users for user involvement?
• How much of the user input actually makes it to the final design?
• How can we best combine user knowledge with professional expertise?

As a first step, we advocate an extensive reporting of the specific strategies used to involve users, and a transparent reporting of the communication and decisional process in the team, showing where users were involved and how much of their input was retained in the final design. The way forward may not be to have less user involvement, but to have user involvement for the right reasons and at the right time.

Bio:

Ann DeSmet (@ann_desmet) is a postdoctoral fellow funded by Research Foundation Flanders, and part of the Physical Activity and Health team (@PA_Health_UGent) at Ghent University. Her main research interests are e&mHealth and innovative techniques to improve engagement and effectiveness of digital health interventions. Ann’s research to date has focused on serious games for health promotion; digital interventions to promote sleep, physical activity and reduce sedentary behavior; and associations between healthy lifestyles and mental health.

 

Geert Crombez is professor of health psychology. He is coordinating the research on the psychology of health and ill-health of the Ghent Health Psychology Lab (www.ghplab.ugent.be). Foundational to his research is a motivational perspective that is built around the powers of goals and self-regulation. This perspective allows different levels of analysis, and is applicable to a wide range of health-related problems (health promotion, treatment adherence, management of chronic illness). Taking his research a step further, he is developing eHealth tools for health promotion and coping with illness.