Attrition in digital behavioral research: Divided we fail…
By Carmen E Lefevre, on 19 January 2017
Modification of health-related behaviors has proved to be a challenging target for public health interventions with more than 8,000 published studies cited in PubMed for physical activity and healthy diet alone and with no single intervention consistently winning the race so far.
A few years ago, sixteen partners representing major stakeholder groups formed the Credits 4 Health Consortium (C4H) aiming at creating and critically appraising a system effective in engaging, nurturing and keeping people committed in the adoption of personalized wellness paths and healthy life-styles. Physical activity and Mediterranean diet were the primary targets and, at the same time, the intervention platform had unique motivation-enhancing elements. Our collaborative effort faced right from the start many challenges common in digital behavioral interventions ranging from adopting a common language between the academia, the media, the tech industry, the business developers, and the ethics experts, to interpreting the findings of the study at the various domain-specific levels.
Among these challenges, attrition has been a constant consideration in all our design actions with pressing questions constantly discussed within the consortium: How would non-usage attrition and dropout attrition operate in the C4H environment? Given the complex nature of the intervention which attributes would contribute to the observed attrition rate? Which participants’ groups would suffer the most? Would incentives help or make things worse due to failing expectations?
Although attrition is adequately studied in the areas of weight loss and alcohol management, for physical activity and healthy diet, factors affecting attrition are not well understood. Earlier research has shown a large number of parameters that affect attrition; participants who take more effort to be recruited, are younger, are less physically active, and have worse self-perceived physical health are more likely to withdraw or be lost to follow-up. Attempts to determine a profile for predicting attrition among adults have failed and real-life data make the attrition rates observed in clinical trials look misleadingly good; almost two-thirds of new fitness center members will abandon activities before the third month, and less than 4% will remain for more than 12 months of continuous activity.
Hoping for the best, we chose to implement design features that were expected to tackle attrition; the valid behavioral model, the robust scientific background fully communicated to the participants, the incentives, the continuous prompts, the social activity were all used on top of “classic” attrition-proof features including but not limited to a user-friendly digital environment or optimal personalization.
And there lies a fundamental issue regarding clinical trials in behavioral science: How strongly can you intervene during an RCT to re-engage participants before it no longer can be considered an RCT? In C4H, we observed an ameliorating attrition rate moving from preliminary testing experiments to the full-blown randomized study. The design parameters listed above all contributed in keeping attrition rates at bay (around 40%) considering the general pattern in the field. Do these approaches blur the distinction between efficacy (“Does the intervention work?”) and the much-desired effectiveness (“Does the intervention benefit the participants?”)?
Obviously yes, if we think in terms of the traditional “pharmacological intervention” RCT. But, for digital behavioral interventions, all these attributes are part of the intervention per se. A first challenge here pertains to introducing every new attribute in a structured way that will allow the new version of the intervention to be coherent and reproducible. Another challenge is related to the identification of the individual contribution of each parameter on the intervention’s performance which opens the long discussion of the nature and assessment of complex interventions, including but not limited to using holistic and individual component analytic approaches, or the important issue of incomplete and inconsistent reporting of elements critical to understanding the success and impact of multicomponent interventions.
Measures and design features that will guard against attrition are not the business of the research physician, or the phycologist, or the platform designer, or the community organizer, or the campaign designer alone. Attrition can be addressed only when a multicomponent intervention is treated as such and a full stakeholder panel is functional. Last but not least, participant feedback and active participation in the design (and redesign) process is fundamental.
BIO: Evangelia Ntzani is a pediatrician, a research methodologist and an Associate Professor of Epidemiology in the Department of Hygiene and Epidemiology, University of Ioannina, School of Medicine, Greece (also member of the WHO HEPA Europe network). She is co-investigator of grants on motivating behavior change across diverse areas including physical activity and diet. Her research interests include research design, methodology and bias, evidence-based medicine, clinical and molecular epidemiology, and assessment of large-scale clinical and molecular information.
- Preventing Chronic Diseases. A Vital Investment: WHO Global Report. Geneva: World Health Organization, 2005. pp 200. Accessible at: http://www.who.int/chp/en/
- Eysenbach G. The law of attrition. J Med Internet Res. 2005;7(1):e11.
- McCambridge J1, Kalaitzaki E, White IR, Khadjesari Z, Murray E, Linke S, Thompson SG, Godfrey C, Wallace P. Impact of length or relevance of questionnaires on attrition in online trials: randomized controlled trial. J Med Internet Res.2011 Nov 18;13(4):e96.
- Howie EK, Straker LM . Rates of attrition, non-compliance and missingness in randomized controlled trials of child physical activity interventions using accelerometers: A brief methodological review. J Sci Med Sport. 2016 Oct;19(10):830-6.
- Crutzen R, Viechtbauer W, Spigt M, Kotz D. Differential attrition in health behaviour change trials: a systematic review and meta-analysis. Psychol Health. 2015 Jan;30(1):122-34.
- Schwarzer R, Fleig L, Warner LM, Gholami M, Serra-Majem L, Ngo J, Roman-Viñas B, Ribas-Barba L, Distante A, Ntzani E, Giannakis G, Brandi ML. Who benefits from a dietary online intervention? Evidence from Italy, Spain and Greece. Public Health Nutr. 2016 Nov 10:1-10.
- Trikalinos TA, Wieland LS, Adam GP, Zgodic A, Ntzani EE. Decision Aids for Cancer Screening and Treatment [Internet]. Rockville (MD): Agency for Healthcare Research and Quality (US); 2014 Dec.
- Guise JM, Chang C, Viswanathan M, Glick S, Treadwell J, Umscheid CA, Whitlock E, Fu R, Berliner E, Paynter R, Anderson J, Motu’apuaka P, Trikalinos T. Agency for Healthcare Research and Quality Evidence-based Practice Center methods for systematically reviewing complex multicomponent health care interventions. J Clin Epidemiol. 2014 Nov;67(11):1181-91.