Expanding our Toolkit to Evaluate Commercial Mobile Health Apps

By Dr. Danielle Jake-Schoffman, University of Massachusetts Medical School

While there has been a proliferation of commercial mobile apps to support health behavior change in recent years, the evidence to encourage using these apps is limited in a variety of domains. This is not a groundbreaking conclusion given a flurry of review papers that have examined a selection of commercial apps and have consistently found that they come up short in terms of their use of evidence-based guidelines and behavior change techniques. However, evaluations of commercial apps and the types of solutions to bridging potential deficiencies in their features have been limited in scope, also limiting our ability to determine the value of using apps in research and clinical settings.

We recently published a detailed guide to methods for evaluating apps across the domains of content, usability, and efficacy. Within each domain, we describe a variety of research questions and methods that can be used to explore commercial products to determine their suitability for use in research and practice. The goal of the paper is to shed light on the diversity of methods that could and likely should be used to better evaluate commercial apps and other technology products. This pushes research questions of behavioral scientists past the domains most closely related to our training—e.g., content analysis—and stretches us to consider for example factors that could influence usability of apps and what can potentially be done to enhance commercial apps or work around their shortcomings in order to avoid having to start development from scratch. While user-centered design principles and usability testing are becoming more common in the development of researcher-driven apps, their use is still rare in the evaluation of commercial apps. There is a need for more field and laboratory testing of commercial apps, with large user bases and sophisticated design elements, to better understand what is working well and why. Further, there could be opportunities to partner with industry to use research insights to offer suggestions for the improvement of future iterations of apps.

Ultimately, the speed and dynamics of technology development from the industry pipeline will likely continue to outpace that of researcher-developed products. Instead of criticizing the lack of behavioral science included in these commercial apps, perhaps it is time to focus on other methods of working with what exists and bridging gaps with supplemental materials or training provided to research participants where needed. If researchers still elect to develop their own apps, conducting content, usability, and efficacy testing of what exists commercially can still shed important light on features that might be critical to successful development and user adoption.

Although many commercial health apps are lacking from a behavioral science perspective, consumer use of these tools continues to increase (recent survey showed app use has tripled since 2014).

It is critical for the research community to understand what these tools offer, what they are missing, and how we can leverage their strengths and popularity to move the needle on behavior change at scale.

Discussion questions:

• What role should commercial health apps play in future behavior change research?
• Should health behavior researchers focus on development of new apps or finding ways to make use of existing commercial products?

Bio:

Danielle Jake-Schoffman, PhD is currently a postdoctoral research fellow in the PRACCTIS (Prevention and Control of Cancer Training in Implementation Science) program at the University of Massachusetts Medical School. Her research focuses on leveraging mobile technology to operationalize evidence-based strategies for chronic disease prevention, specifically, for the promotion of physical activity, healthy eating, and weight loss. Follow Dr. Jake-Schoffman on Twitter (@DrJakeSchoffman) for all things #digitalhealth and behavioral science.