X Close

CBC Digi-Hub Blog

Home

Menu

Archive for the 'Cross-sector Collaboration' Category

Designing for engagement with digital behaviour change interventions: state of the art and chartering new territories

CBC Digi-Hub Blog14 February 2022

Written by Dr Kristina Curtis (UCL Centre for Behaviour Change)

This blog article is dedicated to a panel discussion from the CBC Conference Enabling Behaviour Change to Build Back Better for health & sustainability: 1 – 3 Nov 2021. An interdisciplinary panel of experts took us on a journey from how we define and measure engagement to how we learn from big data and produce a generalisable science of factors that promote engagement.

We kicked off the engagement panel with Dr Kevin Doherty from Technical University Denmark, who began his quest to conceptualise engagement from a review of the computer science literature. Kevin was struck by the wide range of perspectives and conceptualisations of engagement from ‘a quality of user experiences with technology’ (O’Brien and Toms, 2008) to ‘..the phenomena of being captivated and motivated’ (James et al. 2011). To help understand where these different perspectives were coming from, Kevin and his team turned to the original theories, revealing a huge range from most cited Flow Theory (50 mentions) to least cited Richness Theory (2 mentions). Still struggling to pin down what exactly was meant by engagement, Kevin and his team looked next at measurement for further insights. They uncovered a long list of measures such as questionnaires, behaviour logging, facial analysis, and gesture tracking. Next, they looked at the design strategies used to design for engagement and reported a variety of strategies including recommendations such as ‘usability’, ‘immersing users’ and ‘inspiring fun’. As these strategies were employed across a range of contexts and technologies, they were still no closer to pinning down exactly what was meant by ‘engagement’. A combination of Kevin’s applied work and academic review led to thinking about engagement not only in terms of what it is, but how we might design for engagement as a process and how users engage with technology as a process in itself. Kevin ended his talk around remaining challenges, one being how we build interdisciplinary teams and the necessary knowledge to facilitate this kind of approach to engagement.

Our next panellist was Professor Rik Crutzen from Maastricht University brought us on to the topic of measuring engagement. Rik stated as with any psychological construct, it’s important to ‘disentangle’ this conceptualisation from how it is measured. These are not two separate entities but related to one another. As with measuring psychological constructs, we use a plethora of methods to measure engagement such as surveys and reaction times. Interestingly and in contrary to some of the audience’s views, Rik argued against a standardised way of measuring engagement. Instead we ought to keep a ‘decentralised’ meaning, as ultimately it will depend on the context and how we are conceptualising engagement in the first place. Rik’s second point related to the distinction between measuring intended and actual use of an intervention and methods employed to understand usage behaviours. For example, ‘Think aloud’ procedures can help to uncover if certain user groups go to different places on a website and whether any content is misinterpreted, allowing an organisation to continually refine their intervention. Rik’s final point addressed the novel ways to reduce participant burden and increase engagement. For example, avatars can be used as a proxy for psychological constructs and skip logic in questionnaires can help to reduce participant burden via shorter questionnaires. Indeed, reducing participant burden is key to sustaining users’ engagement with an intervention and must be prioritised.

Next on the stage was the panel’s industry expert: Charlotte Summers COO and Director of Research at DDM Health. Charlotte Introduced GRO – a digital platform providing disease-specific education, behaviour change support and feedback to optimize health and wellbeing. Gro has been developed with leading clinicians to address all modifiable risks. Charlotte shared encouraging research findings from peer-reviewed published health outcomes including: Type 2 diabetes remission, weight loss, reduce medication, reduced anxiety, depression and perceived stress. So, what’s in this successful platform? Charlotte described the following features: personalised resources; one-to-one behaviour change coaching; goal setting; community support and integrated tracking. It also uses AI software to optimise engagement and health outcomes. This platform is a patient-healthcare platform to allow real-time health monitoring and engagement information. Charlotte demonstrated through a case study how ‘big data’ can be used to provide answers on intended vs actual use. They noticed that it was actually the younger family members using the app on behalf of their older relatives that did not speak the language or were not digitally literate. These insights allowed them to further refine the app and offer even more personalisation, with a recognition that is important to engage not just the individual, but the family as a whole in helping to manage type 2 diabetes.

The final panellist was Dr Olga Perski from UCL who took us through a stimulating presentation on whether current approaches are getting us closer to identifying ‘timeless’ design elements. Olga drew our attention to the current state-of-the-art around identifying design elements that reliably influence engagement. This of course is important for designing websites/apps that people want to use and saves money through reduced testing of design iterations. Olga’s talk highlighted the importance of an interdisciplinary team with expertise in how we use a combination of user-centred design (e.g., co-design workshops) and experimental methods (e.g., factorial trials) to better understand design elements that influence engagement with health and wellbeing websites/apps. Findings from these kind of research methods have provided important insights on broader design elements such as ‘ease of use’, ‘personalisation’, and ‘interactivity’ which are important drivers of engagement. However, the challenge exists in the myriad of ways these design elements can be operationalised. One could argue, this is also where the creativity and innovation comes into play and again the importance of testing design features with the relevant target audience. In addition, Olga points out that design preferences will change over time and across contexts, which is in part due to the changing social and cultural norms/expectations. Research shows that users tend to rely heavily on “social proof” to help navigate which technologies to adopt. With deeply contextual and ever changing social, cultural norms and expectations, it’s looking increasingly challenging to identify “timeless” design elements…For example, although chatbots were seen as moderately acceptable by internet users in 2019, they will likely be more commonplace and widely acceptable in 2030. So, what’s the solution? Olga left us with the idea of embracing complexity. In line with the common theme running throughout this discussion panel, design principles, as with the conceptualisation and measurement of engagement, will largely depend upon context.

Papers panellists drew on are listed below:

Doherty, K et al (2018). A mobile app for the self-report of psychological well-being during pregnancy (BrightSelf): qualitative design study. JMIR mental health, 5(4)

Doherty, K., & Doherty, G. (2018). Engagement in HCI: conception, theory and measurement. ACM Computing Surveys (CSUR), 51(5), 1-39

Doherty, K., & Doherty, G. (2018). Engagement in HCI: conception, theory and measurement. ACM Computing Surveys (CSUR), 51(5), 1-39

Doherty, K. (2019). Designing the Self-Report of Wellbeing in Pregnancy (Doctoral dissertation, Trinity College Dublin).

Doherty, K et al (2018). A mobile app for the self-report of psychological well-being during pregnancy (BrightSelf): qualitative design study. JMIR mental health, 5(4)

Metz, G., Roosjen, H., Zweers, W., & Crutzen, R. (in press). Evaluating use of web-based interventions: an example of a Dutch sexual health intervention. Health Promotion International.

Perski, O., Blandford, A., West, R., & Michie, S. (2017). Conceptualising engagement with digital behaviour change interventions: a systematic review using principles from critical interpretive synthesis. Translational behavioral medicine, 7(2), 254-267.https://doi.org/10.31234/osf.io/zhj86

Perski, O., & Short, C. E. (2021). Acceptability of digital health interventions: embracing the complexity. Translational Behavioral Medicine.

Saslow, L. R., Summers, C., Aikens, J. E., & Unwin, D. J. (2018). Outcomes of a digitally delivered low-carbohydrate type 2 diabetes self-management program: 1-year results of a single-arm longitudinal study. JMIR diabetes, 3(3), e12

Short, C.E., DeSmet, A., … & Crutzen, R. (2018). Measuring engagement in e- & mHealth behaviour change interventions: viewpoint of methodologies. Journal of Medical Internet Research, 20, e292. https://doi.org/10.2196/jmir.9397

Short, C.E., Smit, E.S., & Crutzen, R. (in press). Measuring psychological constructs in computer-tailored interventions: novel possibilities to reduce participant burden and increase engagement. The European Health Psychologist. https://doi.org/10.31234/osf.io/hz593

 

 

 

 

 

Introducing the Human Behaviour-Change Project (HBCP): Using Machine Learning and Artificial Intelligence to improve behavioural science

ucjubil11 April 2017

By: Dr Emma Norris, Dr Ailbhe Finnerty and Prof Susan Michie

The need

Human behaviour needs radical change to protect our individual and collective health and well-being. To achieve this, we need to develop more effective behaviour change interventions, tailored to the behaviour, population and setting. The Human Behaviour-Change Project (HBCP) is working to build an Artificial Intelligence system to continually scan the world literature on behaviour change, extract key information, and use this to build and update a model of human behaviour to answer the big question: (more…)

Cross-sector collaboration in digital health

Carmen E Lefevre8 October 2016

By Dr Fiona Adshead, Chief Wellbeing Officer at Bupa

Earlier this year, at the UCL Behaviour Change Conference on Digital Health and Wellbeing, I chaired a thought-provoking discussion on Challenges to creating sustainable, high impact interventions.

One of the central themes of that discussion – and of other conversations over the two days – was: how can we get better collaboration across academics, researchers, healthcare professionals and industry in digital health.

Why is collaboration so important?

If there is one message that came through loud and clear it’s that digital health isn’t just about digital.

Whilst digital technologies offer us unprecedented ways to engage, enable, quantify, measure and tailor interventions – digital health fundamentally comes down to changing human behaviour.

Those who attended the conference may remember the brilliant video ‘Uninvited Guests’, wherein an older man has a range of ‘smart’ devices to improve his health foisted onto him by his well-meaning children. What this video so wonderfully demonstrates is that technology alone does not guarantee better health, where human beings and behaviour are concerned.

The best technology is useless without designing for context and starting from a deep understanding of the individual. Harnessed properly, however, technology can play a powerful role at every stage of the behaviour change journey – offerings us scalable, targeted and economical ways to transform outcomes.

Without collaboration across sectors, we cannot fully address the complex, multi-faceted challenges of creating sustained behavioural change, or realise the full potential of what can be achieved.

We need to bring together different perspectives, resources, evidence and creativity to properly understand the problems we want solve and how we can best do it.

What works?

There is no ‘one size fits all’ approach to effective collaboration, but there are certainly some guiding principles.

Collaboration of any kind – even within an organisation or within a single sector – is inherently challenging. Differing priorities, ‘cultural’ differences and conflicts of interest can all stunt effective collaboration. However, if managed well, they can create a very positive tension, driving innovation and dynamism.

For me, some key principles are:

  • Choose the right partners: As in life, the right partners are not necessarily those who are most similar to us, but those who share our underlying values and complement our strengths. It may be less about sector, and more about ambitions and what skills and resources each partner can bring.
  • Don’t overlook the private sector: Speaking from my current vantage point in industry, it is important not to see the private sector as one homogenous group. It is a much more nuanced landscape of different kinds of businesses, with different values, business models, geographical footprints and expertise. By thinking more laterally about who you might find shared value with, you can unlock collaboration opportunities beyond the traditional territories of corporate sponsorship or charitable donations.
  • Find the shared value: It is very tempting to start from what matters to us and what we need. However, just talking about what we care about tends to switch others off. Whether you’re approaching potential partners or working together on a shared agenda, it is really important to focus on shared value. You need to understand each others’ priorities, and find the areas of overlap. During my time in Government, I saw many people come to speak to ministers about a particular challenge or cause. Very few would think through why it was relevant to the minister, but if they did come at it from that angle, they got far better results.
  • Agree on a clear, common goal: Simple to say but often hard to do – you need to be clear on what you are trying to achieve. If your ambition is clear and owned by all, it is much easier to overcome any conflicts or challenges that arise.
  • Recognise and encourage diversity: It is often easier to work with familiar organisations, people, evidence or techniques. However, the point of collaboration is to reach a whole that is greater than the sum of the parts. So we need to look beyond the obvious and be willing to leave our comfort zones. We need to accept and respect differences, find ways of working that play to everyone’s strengths, and be clear about what each party can bring.
  • Be human: Collaboration is ultimately about people and power, trust and relationships. When it is successful, it is because those involved are willing to give up some power to others. Collaborative relationships must be characterised by trust, transparency and a willingness to have frank conversations in the pursuit of a common goal

 

Putting it into practice

Collaboration is something we are passionate about at Bupa – we know we can unlock far more with others than we can alone. By working with UCL’s Centre for Behaviour Change we aim to bring evidence into our practice, and share what works to create tomorrow’s evidence.

As someone who has worked across sectors – at Bupa, the WHO, in government and as a doctor – I know it can be hard to cross the barriers between public, private, NGO, IGO and academia.

It can initially seem we are coming from totally different worlds and – even if we find potential partners – we are speaking different languages and can’t get genuine, effective collaboration off the ground.

However, I genuinely believe these are barriers that can be and should be overcome – by following principles like the above and being open-minded to the possibilities.

The bottom line: try to think laterally about who you could work with. Don’t start from their sector, start from their values and ambitions. And always think through how you can approach people in terms of what matters to them, as well as what matter to you.

You never know – they might be as passionate about improving health by changing behaviour as you are.

BIO: Fiona is Bupa’s Chief Wellbeing Officer, and is responsible for globally leading work on wellbeing, behaviour change and workplace health. Before joining Bupa, Fiona was Director of Chronic Disease and Health Promotion at the World Health Organization, with five years’ experience as Deputy Chief Medical Officer and Director General in the UK Government, responsible for health improvement and health inequalities. Fiona is currently an international adviser to the Royal College of Physicians, a visiting Professor at UCL, Brunel, Exeter, and the Chinese University of Hong Kong. She is a Senior Associate at Cambridge University’s Sustainability Leadership Programme, where she teaches regularly. Fiona is also a trustee of the UK Health Forum and the London Sustainability Exchange.

If You Build It, Will They Come?

Carmen E Lefevre28 September 2016

By Dustin DiTommaso, Behaviour Change Design department head at Mad*Pow

We are in the (messy) midst of a digital health renaissance where health systems, insurers, big pharma, governments, tech giants, startups, behavioural scientists and health researchers are exploring digital technologies to transform the ways in which healthcare is delivered and patients and consumers engage in beneficial health conducive activities. A noble cause to be sure, as many of our infrastructures for providing quality care are in need of radical innovation. Arguably, at the center of our efforts to improve outcomes, lies human behaviour and our attempts to influence and direct it through the provisioning of mobile apps, web platforms, wearables and the like. Well-designed, evidence-based digital behaviour change interventions carry tremendous potential to positively impact clinical, economic and distributional outcomes but these objectives can only be met if and when people engage with them. This leaves us thinking about how to address the perennial question of “If you build it, will they come?” and how to best go about developing digital interventions with impact.

When we survey the digital landscape we find no shortage of tools and services available for public consumption, some highly rated and quite popular (e.g., 4/5 star ratings, millions of downloads) and others much more modest in their reach (e.g., less than 500 downloads). While many of us know, and more continue to discover – “Popular does not equal effective” and yet being effective requires a certain amount of popularity. Our process for designing and implementing digital interventions must not only be thoughtful and precise in terms of change objectives and outcomes but also in promotion, rollout, uptake and usage. To this end, it can be most productive to take a multi-disciplinary approach to the design and implementation of digital interventions where those expert in the underlying mechanics of change and intervention design (e.g., health researchers & behavioural scientists) collaborate with those expert in creating products and services that provide value and resonate deeply with their target audience/customer base (e.g., experience/service/interaction designers & marketing/content strategists). Hybrids welcome, of course.

In this configuration, research and science lead the framing and diagnosis of the problem space and the intervention design strategy (i.e., intervention functions, behaviour change techniques, etc.) while designers and strategists work to codify the unmet needs, wants and preferences of the target audience to guide the voice and tone of the intervention copy, value proposition and persuasive communication strategy and supporting artifacts. Creative translation of the intervention design strategy into a solution that is appealing, engaging and valued while retaining the fidelity and intent of the intervention functions and behaviour change techniques is where art and craft meets science (and trial & error). Close collaboration and iteration is critical here to ensure that the active ingredients of a digital intervention are being delivered in ways that are maximally consumed by the intended audience.

As digital interventions often require active opt-in via download, sign-up or sign-in, recipients must be both aware that it exists and interested in engaging with it. Creating a persuasive messaging strategy that details the audience relevant benefits as well as where and when to reach them along with a staged roll-out plan for how to scale up to critical mass while learning and adapting along the way is where content and marketing strategists can really shine. This iterative approach also applies to the development of the digital intervention itself, of course.

Though there are certainly several different configurations that teams can take, collaborative approaches like the ones outlined above help to align resources on all sides of the impact equation – uptake, engagement and outcomes – allowing each team to focus on what they do best and maximizing the chances that the digital interventions developed together will have the greatest intended effects possible.

What collaborative configurations have you tried in the past? What has worked well and where have you gotten stuck? Where do you think the greatest opportunities for collaboration lie? I would love to learn about and discuss your experiences in the comments below!

 

BIO: Dustin DiTommaso heads the Behaviour Change Design department at Mad*Pow and teaches Behaviour Change Design at Rhode Island School of Design. A designer and researcher, Dustin’s work involves the study and application of behavioural science, motivational psychology and human-computer interaction to the design of technology-assisted behaviour change interventions, products and services. His client portfolio includes partnerships with a range of innovative start-ups, non-profits, government agencies and Fortune 500 companies across domains, including healthcare, financial services, education and social impact.  

Academic collaboration – a startup point of view

Carmen E Lefevre20 September 2016

By Dr Jussi Tolvi, co-founder and behavioural lead at Club Soda

Club Soda is a social startup business. We use technology to help people set goals for their alcohol use, and to reach those goals. We also work with pubs and bars, nudging them to being more welcoming for mindful drinkers – customers who want to drink less or not at all.

One of the key values of Club Soda is to use evidence in everything we do, from product development to content creation. We benefit enormously from research findings, seminars and conferences, and talking to academics about their work. And we know that our customers (though we prefer to call them members) also like it that we have some science behind us.

We would like to do more research and evaluation work ourselves as well, but the reality is that we just do not have the resources for much of it at the moment. Most academic research grants may not be measured in millions, but we have to make do with much less, and without secure monthly salaries to fall back on.

And we would love to collaborate more with our academic partners. Of course we will always help them find study participants when needed, and will happily share what we have learnt. But we also have to think of ourselves a bit: if a grant funded academic team creates an app in our domain, it might not be in our best interests to have that app freely available to our potential paying customers. We will always have to have one eye on revenue to pay for our continued existence.

Understanding consumer demand is not a bad thing for academic research either. There is real value in learning from and about people who have made a choice, and spent their cash on a product or service offered to them by a business. Club Soda is more than a set of behaviour change techniques. We offer people membership in a tribe: who you get sober with is as important as who you get drunk with – and we know there is fascinating research material in our community on this topic too!

There is bound to be some tension between academia and business. Sometimes we will have shared aims, sometimes not. We won’t always understand where the other one is coming from. But we believe it is beneficial for us both to keep cooperating, learning from, and sharing our unique areas of expertise.

With that in mind, I wonder whether there are particular types of businesses that academic researchers would prefer to work with?

BIO: Dr Jussi Tolvi is a former academic and City worker, now co-founder and behavioural lead at Club Soda, a startup nudging people and licensed venues towards healthier and more mindful drinking. He studied economics before behavioural economics was fashionable, so has had to work hard to forget most of what he learned at university.