Online social networks are becoming an increasingly prevalent strategy for public health and other stakeholders to cut down unhealthy behaviors. Whether they’re built for a mainstream audience or to address the issues of a focused patient population, these digital communities all share a major challenge: members who aren’t interested in participating.
“Poor engagement in a digital behavior change intervention really is poor treatment received,” Molly Waring, assistant professor and director of methodology core at the University of Connecticut’s Center for mHealth and Social Media, said during a presentation this week at Mad*Pow's Health Experience Design (HXD) Conference in Cambridge, Mass. “[Think] of all the health apps where people download them once, look at them once, never use them again — they’re really not receiving the app at intervention. … Thinking about this for in-person treatments, if you can’t get people in the room and you have a room of empty chairs, no one is getting the intervention and you’re not going to get good outcomes.”
During back-to-back talks hosted at HXD, Waring and Amanda Graham, SVP of innovations and a research investigator with the Truth Initiative, each described their recent research regarding social network-based behavioral interventions for postpartum weight loss and smoking cessation, respectively. While Waring’s smaller Facebook-based group of about 20 participants didn’t resemble Graham’s investigations involving thousands of BecomeAnEX users, both of the researchers reported generally improved behavior-based outcomes among community members.
“The really unique thing about an online social network is it brings current and former smokers together in a way to support each other that really no other treatment modality is capable of,” Graham said in reference to her group’s longstanding cessation program and online community. “When we think about telephonic interventions and face-to-face interventions, they’re time limited. They happen at certain periods of time. At two in the morning, six in the morning, a smoker can log on and be connected in a really meaningful way with not only other people who are struggling to quit, but people who are sharing their tips and strategies and real-life experiences of success, and that can be really powerful.”
Taking a deeper dive into their data, however, the researchers had a number of insights regarding how driving user engagement can stoke improved behaviors and outcomes — most of the time.
“There’s sort of an assumption that more engagement is better … but not all engagement is really created equal,” Waring said. “In one study, we actually incentivized a handful of participants to engage more to get everyone else to engage more in their group. We found that was successful in … increasing overall engagement in that group. However, the people we incentivized to participate more and the people they [encouraged] did not lose more weight. In fact, they didn’t lose an impressive amount of weight at all — many other people lost much more weight than they did. So it’s not a simple story.”
After noting these and other discrepancies — for instance, a single outlier patient who was the highest engager as well as the greatest weight gainer — Waring and her colleagues initiated follow-up studies focused on measuring the impact specific engagement actions had on outcomes.
What they found, she said, was that posts and comments that asked peers for help or reported healthy choices were associated with increased weight loss, while those describing slip-ups were not. What surprised Waring was that posts coded as containing negativity or a lack of motivation were also associated with improved weight loss outcomes.
Similarly, in another study investigating the types of content that can be posted in a social network to improve member engagement, Waring and colleagues found that posts asking members to report their weights or their goals received the most written replies, while other posts or videos containing informational material only received Likes and generated few conversations. However, speaking to study participants afterward revealed the latter posts to be by far the most popular and well received.
“We’re starting to ask ourselves, should every intervention post start a conversation?” Waring said. “There are posts that maybe just increase a sense of group cohesion, or connectedness, or are fun so people are engaged and it keeps them coming back, but we wouldn’t consider it meaningful engagement in that it doesn’t lead to behavior change or outcomes. Or, [posts] that provide information that participants find useful — most of the informational posts did not get any engagement, but when we ask what you like most, people are like 'Oh, I loved the recipes and the links to the mommy and me exercise videos.'”
Graham’s investigations weren’t free from unexpected data either. She described findings from a trial of 5,290 current smokers using BecomeAnEX that were randomly assigned to one of four cohorts — use of the social network alone, use of the network supported by free nicotine replacement therapy (NRT), use of the network with certain users actively working to engage and integrate them into the community, or all of the above.
To the researchers’ surprise, smoking cessation rates were consistent across the study groups and not, as they had hypothesized, significantly increased when combining the independently proven interventions. Upon reviewing the data for a potential explanation, Graham and her team noticed a trend: each of the groups was riddled with non-compliers and participants who were initially assigned one intervention but engaged with another.
“[This behavior is] actually pretty common, especially for designs that are designed to mimic the real-world conditions where we’re testing interventions and we have very little experimental control,” she said. “What we saw [when we re-evaluated outcomes by the interventions participants actually used] was more in line with what was hypothesized. … In the treatment arms where we had not delivered the social network intervention, use of the community had a stronger impact on user outcomes. And in the arms where we hadn’t provided NRT, NRT had a stronger impact on outcomes. Those of you who know and work with self-determination theory, this is really consistent with the notion that behaviors that are self-determined and volitional have a stronger connection to health behavior change.”
Seeing first-hand how users gravitated toward their own preferences reinforced Graham’s belief that investigations of digital behavior change interventions — as well as the platforms and interventions themselves — need to be designed around the tendencies of those who are being treated. Doing so will require study and platform designers to combine quantitative measures of user engagement or behavior outcomes with qualitative investigations and an overall iterative design process, both researchers said.
“I think that’s where quantitative measures really fall short, and that’s where this user-centered design is really critical to understand somebody’s individual priorities, their motivations,” Graham said. “That’s the thing. There’s what we build for our users, but [also] the systems we can embed users in that inherently address those questions, those ideas.”
Outside of her research, Graham also had more general advice for audience members on how best to grow and develop their own behavioral intervention social networks. After acknowledging the clear advantage BecomeAnEX enjoyed as the focus of a multi-year national marketing campaign, she said that up-and-coming support networks will be most successful if they focus on inclusivity and invest in a dedicated community manager.
“We made a strategic decision in building [BecomeAnEX] to keep the community open. A lot of the time you’ll find that the communities sit behind a paywall or research study wall. Probably the biggest driver of the formation of this community is that it’s been open for 10 years, and we’ve had a large volume of users,” she said. “That said, community management is a skill. It’s an art. If you’re trying to build a community, you want to make sure that you have a very trained, talented, warm, emphatic community manager as part of your team — managing anonymous online people can be tricky, managing personalities can be tricky. For every community that succeeds there’s 10 more that don’t, so I think there’s also a little bit of pixie dust, [especially with] who happens to come along and become an elder for the life of a community — their personality and what they offer.”
