Health research studies conducted using health research apps are capable of recruiting a large number of participants, but these efforts often see early dropouts and do not enrolls samples representative of the larger population, according to a study published this week in NPJ Digital Medicine.

The investigation, headed by the science research-focused nonprofit Sage Bionetworks, reviewed user engagement data from eight such digital health studies conducted between 2014 to 2019. In reviewing the behaviors of nearly 110,000 study participants across more than 850,000 study days, the researchers also identified a handful of study design decisions that were associated with continued participation.

“In the last five years, several digital health studies, including remote interventions and clinical trials, have been conducted using smartphone technology,” Abhishek Pratap, principal scientist at Sage Bionetworks and the study’s lead author, wrote in a post on the organization’s website. “Despite the success where researchers were able to enroll thousands of research participants in a short amount of time, participant retention and long-term engagement in fully remote research remain a significant barrier for generating robust real-world evidence from [real-world data].”

TOPLINE DATA

Across the eight digital studies, the majority of participants fell within the ages of 17 to 39 years, with those aged 60 years or older being substantially underrepresented. Most participants identified as non-Hispanic whites (median percent across studies, 75.3%) followed by Hispanic or Latino (median, 8.21%) and African American or black (median, 3.45%). These proportions were substantially different than distributions at the national and state level, the researchers wrote.

Median retention time varied significantly between the studies, most often falling within two and 12 days with the exception of a 26-day outlier (in which participants were offered monetary incentives). Overall, median retention clocked in at 5.5 days, during which in-app tasks were completed on two days.

The researchers saw significantly increased retention among those who had clinical conditions relevant to the study, as opposed to the non-disease controls. Median retention time was also significantly higher for those referred to a study by their clinician when compared to those who self-selected. While declared gender was not tied to retention behavior, participant age analyses showed those over 60 years to stick with the study for longer than their younger, more prevalent peers.

“Despite the limitations, several factors, such as partnerships with clinicians and providing research participants fair compensation for their time in the study, could help researchers retain diverse participants in future digital health studies,” Pratap wrote.

HOW IT WAS DONE

The researchers compiled user engagement data from four app-based digital health studies that was made available to the public (the MyHeartCounts, mPower, Asthma and Brighten studies) as well as from four similar studies provided by collaborators (the SleepHealth, Start, Phendo and ElevateMS studies). Five of these looked to recruit participants from the general population, while three looked for those with a specific condition of interest. Only the Brighten study offered renumeration for participation — all other studies were purely observational.

Those who participated in the studies enrolled entirely through a website or the study app. The researchers limited all retention behavior analyses to the first 12 weeks of a participant’s activity to match two studies with a fixed 12-week participation period. The researchers harmonized the differences in user activity data and demographic data for their analysis.

THE LARGER TREND

Digital technologies and platforms seeking to revamp long and costly health studies have been proliferating over the better part of the decade. In terms of population health, the most prevalent recent examples are those built on the back of Apple’s devices, three more of which launched in the tail end of 2019.

Pharmas are starting to hone in on the benefits of digital clinical trials as well. Here, a number of startups including Deep 6 AI, Clinical Trial Connect, SubjectWell and PatientWing have sprouted up to answer the call.

IN CONCLUSION

“Left unchecked the ongoing bias in participant recruitment combined with inequitable long-term participation in large-scale ‘digital cohorts’ can severely impact the generalizability and undermine the promise of digital health in collecting representational real-world data,” the researchers wrote.”