Dr. Robert Kaplan

The research community appears to be relishing the challenge of producing enough scientific evidence to prove or disprove the efficacy of mobile health technologies while also remaining open to new methods of evaluation that can keep pace with rapid innovation.

"The literature has really mushroomed in the last couple of years," Robert M. Kaplan, director of the Office of Behavioral and Social Sciences Research at the National Institutes of Health, told MobiHealthNews.

"There is a growing evidence base," agreed Wendy J. Nilsen, a health scientist administrator in the same NIH office.

That was not the case when NIH convened a workshop in August 2011 to discuss the state of mobile health research.

"[W]hether mHealth leads to better overall health outcomes and reduced disease burden is still unknown," says a report from that workshop, published last week in the American Journal of Preventive Medicine. "In a healthcare system already burdened with suboptimal outcomes and excessive costs, premature adoption of untested mHealth technologies may detract from, rather than contribute to, what is needed for true overall health improvement," the report cautions.

In addition to changing the way healthcare is delivered, mobile technologies are changing the way medical research is conducted. "Because many measures collected by mHealth can be obtained remotely in real time without the subject having to participate in traditional measurement visits, data analysis can be conducted more quickly, sometimes in real time," according to the paper, which has 19 named contributors, including Nilsen.

"This can enable studies to be concluded earlier than planned when evidence of outcomes is obtained. Near real-time data analysis also can be used to control various versions of adaptive experimental approaches and designs. Additionally, the streaming data of mHealth can be used for real-time predictive modeling."

These characteristics, along with the fact that innovations in mobile technology are occurring so much faster than the medical establishment is used to, has led some to question whether the long-accepted gold standard of research, the randomized controlled trial, is even appropriate for mobile apps. (One of the 19 contributors, Bonnie Spring, director of the Center for Behavior and Health at Northwestern University, defended RCTs in the name of patient safety at the 2012 mHealth Summit, but did allow that traditional trials might be too slow for mobile health.)

The NIH officials say there is room for other approaches. "We have heard a lot in mobile about just ditching randomized controlled trials," Nilson tells MobiHealthNews. "Randomized trials are right in the right situation … but I think researchers are ready to embrace new methods that reduce burden," she adds.

"The real issue is bias," according to Kaplan. "RCTs reduce bias. We definitely want to preserve the protection against bias," Kaplan says.

However, the ability to collect data outside of traditional laboratory settings might be shifting the bias. The lab, according to Kaplan, is "not representative" of how technologies work and patients behave in the real world. "This is just a revolution in the collection of data in ways that we couldn't get data before," he says.

"In the past, we collected very little data," Nilsen says. "This is joyful."

The continuous collection of data for evaluation deserves to be tested as an alternative to the RCT, Nilsen says, as does the role mobile technologies can play in recruiting subjects for traditional trials. "Some of the things that are slow are not really the randomized trial's fault," she says.

"By increasing scalability, reducing burden, and spreading access to people beyond the reach of traditional health research, new populations may consider participating who would have never done so before. By broadening the participant pool for research studies, research can be not only more efficient but also more generalizable," the journal article states.

There does need to be a balance between measuring results in real life – data that may be skewed by external factors – and taking readings in the "perfect" conditions of the lab, according to Nilsen. "The computing community is looking at algorithms to reduce noise," Nilsen says.