Once a promising futuristic industry, digital health is merging into mainstream medicine. Now in the limelight, conversations around how these digital tools should be validated are starting to take place. 

But for startups, getting validation often means securing money and partners, creating potential barriers for young startups. Last night at Partners HealthCare Pivot Labs digital health event, industry experts dished on the why, when and how of validation. 

Why get validation?

It’s no secret the app market is now flooded with digital health products. Many of those apps remain unregulated and unvalidated. However, validation is one way for developers to set their products apart. 

“A fun fact is that everyday at least 200 apps get added on the app store. Just in the mental health space there are 1,500 apps,” Dr. Ramya Palacholla, research scientist at Partners HealthCare Pivot Labs, said at the event. “We are so passionate about validating solutions because when there are so many different apps —1,500 different apps that claim to do the same thing — then what separates me from the others?"

But validation isn’t just about setting a product apart on an app store — startups seeking to work with big names in the medical world will need to show data, according to panelists. 

“For the program that I run, validation is so critically important because we connect later stage and middle stage startups — entrepreneurs that have solutions ready to go — we connect them with organizations like Mass General, Brigham and Women’s, and they want to see the data, right? They want to know this actually works and it is really solving a real problem,” Nina Kandilian, director of operations of Vertical Programs at MassChallenge, said. 

When it comes to validating a technology money matters, according to panelists, stakeholders need validation that the product can produce a return on investment. 

“You could do all these amazing things and to solve all these problems but at the end of the day if there is no financial incentive tied to it there is only so long you can work on it,” Kandilian said. 

Different validation for different stakeholders

Validation doesn’t always mean a standard clinical trial. It could mean a lot of different things depending on the users and the purpose. 

“I think we should define what validation is. So there is clinical validation, but there is validating what is a useful tool and even earlier than that validating if it is needed,” Dr. Mark Zhang, medical director of the digital innovation hub at Brigham and Women’s Hospital, said during the panel. “We see ideas and concepts at all stages.”

Validation expectations also different significantly depending on the end user. 

“Value looks different to different stakeholders. As a clinician, the value I look for is better workflow efficiency, decreased workload and just improving patient outcomes,” Palacholla said. “As a patient I may look for increased convenience or just a pleasant experience. So I think it is really important to determine what value you are looking for and the type of stakeholder also determines the type of value you are looking for.”

Looking to the future, Zhang wondered whether the digital health field will develop in a way similar to the drug world, with prescription and over-the-counter drugs. Many digital health consumers are already using products in this way. 

“Eventually we are going to be able to say we have a certain ailment and I need to find an app for it,” Kandilian said. “A lot of people are [already] doing that for meditation, exercise, weight loss."

While there are a significant number of untested products on the market, the industry is also seeing the rise of FDA-cleared digital therapeutics—which require substantial validation. 

“Think about the digital therapeutics space exploding," Kandilian said. "It is a space we don’t know that much about. Think about Pears and Akilis of the world … these apps that are going through FDA approval right now. It’s amazing. I was talking to a VC a couple of weeks ago that said digital therapeutics are the direction we are all headed because it creates that framework and guidelines, and all of the clinicians know there is a process to validate and scientifically understand if something works. Digital is headed in that direction.”

Validation on a budget

Validation can be expensive — posing a potential hurtle for unfunded startups. But there are options for these companies, according to panelists. 

“So early stage entrepreuners, what the VCs really want is the data,” Kandilian said. “They want to know that it works, and it is really hard to do that when you have no money. So I have found a lot of startups in our program that are earlier stage have mastered the art of finding grants.”

Seeking venture dollars might not be the best course of action for every young startup, Kandilian said. 

“There are so many other alternatives to regular VCs," she said. "Do you really want to give up all of your company at the beginning? A lot of the angel VCs you want to find are people with money that can also help you with your business."

Partnerships is another avenue for startups to validate their tools. However, panelists stressed the importance of pitching to the right clinicians. 

“It’s a lot easier to make a connection if there is a target," Zhang said. "It’s a lot harder if you have a solution and you want to connect with [for example] neurology …  that can be harder. We may have a contact, but we may not. So if you’ve already done the leg work that is always helpful.”

That leg work can mean getting involved in the entreprenuer world as well as researching the best fit. 

“I searched a lot of clinical papers and found out who are the clinicians that are looking for advanced digital health solutions to monitor patients' temperature or monitor their fever status,” Rong Xia, CEO of Raiing Medical, said. “I also followed Pivot Labs frequently, and attended events and subscribed to their newsletter to find out who are the early adopters, and can help us to identify the validation's direction, and help us to facilitate study protocols and study processes.”

Another resource for emerging companies is new professional guidelines, which Palacholla said could help shape the future. 

“I think we are heading in a positive direction there are a lot of frameworks and guidelines that are being put out by different organizations, whether it is the NIH or AMA or even the World Health Organization, that two days ago released some guidelines,” Palacholla said. “So all of these guidelines are moving from infancies, from where we just cited products and great products for different users, to actually forming a body of evidence and standardizing and coming up with all of these different guidelines.”