Attendees to the Connected Health Conference in Boston this week got a sneak peek into the workings of the FDA's much-discussed Pre-Cert program. In a panel discussion at the event, representatives from Verily, Pear Therapeutics, Tidepool, and Johnson & Johnson candidly discussed their experiences so far with the program, describing an evolving process that is collaborative and rigorous. 

"For me, the key takeaway is FDA is really committed to this, they’re in the weeds," Dave Amor, VP of quality and regulatory affairs at Pear Therapeutics, said. "...The tone of the engagement with FDA during this process has been not combative, not antagonistic; it’s been really positive and really working toward this common goal."

Verily Senior Counsel and Trust and Compliance Officer Afia Asamoah said that her company was initially surprised by how involved they would be in shaping the program.

"I think in our first meeting everyone realized we were truly engaging in helping the FDA think about and develop the program," she said. "I think we assumed going in that the framework was already there and we’d give some comments and feedback and be ready to go. But we’ve been really pleasantly surprised with the FDA coming in with an open mind and listening to diverse perspectives to make sure this program is scalable and effective."

Eduardo Sanchez, Johnson & Johnson's director of quality compliance, stressed the revolutionary nature of the program.

"The FDA was really pretty bold in announcing this," he said. "If anyone has spent an appreciable amount of time working with health authorities, this is not how the normal interaction between industry and FDA operates. It’s so much more collaborative."

Excellence appraisals have begun

Howard Look, president and CEO at Tidepool, revealed that the FDA has already held the first early iterations of what the program's "excellence appraisal" might look like. Look says Tidepool volunteered to go first because, as the only nonprofit in the pilot, it has the least to lose by revealing its operating procedures to its peers.

"Last week we had two different sets of appraisal," Look said. "And basically we were doing a comparison of two modalities, one was CMMI [Capability Maturity Model Integration] and for the other we did an appraisal with representatives from Pear and Verily — we called it a peer appraisal. FDA specialists sat in the room watching how it’s done, and then my team was on the hot seat for four days. ... The appraisers learned a ton and we as a company learned a ton. We recently had a debrief with FDA folks that was really helpful."

Asamoah said it's been equally informative from the reviewer's side.

"Our team was comprised of someone with expertise in software quality systems, general expertise in software quality, and one of our physicians. The first takeaway was that it was incredibly helpful to have folks with such different backgrounds on the audit team," she said. "The second was a huge appreciation for Tidepool in terms of their willingness to answer questions and not only that but actually show how it’s done. The auditors said that was very helpful and even necessarily. ... The test run has been invaluable. The takeaway is it works, but there are improvements to be made."

Look said that the final appraisal will strike a balance between the scalability of the CMMI model and the depth of the peer review, which he described as extremely rigorous.

"I can tell you with a very high degree of certainty that in a peer audit where Verily and Pear Therapeutics … keep diving down until they essentially get to the laws of physics, there is very little chance that a product can fake it’s way through that process," he said.

Sanchez says his company, Johnson & Johnson, is the next to be appraised. Again due to security concerns, the company will forego the peer review. 

"Different is not worse"

The Pre-Cert program is starting to receive some pushback, notably in the form of a recent letter from a small group of senators, detailing a number of questions about the program.

"When I read that letter, I said ‘Yep, I asked that question too’," Look said. "I think those are all reasonable questions to ask and they are exactly the questions we as pilot participants are trying to help the FDA come up with good answers for. It’s a little bit like asking questions about alpha software. Yes, it is very rough around the edges, but we’re doing everything we can to help solidify and answer those questions. Iterating and doing that iteration out in the open makes it better."

But one thing panelists agreed upon was that the goal of the program was to make it easier for companies to get through the FDA process quickly and efficiently, not to make it easier to get FDA clearance.

"We are very much, very interested in ensuring public health by ensuring that that bar for safety and efficacy is not lowered," Amor said. "I think how you demonstrate that for a software product can be very different from how you demonstrate that for a catheter. A software application is dynamic, it’s always changing. But at no point have we ever talked about 'We have to lower the bar so that other software companies can get through'. It’s a very different framework, but that doesn’t mean worse. Different is not worse."

Amor said the best way to weigh in on the Pre-Cert process is through the public comment docket. The agency is listening, he said.

"Anecdotally I can share with you, I hear comments from the FDA saying ‘So and so company submitted this feedback, so we’re going to do this, this and this'," he said. "There are changes in the working model that were prompted by specific docket comments. Go online, find the docket and submit."