Silicon Valley tech giant NVIDIA, which is best known as a hardware company for graphic processing units, is one of the latest companies to look into the telemedicine space.
Researchers at the company are working on an automated speech recognition and natural language processing technology that can transcribe and organize information from a telemedicine visit both for the patient and clinicians. The tool is specifically trained to understand clinical and biomedical language. The application framework, dubbed Jarvis, is not yet on the market but is available for early access.
Yesterday research data about the tool was presented at the Conference on Machine Intelligence in Medical Imaging. The data demonstrated promising results about the tech's ability to accurately automate a speech recognition system that captures clinical names.
While the initial research has been focused on transcribing for telemedicine, in the future there could be other medical applications for the technology.
"This particular one was trained on large biomedical text and then a smaller set of clinical text ... but if we had a large amount of clinical notes like radiologist reports then the language processing trained on them would be kick-ass on recognizing radiologists reports" Hoo-Chang Shin, research scientist at NVIDIA, told MobiHealthNews. "It would be so much better at transcribing radiologist reports and also we have even more recognition of named entities or disease names, or possibly planned treatments, and so on."
WHY IT MATTERS
Researchers at NVIDIA are pitching this as a way to help patients leave telemedicine visits with more information and notes.
Shin, who served as the lead researcher on the project, gave the example, of using telemedicine when his three-year old daughter got a potentially dangerous bug bite.
"The doctor wanted me to show [my daughter's] leg on the web. I was holding my webcam with one hand and with the other hand I'm holding my daughter's swollen leg," Shin said. "As he was seeing my daughter's leg he was saying what could have possibly happened –what kind of bug might have bitten her, what kind of drug to apply to her infected area, and what kinds of drugs she needs to take to prevent further swelling. It went so fast. My two hands were tied with the webcam and my daughter's leg so I couldn't write it down."
While Shin comes from a tech savvy background, Dr. Mona Flores, Global Head of Medical AI at NVIDIA, argued that this tool could be most effective for patients who have less of a background in tech.
"You can imagine an 80-year old trying to do the same and how difficult it would be for them to gather everything that they need, and to document, and remember what happened, and can go and ask questions or share what they learned from their doctor with a loved one or the family," she told MobiHealthNews. "You can image a platform like this is very useful the more technically challenged you are ... You don't have to worry about taking notes. Whether you are the physician, or patient for that matter, you can come back to a faithful transcription of what's happened and be able to ingest it at your own pace."
THE LARGER TREND
This is hardly NVIDIA's first time in the healthcare space. Much of its efforts have centered on radiology. In 2018, it released its AI platform, called Clara, that uses AI to create a virtual medical imaging platform. Then in 2019 it launched the Clara AI toolkit for radiologists, which included 13 classifications, and also segmentation AI and software tools.
It has also dipped into the genomics space. In October of 2018 it teamed up with Scripps Research Translational Institute on an AI and deep learning tool to help analyze genomic and digital health sensor data.
