In new study published in Ophthalmology,  Google AI researchers found that they could improve their AI disease detecting software by using a small subset of images adjudicated by ophthalmologists that specialize in retinal diseases.

"We believe this work provides a basis for further research and raises the bar for reference standards in the field of applying machine learning to medicine," Dr. Lily Peng, an author in the study and a researcher at Google AI, said in a statement.

In the study, images were graded by the algorithm, US board-certified ophthalmologists, and retinal specialists. The consensus of the retinal specialists served as the reference standard to train the algorithm. 

After the AI algorithm was trained using the adjudicated subset it had a sensitivity of .97 and a specificity of .92 on the quadratic-weighted kappa score when looking at moderate or worse diabetic retinopathy. In contrast, the majority decision of ophthalmologists had a sensitivity of .838 and specificity of .981. 

Between this and other image analysis comparisons, researchers concluded that the AI’s performance generally improved after it was trained on a small set of adjudicated diabetic retinopathy grades. Researchers also noted that the algorithm’s performance was in line with board-certified ophthalmologists and retinal specialists.

"For my grandma, I would love to have a panel of subspecialists who actually treat the disease, to sit and debate her case, giving their opinion," Dr. Peng said in a statemenet. "But that is really expensive and it's hard to do. So how do you build an algorithm that gets close to this?"

AI-detected diabetic retinopathy has been an ongoing conversation. 

Researchers at the Medras Diabetes Research foundation in Chennai India, for example, found that a smartphone-based device called Remido ‘Fundus on phone’, along with artificial intelligence software, had a high sensitivity for detecting diabetic retinopathy. In fact, the study, published in Nature Eye, found the AI software had a 95 percent sensitivity and 80 percent specificity for detecting any diabetic retinopathy.