Artificial intelligence deep learning models can be trained to predict self-reported race from imaging results, raising concerns about worsening health disparities, according to a study published in The Lancet Digital Health.

Researchers found models could detect race from different types of chest imaging results, including X-rays, CT scans and mammograms. The ability couldn't be traced back to disease distribution, where one condition is more prevalent among certain groups, or anatomic characteristics.

The study also found the deep learning model could still predict race even when using low-quality images, to the point where a model trained on high-pass filtered images could perform when human radiologists couldn't determine whether the image was an X-ray at all.

"To conclude, our study showed that medical AI systems can easily learn to recognise self-reported racial identity from medical images, and that this capability is extremely difficult to isolate. We found that patient racial identity was readily learnable from medical imaging data alone, and could be generalized to external environments and across multiple imaging modalities," the study's authors wrote. 

"We strongly recommend that all developers, regulators and users who are involved in medical image analysis consider the use of deep learning models with extreme caution as such information could be misused to perpetuate or even worsen the well documented racial disparities that exist in medical practice."

WHY IT MATTERS

Researchers wrote that the persistence of the models' abilities shows that it could be difficult to control the behavior when necessary, and the issue needs to be studied further. Since human radiologists can't usually determine race from imaging results, they wouldn't be able to provide oversight for the models and potentially mitigate any problems that arise. 

"The results from our study emphasize that the ability of AI deep learning models to predict self-reported race is itself not the issue of importance. However, our finding that AI can accurately predict self-reported race, even from corrupted, cropped and noised medical images, often when clinical experts cannot, creates an enormous risk for all model deployments in medical imaging," researchers wrote.

THE LARGER TREND

As AI expands into more areas in healthcare and life sciences, experts have raised concerns about the potential to perpetuate and worsen racial health disparities. 

According to a study published last week in the Journal of the American Medical Informatics Association, finding bias in AI and machine learning requires a holistic approach that requires multiple perspectives to address, as models that perform well for one group of people could fail for other groups.