Online and mobile symptom checkers like WebMD, AskMD, iTriage, and even offerings from the Mayo Clinic and the UK’s NHS, come up with an accurate diagnosis only about a third of the time, according to a study by Harvard researchers recently published in the British Medical Journal.
Researchers looked at 23 free English-language symptom checkers that either offered patients a diagnosis; triaged a condition into emergency care, non-emergency care, or home-care; or did both. Researchers found that 11 symptom checkers offered both diagnoses and triage advice, eight were diagnoses only, and four provided triage advice only. They ran 45 standardized patient vignettes through each symptom checker and evaluated whether the correct diagnosis was presented as the top choice, in the top three, or in the top 20. They also evaluated whether they triaged the conditions correctly.
On average, symptom checkers listed the correct diagnosis first just 34 percent of the time. They had it listed in the top three 51 percent of the time and listed it in the top 20 just 58 percent of the time. Conditions were triaged correctly 57 percent of the time on average, but the symptom checkers erred on the side of sending patients to the emergency room: 80 percent of vignettes requiring emergency visits were triaged correctly, compared to 55 percent requiring non-emergency care and 33 percent requiring self-care.
“Whether this level of performance for diagnosis and triage we observed is acceptable depends on the standard for comparison,” researchers wrote. “If symptom checkers are seen as a replacement for seeing a physician, they are likely an inferior alternative. It is believed that physicians have a diagnostic accuracy rate of 85 to 90 percent, though in some studies using clinical vignettes, performance was lower. However, in-person physician visits might be the wrong comparison because patients are likely not using symptom checkers to obtain a definitive diagnosis but for quick and accessible guidance.” Keep reading>>