In the quest to consumerize healthcare, online physician reviews are a popular attempt to return some agency and control to the healthcare consumer. In fact a 2015 survey showed that more than half of millennials consulted online reviews when choosing a physician. But are these reviews fair or reliable?

A new study from the Mayo Clinic, published in the health system's own journal Mayo Clinic Proceedings, suggests that negative reviews in particular may not accurately reflect a doctor's skill, and in fact are more likely to reflect institutional problems beyond a doctor's control.

“Our study highlights the disconnection between industry-vetted patient satisfaction scores and online review comments,” Dr. Sandhya Pruthi, senior author and an internal medicine physician at Mayo Clinic, said in a statement. “Patients need to be aware of these distinctions as they make decisions about their health. Physicians also need to be aware, as they manage their online reputations.”

The study looked at the online reviews of more than 2,000 Mayo Clinic physicians and found 113 with negative reviews written during a four-month period in 2014. This cohort was matched with a similar group of 113 physicians without negative reviews. Then researchers compared the Press Ganey patient satisfaction survey scores (specifically the parts of the survey that referred to the physician theirself) from the same period between the two groups. What they found is that physicians with negative reviews were no more or less likely to have poor Press Ganey scores than those without negative reviews.

But when researchers looked at the parts of the Press Ganey survey that dealt with outside factors like desk staff, nursing, physical environment, appointment access, waiting time, problem resolution, billing, and parking, they found a strong correlation with negative reviews.

The research, which has an admittedly small sample size, would seem to suggest that when people write negative online reviews of a physician, they are more likely to be reviewing factors outside the physician's control than actually assessing the competency of their doctor.

“These findings … underscore the totality and integrity of processes, elements and encounters – and not just the patient-provider interaction – that all need to be effectively and cohesively in place to ensure optimal patient experience and welfare,” Dr. Bradley Leibovich, a Mayo Clinic urologist, wrote in an editorial discussing the findings.

Liebovich suggests that the longterm solution for hospitals is to offer patients a way to access physicians' internal satisfaction scores, so that online reviews are no longer their only option for learning about doctors.