Physician satisfaction scores on online third-party review sites tend to be skewed and can easily mislead patients, according to a new study by Cedars-Sinai investigators.

This distortion may have significant consequences as consumers rely increasingly on these scores when choosing a physician. Research shows that patients largely trust these ratings as the sole source of information when choosing a physician.

“Patients put so much trust into ratings, and the stakes are much higher than simply choosing a restaurant,” says lead author Timothy J. Daskivich, MD, MSHPM, assistant professor and director of Health Services Research in the Department of Surgery at Cedars-Sinai. “It’s important to interpret this data correctly because selecting the right physician can have a serious impact on health and wellbeing.”

Daskivich and his co-authors pulled reviews from October 2014 to March 2017 on Healthgrades, a consumer ratings website that ranks medical providers from 1-5 stars. The investigators linked this data to providers listed in the U.S. Centers for Medicare & Medicaid Services’ Physician Compare tool. They narrowed the field to 212,933 providers, who had at least four reviews evaluating overall patient satisfaction. They grouped the providers by medical, surgical and allied health specialties, and performed a statistical analysis to examine the distribution of the providers’ average satisfaction scores.

The study, published in the Journal of Medical Internet Research, found that overall satisfaction ratings consistently skewed positively, fell within narrow ranges and had different distributions across specialties. As a result, scores that appear high might actually be comparatively average or low, effectively misleading patients. For example, if 90 percent of physicians in a particular specialty are rated higher than four stars, patients could be misled into thinking the physician they select is at the top of their field.

Providers’ satisfaction ratings also differed significantly by specialty group. Median scores for allied health providers (physical therapists, optometrists) were much higher than those of physicians in medical and surgical specialties.

Some of the differences in rankings between specialties might be related to the nature of the work. That may explain why psychiatrists, who field emotional trauma, get lower ratings than chiropractors, who provide physical relief and lots of one-on-one interaction, Daskivich says. But he and his co-authors think review sites should flag such statistical quirks and caveats.

The study comes at a time when consumers increasingly are turning to third-party websites to post reviews of physicians and comments about a wide variety of healthcare services and experiences. Although there has been a proliferation of these third-party sites (including Healthgrades, Zocdoc and Yelp), they often present information based on a small number of reviews and incomplete or unverified information. As a result, many health systems (such as Stanford, Cleveland Clinic and University of Utah) have begun posting more complete ratings and comments from their own outpatient satisfaction surveys.

These hospital-driven tools measure satisfaction with providers for attributes such as communication, friendliness and time spent with patients. The tools do not measure healthcare quality. Healthcare providers are giving consumers access to this information to offer greater transparency and visibility into feedback from real patients.

The authors of the ratings study suggest that third-party online review sites can do better by posting median star ratings for each medical provider and noting where they rank among peers in their specialty. The investigators also note in the study that they created an online tool, Compare My Doc, which uses a provider’s specialty and online rating to show how that provider compares to their colleagues.

“Scores from third-party sites aren’t going away,” says senior author Brennan Spiegel, MD, director of Cedars-Sinai Health Services Research. “These sites have a responsibility to tell consumers that these reviews can be skewed and to provide more guidance around interpreting them.”