The Definitive Blog

Hospital Ratings: Flawed but Necessary Tools to Gauge Hospital Quality

Millions of prospective patients depend on hospital rankings to help them decide where they seek treatment, and they have a lot of systems to choose from. US News and World Report, The Leapfrog Group, The Joint Commission, and CMS all offer unique rating systems covering multiple aspects of hospital care. So what should a patient do when the ratings conflict, and to what extent do they accurately reflect the care available? Recent events have brought issues with hospital ratings to light, suggesting that agencies are still working to refine and perfect their grading systems.

One report highlighting the potential unreliability of hospital ratings appeared in the Boston Globe in mid July. It described the Manchester VA Medical Center, which recently received a four-star rating from the VA, as suffering from chronic mismanagement, dismal patient care, and poor staff morale. VA rates its hospitals under a system known as Strategic Analytics for Improvement and Learning (SAIL), which was formerly restricted to internal use but became public after hospital scores were leaked to the media. It isn’t as robust as the CMS star rating system, but includes several measures focusing on complications, readmissions, and infection rates. Despite the claims in the Globe report, Manchester VA Medical Center ranked in the top 10 percent among other VA facilities in several measures for Q2 2017, including readmission rates, hospital-acquired infections, and employee satisfaction. One explanation may be that the allegedly substandard care involved spinal care and the overall facility environment, measures not specifically included in the SAIL methodology. Another is that the system only grades each hospital on its performance compared to other VA facilities, as well as its improvement from last year, so it’s technically possible for a high-scoring VA hospital to perform worse than the average private facility.

Though the SAIL system is relatively new to the public, established ratings systems aren’t free from issues and can present a confusing picture to patients. A November 2016 study found that while about half of the over 100 Michigan hospitals sampled ranked highly on any one ratings system, only about 20 percent received an equally high score from another ratings agency, largely due to methodology differences. In a recent article, a Leapfrog Group official said the differences weren’t a problem for patients, suggesting that conflicting reviews presented a better overall picture of a product or service. Even so, a focus group in the study indicated that many respondents didn’t use rankings because they didn’t contain relevant information or were too hard to understand. While ratings systems grade hospitals differently, in some cases prioritizing measures that companies believe are more important, it seems unlikely that many patients would seek to thoroughly understand the specifics of each methodology, especially when they are under revision in the face of consumer and hospital criticism.

CMS attempted to address the problem of poor consumer usability with the introduction of its quality star rating program in 2016. While it used to only display quality scores on its Hospital Compare website, the star ratings were designed to consolidate the over 64 measures into a single score that would be much more easily accessible. The decision drew criticism from a variety of industry groups who claimed it unfairly penalized certain hospitals and didn’t compensate for patient populations. Currently, CMS has reportedly updated its methodology for the ratings since they were first issued, but has delayed the original semiannual release date from July to October due to data issues.

CMS overall quality star ratings 2016

*Source: Definitive Healthcare

Hospitals can and do have some legitimate concerns on the failings and inconsistencies among rating systems, but the overall push for transparency in healthcare means that they must learn how to adapt to them. Some hospitals looking to earn high ratings may try to tailor their efforts to a few select systems that best account for their individual clinical missions and patient populations, to the extent that it is possible. However, given the rate of revisions to any single methodology, a better plan may be to focus entirely on patient quality and experience measures most in need of improvement, and hope that it’s reflected in ratings agencies’ grades. Better to have superior quality that’s unreported, than an undeservedly high rating.

Definitive Healthcare has the most up-to-date, comprehensive and integrated data on over 7,700 hospitals, 1.4 million physicians, and numerous other healthcare providers. Our hospital profiles include detailed clinical and quality information, including infection rates, complication rates, and performance scores in a variety of federal programs.

Related Posts