Analysts: CMS Hospital Rankings Fail ‘Common Sense Test’

System rates organizations on seven quality domains

The Centers for Medicare and Medicaid Services (CMS) now rates almost all hospitals with between one and five stars (higher is better) depending on their performance on a series of quality metrics. However, patients and families will find the star ratings confusing at best and misleading at worst, according to a Health Affairs blog.

The star system was designed to rank all hospitals based on 64 quality measures from seven quality domains and to make domain “weight” reflect the relative importance of a quality domain to patients. The CMS has assigned greater weight (66% of the overall star rating) to three domains: mortality, readmission, and patient safety. Other domains that focus on processes, such as the efficient use of medical imaging and the timeliness of care, are given less weight.

Nevertheless, the CMS calculated and published star ratings for hospitals that reported on as few as three quality domains, including some hospitals that had data from only one clinical outcome domain, according to the authors. The fewer the clinical outcome domains a hospital reports, the less that hospital’s overall star rating is actually tied to performance on patient outcomes.

Based on data released by the CMS in July, 40% of the 102 hospitals that received a five-star rating did not have the minimum data necessary to report on either mortality or readmissions, the authors point out.

In their article, they cite two Midwestern hospitals as examples of what’s wrong with the CMS rating system. “In the quality domains for which both hospitals’ data were available, Hospital A’s performance was either similar to or better than that of Hospital B,” they write. “In the quality domains for which Hospital B could not provide adequate data, Hospital A was either above or at the national average. Yet, Hospital A received a four-star rating, while Hospital B was awarded five stars.”

According to the authors, the disconnect between hospitals’ star ratings and their actual performance is a result of the fact that when a hospital has insufficient data to report on one or more quality domains, the weights of those missing domains are reallocated to the domains for which there is sufficient data. In the case of Hospital B, weights of the four missing quality domains were simply allocated to the three domains that were available. After reweighting, the one and only clinical outcome domain (in this case, safety of care) accounted for less than half of Hospital B’s overall star rating.

“This is simply inconsistent with the rating system’s original intention of making performance on clinical outcomes the predominant influence on the overall rating,” the authors write.

Eighty percent of major teaching hospitals reported on all seven quality domains. To receive ratings with more stars, these hospitals have to meet a higher standard than hospitals with fewer reported quality domains because of their narrower service areas and less-diverse patient populations, the authors point out. Not only do major teaching hospitals need to achieve performance better than the national average in more quality domains, their overall star ratings will also be heavily tied to the outcomes of their clinical services (e.g., mortality) instead of processes of care delivery (e.g., efficient use of medical imaging).

Many major teaching hospitals also serve large populations of patients who live in poverty and economically deprived neighborhoods, and numerous studies have linked low socioeconomic status (SES) to an increased risk of readmission after discharge. Nevertheless, the readmission risk associated with patients’ SES is not currently accounted for in quality domains, such as readmissions, the authors note. In their opinion, the fact that 70% of the major teaching hospitals with the highest share of low-SES patients (the top quartile) received one or two stars reflects the systematic bias in the ratings system.

The authors conclude: “To provide meaningful information for patients, families, and caregivers about hospital quality, a star ratings system has to make sense. At a minimum, quality performance among hospitals with the same star rating should be consistent. And higher star ratings should reflect better actual quality performance. Unfortunately, the CMS star ratings in their current form fail to meet this basic test and will do more harm than good to patients.”

Source: Health Affairs Blog; November 14, 2016.