Wednesday, March 16, 2016

Why hospital rankings are bogus

At the end of 2015, The Leapfrog Group announced its annual list of America’s top hospitals for quality and safety; 98 hospitals receiving the honor.

Unlike some other hospital rating schemes, Leapfrog’s does not factor in reputation. You won’t find any of the usual suspects on Leapfrog’s list. Instead, Leapfrog uses surveys of hospitals and publicly available quality and safety data.

Leapfrog’s top 98 included 62 urban, 24 rural, and 12 children’s hospitals. Of the 86 urban and rural hospitals, only three were university hospitals—University of California Davis Medical Center, University of California Irvine Medical Center, and University of Tennessee Medical Center.

New York managed to place only one hospital on the Leapfrog list.

Other interesting anomalies are that for several states such as Connecticut, Indiana, and Maryland, no hospitals made the list, and of the 21 California hospitals that did, 17 are Kaiser-affiliated. Looks like Kaiser knows how to play the game.

Here’s where it gets interesting. Seven of the urban and rural hospitals also appear on Medicare's list of 758 hospitals slated to receive a 1% reduction in payments in 2016 because of “high rates of potentially avoidable infections and complications such as blood clots, bed sores and falls.” Five of the seven—Kaiser Foundation Hospital Antioch and Oakland/Richmond in California, Pennsylvania’s Geisinger Medical Center, Parkland Health and Hospital System in Texas, and Sterling Regional Medcenter in Colorado—were also penalized by Medicare in 2015. The two other hospitals on both lists are Reading Hospital in Pennsylvania and Midwestern Regional Medical Center in Illinois.

How can those seven hospitals be among Leapfrog’s top 98 in the country for patient quality and safety and be sanctioned by Medicare for safety problems?

I asked Leah Binder, President and CEO of The Leapfrog Group, to comment. She said, "The HAC [Medicare] Reduction Program considers a composite of safety measures, weighted heavily toward infection measures, and looks at different time periods." But she stated that the data hospitals report to Leapfrog are crosschecked "against publicly available sources such as CMS."

Other concerns about the Leapfrog rating system are that surveys were submitted by only about 1700 of the more than 5000 hospitals in the United States. According to Ms. Binder about 20% of the data submitted by hospitals to Leapfrog are self-reported which can be unreliable. That may help explain why some hospitals were both ranked highly by Leapfrog and disciplined by Medicare.

Oh, that one hospital in New York that made Leapfrog's top hospitals list? It’s Harlem Hospital. In 2012, Consumer Reports ranked it one of the worst in the US for patient safety, readmissions, and infections—yet it is one of Leapfrog's top 98 hospitals.

Have things changed since 2012? Here is a Healthcare IT News article about CMS (Medicare) updating its star rankings for hospitals in July of 2015. If you type “Harlem Hospital” in the search field, you will find that it received 1 star of a possible 5.

From the Healthcare IT News third paragraph: “Based on user answers to the Hospital Consumer Assessment of Healthcare Providers and Systems, these quality ratings by the CMS in many ways carry more weight than findings from the likes of Leapfrog [emphasis mine], Healthgrades, Truven and US News because they are often tied to value-based reimbursement levels in Medicare.”

When asked about whether she would choose Harlem Hospital for her own care, Ms. Binder said, "I would consider it depending on what services I needed, but I don't live in New York so I’d probably go somewhere more local."

Draw your own conclusions.

This post appeared on 12/15/15 on Physician’s Weekly. It has been partially rewritten and updated with new information.

5 comments:

Michel Accad said...

Excellent. Indeed, those at the top of the rankings shine at producing metrics, though not necessarily at providing care. Another example of Campbell's law. https://en.wikipedia.org/wiki/Campbell%27s_law

Rocky Bilhartz said...

One of the hospital systems that I'm not affiliated with, but know well, in my area (Texas), has one of its tiny "spoke" hospitals on the list of 98.

Of course, none of its "mothership" hospitals are listed (which are the places that actually provide the advanced services one might really desire in a time of need).

It seems obvious to me that the current information provided by Leapfrog does nothing of the sort for educating the public on where they should actually go in a time of need.

I try hard not to fault "legitimate" effort most of the time, but I would propose that (in its current state) rankings such as Leapfrog's may actually be detrimental/harmful to certain patients who choose to use the information to select a hospital.

Help us all if another third-party industry is soon to be on the brink of establishment... one that evaluates the "quality" of the "quality" rankings.

Skeptical Scalpel said...

Michael, thanks for commenting. For those too lazy to copy and paste the link, here's the definition of Campbell's law. "When a quantitative social indicator (or even some qualitative indicator) is used for social decision-making, the more subject it will be to corruption pressures and the more apt it will be to distort and corrupt the social processes it is intended to monitor."

Rocky, thanks for the information. Good point about the potential for patient harm. Luckily I think most people in the real world Have never heard about things like The Leapfrog Group or the Surgeon Scorecard.

Anonymous said...

Kaiser is terrific from my point of view. My family is replete with MDs (I'm a retired computer scientist) and HMO's were anathema in family discussions. But finally my wife went to Kaiser and eventually I tried it, too. It's incredibly efficient. One time I had a small growth on my nose and the GP (or gateway physician, or whatever) said I should consult a dermatologist. He got on the phone and ten minutes later the dermatologist came to me! I don't know if the GP or the dermatologist know the first thing about medicine but the efficiency is so welcome. Even with my anesthesiologist father running interference, medical practice in my youth involved interminable waits and visits to separated clinics. My ancient mother is now in Florida and health care there is like something out of Kafka.

Skeptical Scalpel said...

Anon, thanks for commenting. I guess if efficiency is your primary concern, you have hit the jackpot.

Post a Comment