New Leapfrog Hospital Safety Scores: Better, Worse, and the Same.

leapfrog-map-nov2012-croppedIn June 2012, the Leapfrog group released the first iteration of its Hospital Safety Scores. Drawing heavily on information collected by the federal government for the Medicare program, and supplemented by additional information submitted by hospitals voluntarily to the Leapfrog Hospital Survey, the Safety Scores attempt to highlight aspects of hospital operations that are indicative of safety as contrasted to elements relating to quality. Obviously a distinction between safety and quality is very fuzzy indeed, but it is my impression that the Safety Scores attempt to focus on preventable occurrences in hospital that cause harm. The information underlying the Safety Scores represent a combination of procedures in place for prevention as well as a count of selected actual adverse events. In November 2012, an updated set of Hospital Safety Scores was released.

The first iteration drew considerable criticism from hospitals that did not do well as well as from the hospital industry lobby. Much of the criticism was in the nature of unsupported excuses, but there are legitimate concerns inherent in any new system or the validity of the data underlying the scores. Because in the new world of healthcare transparency and accountability measurements of hospital quality and safety are not going to go away, like it or not, most hospitals will try to rank better in ratings of both safety and quality. The results for Kentucky’s hospitals were presented in more detail in the KHPI Policy Blog.

In the November release, 54 individual Kentucky hospitals or hospital systems were ranked. Scores were again awarded as letter grades. For Kentucky’s hospitals, there were 12- ‘A’s, 20- ‘B’s, 21- ‘C’s, 1- ‘D’, and no ‘F’s. Inasmuch as Kentucky has 65 acute-care hospitals eligible for Medicare payments and many more other kinds of hospitals, it is obvious that many are unranked. Parkway Regional Hospital in Fulton which received a score in June, was not scored in November. However, the November list included three new hospitals not previously scored: St. Clair Regional Medical Center, Williamson ARH Hospital, and Hazard ARH Regional Medical Center. As an indication of the rapidly changing hospital industry, three hospitals on the lists had a change in name.

Some hospital systems, such as Jewish Hospital and St. Mary’s Healthcare, have their facilities lumped together under a single Medicare provider number and choose to receive only a single score. Since it is a leap of faith to assume that all facilities in a hospital system are equally safe, the public may not be well served by this practice. In the June release, Norton Hospital received a single combined Safety Score for its four individual Louisville hospitals. In the November release, the four hospitals have been broken out individually.

Hospitals that appeared in both lists frequently had a change in grade. Of the 47 discrete hospitals appearing in both lists; 25 received the same score both times, 12 improved by one grade, and three improved by two grades. Seven hospitals actually dropped a letter grade. A decrease in safety score does not necessarily mean there was an increase in adverse events but that other competing hospitals might have done so much better. Hospitals are not treading water, but constantly seeking to improve. Those that fall behind the rest will receive a worse grade even if nothing in their operations changes. Where would you rather go?

As was observed nationally, some of the improvement in scores was dramatic. Three Kentucky hospitals actually improved by at least two letter grades: Highlands Regional Medical Center (C to A), St. Elizabeth Florence (C to A), and University of Louisville Hospital (from failing to B). The University of Kentucky Hospital raised its letter grade by at least one from a “failing grade” to a C.

Many were surprised last June at the poor showing for the hospitals of Louisville. Despite big time chest-beating, the highest grade awarded was a C. We do a little better in the newest ranking. The University of Louisville increased its score from the lowest in the state (and one of the very lowest in the nation) to one of the two highest in the city. It shares that honor with Baptist Hospital East which raised its score from a C to a B. The combined and individual Norton hospitals, and the Jewish Hospital and St. Mary’s Healthcare system received ‘C’s both times.

If such dramatic improvements in scores (and presumably safety) are possible, why are we not seeing actual improvement reflected nationwide? How much of the national volatility in scores is simply a matter of bookkeeping, or even worse, gaming of the system or frank falsification? How much difference does it make when a hospital participates in Leapfrog’s own proprietary voluntary reporting system? The University of Louisville Hospital is now enrolled with Leapfrog and I commend its medical staff and management for that. Only a few Kentucky hospitals have been willing to hang out their laundry in that regard.

How some hospitals in Kentucky and elsewhere were able to so dramatically improve their scores in five short months begs to be explained.  I will attempt to do so in future analyses. Without such understanding, our confidence that current methods of evaluating safety and quality are reliably measuring what we expect of them is substantially undermined.

A complete list of Kentucky hospitals that received scores in either June or November can be downloaded here.

Peter Hasselbacher, MD
President, KHPI
Emeritus Professor of Medicine, UofL
8 January 2013

One thought on “New Leapfrog Hospital Safety Scores: Better, Worse, and the Same.”

  1. Top Hospitals for 2012
    Shortly after announcing its November list of updated Hospital Safety Scores, the Leapfrog Group published its list of Top Hospitals for 2012. The list includes 67 urban hospitals, 13 rural hospitals, and 12 children’s hospitals.

    To be included in the Top Hospital list required that the hospital receive a Safety Score of A, but also included the results of a more extensive evaluation of operational systems, quality, and performance outcomes. Alas, no Kentucky hospital made the list.

    Louisville has staked its economic development future on the healthcare industry, and yet, despite self-promotional claims of “world-class” hospitals, it is rare that outside observers find either the quality or value of our local hospitals to be noteworthy. If we do not expect more and ask for it, little will change.

    Of interest is how many hospitals on the list were part of coordinated hospital systems. For example, 17 of the top urban hospitals were part of the Kaiser Permanente system. What are they doing that we are not? A number of prominent and even not so prominent University and other teaching hospitals are on the list. Why are ours not? The University of Louisville Hospital managed a Safety Score of B on the last go-around. That is a good start and intimates an openness to do better. However, the Hospital’s quality, outcome, and efficiency measures have a long, long way to go.

    You can see the Leapfrog Top Hospitals List 2012 here.

    Peter Hasselbacher, MD

Comments are closed.