Lean methodology and patient safety

A recent story in U.S. News & World Report described how a Seattle hospital is taking a systems approach in improving healthcare quality and cutting costs. It said, "Virginia Mason Health System...has looked to adopt many of the much-admired and often-emulated business philosophies from Toyota."

The best-known of those philosophies is the so-called "lean methodology" which is based on eliminating waste and focusing on things that add value.

Attempts to incorporate lean into healthcare have met with varying degrees of success. I blogged about this six years ago and pointed out that a literature review done back then found "significant gaps in the [lean and six sigma] health care quality improvement literature and very weak evidence that [lean and six sigma] improve health care quality."

Randomized prospective trials of lean in medicine are lacking. A recent paper from the Journal of the American College of Radiology found only seven studies on the use of lean in radiology and they showed "high rates of systematic bias and imprecision." The authors concluded there was "a pressing need to conduct high quality studies in order to realize the true potential of these quality improvement methodologies [lean and six sigma] in healthcare and radiology."

In addition to the debatable evidence that lean actually works and the cost and time to develop and implement lean measures, the use of Toyota as a model for quality is also highly questionable.

In 2010, Toyota had recalled more than 9 million vehicles for various defects. Nothing has improved. So far this year Toyota has recalled over 11,654,000 vehicles. The problems included exploding airbags, brake failure, fuel tank defects, and minivan doors opening while cars were in motion.

Having adopted lean methodology in 2002, Virginia Mason is not really a new story. How is it doing?

About as well as Toyota.

In May of this year, the Joint Commission paid a surprise visit to Virginia Mason Medical Center and found 29 instances where the hospital was out of compliance with standards. The Seattle Times wrote that among the problems were not having an adequate infection prevention and control plan, failure to store medication safely, and failure to provide a "care, treatment, services and an environment that pose[d] no risk of an immediate threat to health or safety."

On September 17, Virginia Mason regained full Joint Commission accreditation status, and 6 weeks later the hospital announced that it received an "A" grade for patient safety from the Leapfrog Group.

A hospital that failed a Joint Commission site visit because of multiple safety issues gets an "A" for patient safety in the same year? I discussed problems with the Leapfrog patient safety rankings in a previous post.

And if lean works so well in healthcare, can anyone tell me how does a hospital that has been practicing lean methodology for 14 years achieve 29 Joint Commission citations?

Process vs. Outcomes Part II: New Head of Medicare and Medicaid

The New York Times reports that Donald Berwick, President Obama’s new appointee as head of Medicare and Medicaid, is coming under fire for some of his views such as his professed love of the British healthcare system. Something else caught my eye in that article. It says, “He has urged hospitals to emulate Toyota’s emphasis on efficiency and quality control…”

Most people who work in hospitals are familiar with the terms “Six Sigma” and “Lean.” Lean (L) was developed by Toyota and is a program of cost and waste reduction while maintaining quality. According to the folks at Motorola University, Six Sigma is a “metric*, a methodology and a management system.” Specifically, Six Sigma (SS) refers to six standard deviations from the mean or in the case of manufacturing, an attempt to achieve fewer than 3.4 defects for every one million opportunities.

Since 1999, these two tools have been adopted by many healthcare organizations. But despite more than 10 years’ experience, evidence of improved outcomes based on re-engineering of processes using SS/L is lacking according to a recent review of the healthcare literature on the subject.

The authors looked at 177 medical articles on SS/L published in the last 10 years. Only 34 papers reported any outcome data at all and of these, two-thirds did not provide any statistical analyses to justify their results. The methods used in these reports were judged to be of low scientific quality. Only three papers focused on clinical topics.

I asked the lead author of the review, Jami DelliFraine, whether she had expected to find favorable results in the literature on SS/L. She said, “I suspected that I would find a positive association between better processes/outcomes and the use of SS/L.” Thus, an author with a predisposition in favor of SS/L reported negative results in her review of the subject. I also questioned her about the cost of training personnel and she answered, “It is very expensive to implement these programs and train people for these programs.”

The proponents of SS/L must respond to the challenge and in the era of evidence-based medicine, produce some proof that the investment in time, money and personnel in SS/L will yield significant improvements in quality.

Finally, it is hard to resist a jab at SS/L via the Lean pioneers and Dr. Berwick’s example of what medicine should aspire to, Toyota, which has recalled some 9 million vehicles in the last 10 months. Using SS and taking the position that a defective vehicle represents one opportunity, Toyota would have to manufacture 3.4 billion consecutive flawless vehicles to achieve a rate of 3.4 defects per million. Guddorakku (Good luck).

*A metric is any number that you can put in a box in a computerized nursing record, and in turn be chewed over by a second layer of bureaucrats, resulting in a another number used to beat physicians over the head.

Bad News for Devotees of Process-Oriented Quality Assurance (Part 1)

The Surgical Care Improvement Project (SCIP) and its antecedent, the Surgical Infection Prevention project, have been around for several years. In short, these consist of several rules issued by various self-appointed agencies with important-sounding names and the Centers for Medicare and Medicaid Services (CMS), a federal agency. The main rules are (1) administer the correct prophylactic antibiotic before surgery, (2) give the antibiotic within one hour before the skin is incised and (3) discontinue the antibiotic within 24 hours of the end of the operation. The stated goal of these initiatives was to reduce the rate of surgical wound infections by 25% by the year 2010. It didn’t happen.

Surgeons and administrators have been obsessively following the rules and documenting their activities with squads of internal auditors (thought police). There are numerous papers showing a remarkable increase in the levels of compliance over the years to well over 90% in many hospitals. Compliance data have even been posted on line so that patients can compare institutions. CMS is thinking of linking hospital reimbursements to SCIP compliance rates. Unfortunately, a recent large study in the Journal of the American Medical Association (JAMA) has shown that SCIP has not only not decreased the rate of wound infections by 25%, it actually has had NO IMPACT at all on the infection rate.

So what happened? Why didn’t the rules work? They were based on some sound research. There are several theories. In order to comply with the “within one hour” rule, antibiotics are being given in the operating room and on many occasions, have not been completely infused as of the incision time. Thus, they will not have arrived at the wound in time to prevent the infection from occurring. Mary Hawn, MD, MPH, a surgeon and author of an editorial that accompanied the JAMA article, suggested that perhaps prophylactic antibiotics, which had been given for many years before the advent of SCIP, have already reduced the rate of infection as much as possible, and tweaking the timing may not make that much difference. She also pointed out that there are many other variables that influence the infection rate, such as the surgeon, the condition of the patient and type and duration of the procedure.

I asked Dr Hawn if SCIP should be changed or abandoned. She said, “SCIP is likely too narrow to have a meaningful effect on surgical outcomes. One response would be to add significantly more measures, but at that added burden one wonders if we really shouldn’t collect what we all care about – outcomes.” Of course, one reason that process metrics* are so popular is that processes are much easier to define and measure than outcomes. But would you as a patient rather choose a hospital that has a high rate of compliance with SCIP or a very low wound infection rate?

What we have here is the inevitable disconnect between process (the rules) and outcome (the infection rate). It’s not the first time, nor will it be the last. There will be more to come on the topic of process vs. outcome in future blogs.

*Metric: A metric is a measure for quantitatively assessing, controlling or selecting a person, process, event, or institution, along with the procedures to carry out measurements and the procedures for the interpretation of the assessment in the light of previous or comparable assessments. (Author’ note: Even the definition of a metric is convoluted. I promise I will not use the word “metric” again.)