How should residents spend their time?

As everyone knows, residents are now restricted to working 80 hours per week. One of the lesser known side effects of this work hours limitation is the drastic loss of educational conference time.

Since at least one third of the residents must now go home after morning rounds, afternoon conferences are no longer possible. Most residency programs now devote part of at least one morning per week to dedicated educational time.

A new paper from JAMA Surgery describes how one program chose to comply with the mandate to teach some of the more fuzzy resident core competencies. Their weekly didactic schedule of 1 hour of grand rounds and 1 hour of small group learning now includes 10 hours per year [representing 10% of the 100 hours allotted to formal teaching] on practice-based learning and improvement, interpersonal and communication skills, systems-based practice, and professionalism.

From the paper's Table 3, the specific topics are structure and policy of US health care, advocacy, medical economics and finance, history and consequences of major legislation, innovation in health care, health information technology, comparative effectiveness, health care disparities, basic management principles, quality, performance improvement, patient safety, coding and billing compliance, legal issues, litigation, risk management, clinical practice models, contracts, relative value units (RVUs), personal leadership styles, power and influence: organization psychology, negotiation and conflict resolution, communication, ethics, and last but certainly not least, one of my favorites—Six Sigma. [What, no mention of "Lean"?]

I don't mean to disparage the authors of this paper. They're only trying to follow the rules. I'm just glad I'm no longer a residency program director having to commit 10% of my program's precious educational time to things like organization psychology, history and consequences of major legislation, and Six freakin' Sigma.

But I guess it could be worse. At this year's meeting of the Association of Program Directors in Internal Medicine, the following slide suggesting books that should be read by every chief resident was shown.

I would love to meet the chief resident who had not only the time, but also the inclination to read all 17 of these books.

Patients Are Not Airplanes

As promised in my post “Surgeons Are Not Pilots,” today I will address the issue of whether patients can be compared to airplanes. Honestly, I cannot think of even one thing that patients have in common with airplanes.

As alluded to yesterday, probably the most glaring difference is that, unlike an airplane, each patient is unique. If a pilot sits at the controls of any Airbus A320 aircraft, he can be reasonably sure that pulling back the stick a certain amount will result in a very consistent response from the plane. Therefore, practicing on a simulator will enable the pilot to prepare for any emergency with the knowledge that what he did on the simulator will in fact be reproducible in a real emergency.

Contrast that with a patient. Often patients with similar illnesses will behave very differently because human beings are not engineered like airplanes. For example, let’s say I am performing a difficult laparoscopic cholecystectomy (removal of the gallbladder) and I am having trouble locating the cystic artery (artery to the gallbladder). I know that the anatomy of the cystic artery is highly variable. This link illustrates 11 of the most common anatomic variations in the location of that vessel. This means that there is not a simple maneuver that will help me find the artery in every case.

A pilot can be confident that a 5% increase in power will result in a very predictable response in airspeed. Contrast that with a patient’s response to a medication. I have had patients fall asleep with an intravenous injection of only 2 mg of morphine and I’ve had other patients who loudly complain of persistent pain after receiving 10 mg of the same drug.

A word about simulators. Because all stimuli are external to a fixed object, the pilot, aircraft simulators are easy to design and are very realistic. They recreate the motion, sounds and visuals of flying very well. When I was in the Navy stationed on an aircraft carrier, I heard pilots talk about their experiences in simulators. They said the intensity of the experience was very similar to that of flying a real plane. While surgical simulators have become more sophisticated, they still lack the realism of aircraft simulators because the action is all occurring on a video screen and the surgeon is sitting in a lab somewhere. There is no way that a surgical simulator can give you the adrenaline rush and anal sphincter-puckering feeling of seeing a sudden squirt of blood that covers your laparoscope and totally obscures your vision during a tough case. And there are no realistic surgical simulators for open (non-laparoscopic) cases. Operating on a pig just doesn’t cut it. (Pun intended.)

So please, stop trying to compare patients to airplanes.

A postscript on yesterday’s “Surgeons Are Not Pilots” blog.

Captain Chesley Sullenberger, the appropriately acclaimed hero of the successful Hudson River landing of US Airways Flight 1549, is hardly a typical pilot. According to one biography, he had over 27,000 hours of flying experience with more than 19,000 of those hours in Airbus A320s or like aircraft. He also had written about air safety and worked as an NTSB investigator. The hours alone qualified him as an “expert” pilot.

Compare Sully’s background with that of the pilots of Colgan Air Flight 3407, which crashed in Buffalo two years ago, killing 50 people. The following is from the Wikipedia entry about this incident, “The crew of four was led by Captain Marvin Renslow … who was hired by Colgan in 2005 and had flown 3,379 hours. 261 of these hours were on the Dash-8 Q400 (including 109 as a captain). First Officer Rebecca Lynne Shaw … was hired by Colgan in January 2008, and had flown 2,200 hours, 772 of them on the Q400.” The cause of the fatal accident was several pilot errors and failures to follow protocols.

All the checklists in the world didn’t prevent that crash. As long as humans are going to perform surgery on other humans, bad outcomes will occur. Yes, we should endeavor to minimize errors as much as possible, but “zero defects” and Six Sigma are not possible in medicine (or even in manufacturing).

How Does Science Daily Pick Its Subject Matter?

Ever wonder how Science Daily chooses which research articles to feature on their website? I have. As a total skeptic regarding the applicability of Six Sigma to medicine [see my previous blogs about Six Sigma], a recent post on Science Daily caught my eye. This was a glowing report entitled “Reducing waiting time at an emergency department using design for Six Sigma and discrete event simulation.” It’s about an emergency department in Jordan whose patient waiting times and lengths of stay [LOS] purportedly have been significantly reduced thanks to the miracle that is Six Sigma. The paper appeared in a journal called the International Journal of Six Sigma and Competitive Advantage.

I obtained a copy of the complete paper and have the following observations:

1. The authors already knew the waiting times and LOS were long.
2. They developed a mathematical model based on patient surveys to prove what they already knew.
3. The waiting time data were based on “…a random sample of 96 patients…measured over two random months and shifts.” The LOS data were from “…a random sample of 67 patients…measured over two random months and shifts.” The method of randomization was not stated. The number of patients studied represents 0.075% of all patients seen per year in their ED.
4. There are complex tables and flow charts.
5. Verification that the process was improved was based solely on simulation, not actual performance.
6 The impact factor of the International Journal of Six Sigma and Competitive Advantage is just about zero.
7. Don’t look for this article in PubMed.

So, you might ask, why was this paper featured on Science Daily? This is important because a posting on Science Daily is read by far more people than those who read most journals. Most of the information posted on Science Daily is from press releases generated by the authors of papers or their institutions. When describing how to contribute material, Science Daily states "Please note that we cannot guarantee posting of all the releases we receive, since we try to select those which we think would be of most interest to our readers." Someone from Science Daily is choosing what to post and we don't know how or why.

An email, the only way to contact Science Daily, sent on 8/31/10 asking about this has yet to be answered.

Process vs. Outcomes Part II: New Head of Medicare and Medicaid

The New York Times reports that Donald Berwick, President Obama’s new appointee as head of Medicare and Medicaid, is coming under fire for some of his views such as his professed love of the British healthcare system. Something else caught my eye in that article. It says, “He has urged hospitals to emulate Toyota’s emphasis on efficiency and quality control…”

Most people who work in hospitals are familiar with the terms “Six Sigma” and “Lean.” Lean (L) was developed by Toyota and is a program of cost and waste reduction while maintaining quality. According to the folks at Motorola University, Six Sigma is a “metric*, a methodology and a management system.” Specifically, Six Sigma (SS) refers to six standard deviations from the mean or in the case of manufacturing, an attempt to achieve fewer than 3.4 defects for every one million opportunities.

Since 1999, these two tools have been adopted by many healthcare organizations. But despite more than 10 years’ experience, evidence of improved outcomes based on re-engineering of processes using SS/L is lacking according to a recent review of the healthcare literature on the subject.

The authors looked at 177 medical articles on SS/L published in the last 10 years. Only 34 papers reported any outcome data at all and of these, two-thirds did not provide any statistical analyses to justify their results. The methods used in these reports were judged to be of low scientific quality. Only three papers focused on clinical topics.

I asked the lead author of the review, Jami DelliFraine, whether she had expected to find favorable results in the literature on SS/L. She said, “I suspected that I would find a positive association between better processes/outcomes and the use of SS/L.” Thus, an author with a predisposition in favor of SS/L reported negative results in her review of the subject. I also questioned her about the cost of training personnel and she answered, “It is very expensive to implement these programs and train people for these programs.”

The proponents of SS/L must respond to the challenge and in the era of evidence-based medicine, produce some proof that the investment in time, money and personnel in SS/L will yield significant improvements in quality.

Finally, it is hard to resist a jab at SS/L via the Lean pioneers and Dr. Berwick’s example of what medicine should aspire to, Toyota, which has recalled some 9 million vehicles in the last 10 months. Using SS and taking the position that a defective vehicle represents one opportunity, Toyota would have to manufacture 3.4 billion consecutive flawless vehicles to achieve a rate of 3.4 defects per million. Guddorakku (Good luck).

*A metric is any number that you can put in a box in a computerized nursing record, and in turn be chewed over by a second layer of bureaucrats, resulting in a another number used to beat physicians over the head.

Bad News for Devotees of Process-Oriented Quality Assurance (Part 1)

The Surgical Care Improvement Project (SCIP) and its antecedent, the Surgical Infection Prevention project, have been around for several years. In short, these consist of several rules issued by various self-appointed agencies with important-sounding names and the Centers for Medicare and Medicaid Services (CMS), a federal agency. The main rules are (1) administer the correct prophylactic antibiotic before surgery, (2) give the antibiotic within one hour before the skin is incised and (3) discontinue the antibiotic within 24 hours of the end of the operation. The stated goal of these initiatives was to reduce the rate of surgical wound infections by 25% by the year 2010. It didn’t happen.

Surgeons and administrators have been obsessively following the rules and documenting their activities with squads of internal auditors (thought police). There are numerous papers showing a remarkable increase in the levels of compliance over the years to well over 90% in many hospitals. Compliance data have even been posted on line so that patients can compare institutions. CMS is thinking of linking hospital reimbursements to SCIP compliance rates. Unfortunately, a recent large study in the Journal of the American Medical Association (JAMA) has shown that SCIP has not only not decreased the rate of wound infections by 25%, it actually has had NO IMPACT at all on the infection rate.

So what happened? Why didn’t the rules work? They were based on some sound research. There are several theories. In order to comply with the “within one hour” rule, antibiotics are being given in the operating room and on many occasions, have not been completely infused as of the incision time. Thus, they will not have arrived at the wound in time to prevent the infection from occurring. Mary Hawn, MD, MPH, a surgeon and author of an editorial that accompanied the JAMA article, suggested that perhaps prophylactic antibiotics, which had been given for many years before the advent of SCIP, have already reduced the rate of infection as much as possible, and tweaking the timing may not make that much difference. She also pointed out that there are many other variables that influence the infection rate, such as the surgeon, the condition of the patient and type and duration of the procedure.

I asked Dr Hawn if SCIP should be changed or abandoned. She said, “SCIP is likely too narrow to have a meaningful effect on surgical outcomes. One response would be to add significantly more measures, but at that added burden one wonders if we really shouldn’t collect what we all care about – outcomes.” Of course, one reason that process metrics* are so popular is that processes are much easier to define and measure than outcomes. But would you as a patient rather choose a hospital that has a high rate of compliance with SCIP or a very low wound infection rate?

What we have here is the inevitable disconnect between process (the rules) and outcome (the infection rate). It’s not the first time, nor will it be the last. There will be more to come on the topic of process vs. outcome in future blogs.

*Metric: A metric is a measure for quantitatively assessing, controlling or selecting a person, process, event, or institution, along with the procedures to carry out measurements and the procedures for the interpretation of the assessment in the light of previous or comparable assessments. (Author’ note: Even the definition of a metric is convoluted. I promise I will not use the word “metric” again.)