No improvement in complication rates after instituting an operating room checklist

A before and after study at the University of Vermont Medical Center found that a 24-item operating room checklist did not significantly reduce the incidence of any of nine postoperative adverse outcomes.

More than 12,000 cases were studied, and outcomes included mortality, death among surgical in patients with serious treatable complications, sepsis, respiratory failure, wound dehiscence, postoperative venous thromboembolic events (VTE), postoperative hemorrhage or hematoma, transfusion reaction, and retained foreign body (FB).

After the checklist was established, respiratory failure rates decreased significantly on the initial analysis, but the difference disappeared when the Bonferroni correction* was applied to the data set.

Why didn’t the checklist work? I have discussed this in previous blog posts here and here. As was true in previous papers of this nature, many of the complications studied—respiratory failure, wound dehiscence, transfusion reaction, postoperative hemorrhage or hematoma—could not have been prevented by a checklist.

The checklist from hell

This one-page form illustrates much of what is wrong with medical care today.

Just imagine the amount of time it would take to complete all 11 steps. Wouldn't you love to hear what goes on in a "post Fall Huddle"?

Thanks to @anish_koka for tweeting the form.

Do operating room checklists improve outcomes?

The other day Atul Gawande tweeted the following:

I am not against checklists. When I was a surgical chairman, I implemented and used one in both the operating room and the ICU. They do not add costs and may be helpful.

However, the randomized trial that Gawande referred to does not necessarily settle the issue about whether checklists really do reduce complications and deaths.

The paper, published online in Annals of Surgery [full text here], looked at 5,295 operations done in two Norwegian hospitals. The intervention was a 20-item checklist consisting of three critical steps–the sign in before anesthesia, the timeout before the operation began, and the sign out before the surgeon left the operating room. Using a stepped wedge cluster design, patients were randomized to control or the checklist.

Complications occurred in 19.9% of the control patients and 11.5% in those who got the checklist, a significant difference with p < 0.001.

A look at Table 2 finds that of 27 complications or groups of complications, 14 occurred in significantly fewer patients in the checklist group.

Of the significant 14, a few, such as cardiac or mechanical implant complications, could possibly have been prevented by the implementation of the checklist.

For most of the others, the relationship between the use of a checklist and a post-operative complication is tenuous. How could a checklist possibly prevent technical complications like bleeding requiring transfusion, surgical wound dehiscence, and unintended punctures or lacerations?

Here are a few more of the complications that occurred significantly less frequently in the checklist cohort—urinary tract infection, pneumonia, asthma, pleural effusion, dyspnea, and the nebulous categories of "complications after surgical and medical procedures" and "complications to surgery not classified."

What item on a checklist prevents asthma, UTI or any of those on that list?

Embolism, sepsis, and surgical site infection, three complications one would expect a checklist to impact because of reminders to give prophylactic antibiotics and anticoagulation, did not occur at significantly lower rates in the checklist group.

Even the cardiac complication category is open to question because none of the 5 subcategories (cardiac arrest, arrhythmia, congestive heart failure, acute myocardial infarction) differed significantly between the two groups. Only when the 5 were combined did statistical significance emerge.

In the 300-bed community hospital, checklist use was associated with a significantly lower mortality rate than non-use, 0.2% vs. 1.9% respectively (p = 0.02), but no mortality difference was seen in the 1100-bed tertiary care hospital.

The tertiary care institution enrolled 3,811 patients while the 300-bed hospital contributed 1,083. If more patients had been in the latter group, the difference may have disappeared due to the principle of regression to the mean.

Despite the heightened vigilance associated with an ongoing research project, compliance with checklist use was only 73.4%.

Before you go off on me, I will remind you that I do not oppose checklists. Most things we do in medicine are not based on Class 1 evidence.

Just don't tell me that checklists have been proven to reduce complication rates or save lives.

Despite checklists, wrong-site surgery still occurs. Why?

Not long ago, two California hospitals were cited by the state for wrong site surgery.

At one hospital, a skin incision was made in the right groin of a patient who was to have had a left orchiectomy. Fortunately the error was discovered, and the skin was closed. The correct testicle was removed. To his credit, the surgeon told the patient and his wife what happened immediately after the operation.

An investigation found that the patient's groin had not been marked. The time out did not prevent the error because the OR team did not verify the operative site as the protocol mandates. In other words, they had gone through the motions.

The hospital was fined $75,000.

The other California case did not turn out as well. Instead of removing a cancerous left kidney, surgeons did a right nephrectomy. The error was not discovered until the pathology report was reviewed. A number of assumptions and misunderstandings contributed to this error for which the hospital was fined $100,000.