“System Failure” Often Really Means Someone Made a Mistake

Last week, Cleveland Browns quarterback Colt McCoy suffered a helmet-to-helmet hit by Steeler linebacker James Harrison. Controversy arose over the fact that McCoy was allowed to continue playing after a brief absence from the game. Later he exhibited many of the classic symptoms of a concussion.

The players’ union said a “blatant system failure” had occurred and that McCoy had not been examined adequately. Apparently the NFL’s protocol for ruling out a concussion was not followed.

Because the rules were not followed does not mean the system broke down. What it does mean is that one or more individuals did not comply with the rules, i.e., human error occurred. The league is now said to be considering hiring independent neurologists for each game venue.

UPDATE 12/15/2011
The Browns now admit that McCoy was not examined on the sideline after the hit. The medical staff was busy attending to other injured players and didn't see McCoy go down. This does not explain why the coaches and players who did see the hit didn't feel the need to tell the medical staff how violent the collision was. Bottom line: Not a system failure.

It’s easier to blame the system than to admit that someone made a mistake.

This all reminds me of a hospital with which I am familiar. Whenever a medical error happened, investigations took place, risk management meetings were held and many times a “root cause analysis” [RCA] was undertaken. Although in the majority of cases it was obvious that human error was the cause of the adverse event, the decision was usually that a “corrective action” involving what was perceived to be a system problem needed to be taken.

The corrective action meant that a new policy, something like having a neurologist on the sidelines, had to be created. There was no proof that the corrective action would be effective. There was evidence that a lot of work would result from the monitoring and documenting that the corrective action was ongoing. After a while, the corrective action was gradually forgotten.

Let’s look at some literature on the causes of medical error.

In 2010, van Wagtendonk et al published a paper in the British Journal of Surgery about unintended events in surgery units. Of the 881 events studied, 72.3% were the result of human error. A 2008 paper from the University of South Florida reviewed a hospital’s 12-month experience with surgical complications. Technical and judgment errors predominated, with system errors accounting for only 2% of the major complications seen. Surgeons at the University of Southern California looked at their trauma deaths over an 8-year period. Of 51 deaths classified as preventable or potentially preventable, surgeon error was the cause in nearly all cases.

Dr. Peter J. Pronovost, the noted Johns Hopkins patient safety guru, and colleagues reported in JAMA that root cause analysis is not very useful in preventing medical errors. Here are some quotes from that paper.

Many RCAs are performed incorrectly or incompletely and do not produce usable results. Anecdotally, officials in state health departments observe that the quality of the RCAs they receive varies widely.

Formulating corrective actions is more difficult than finding problems, and follow-up on outcomes is rare. A sign of the incomplete adoption of recommendations is that despite having recently completed an RCA for a specific incident, hospitals commonly experience repeat events, which is a reminder of words attributed to Einstein, “Insanity is doing the same thing and expecting a different result.”

Although there have been some benefits, including increased awareness of faulty processes and fixes to specific problems, there is an undercurrent of sentiment that this approach [root cause analysis ] has limited effectiveness.

Luckily, I no longer must sit through 3-hour RCAs. Or as a friend once said, “Would you rather have a root cause analysis or a root canal?” Tough choice.

UPDATE 12/17/2011
See followup post on this subject