Should resident promotion decisions be based on a written exam?

A few days ago, some surgeons on Twitter discussed the role of the American Board of Surgery In-Training Examination, a test which is given every year in January.

The test was designed to assess residents' knowledge and give them an idea of where their studying should be focused. However, many general surgery program directors (PDs) use the test results in other ways. Some impose remediation programs on residents with low scores and even base resident promotion or retention on them. Some even demand that all residents in their programs maintain scores above the 50th percentile.

The Residency Review Committee (RRC) for Surgery frowns upon these practices and states in its program requirements (Section V.A.2.e) that residents' knowledge should be monitored "by use of a formal exam such as the American Board of Surgery In Training Examination (ABSITE) or other cognitive exams. Test results should not be the sole criterion of resident knowledge, and should not be used as the sole criterion for promotion to a subsequent PG [postgraduate year] level."

The problem for program directors is that the RRC also mandates (Section V.C.2.c) that "as one measure of evaluating program effectiveness" 65% of a residency program's graduates must pass both the American Board of Surgery's Qualifying Examination (written) and Certifying Examination (oral) on their first attempts. I have said before that the "65% on the first attempt rule" does not seem evidence-based.

From the trenches: More about grit

The following was compiled from two comments on my recent post about grit written by a doctor who calls himself "Geronimo." It is reproduced with permission.

Grit cannot be assessed by a survey. I wholly agree. As a military physician, my firmly founded opinion is that grit is essential to the practice of medicine. Grit is the elusive characteristic that carries the clinician through the challenges that exceed ordinary capabilities. You cite a paper that argues for surgical training to borrow aspects of SEAL training. I applaud any measure that would allow senior faculty and program directors to unilaterally shape their residents’ training, whether or not it bears any resemblance to the rigors of BUD/S [Basic Underwater Demolition/SEAL training].

The 2011 loss of 30-hour call for medical students and interns was a fatal blow to residency training, in my estimation. I count myself fortunate for having a 30 hour call internship before embarking on my operational career. While downrange, it is not at all uncommon to be woken at inconvenient hours of the night to tend to the wounds of war. If you don’t know how you function cognitively, physically, psychologically, and emotionally while sleep deprived, exhausted, hungry, cold, and pissed off, you’re behind the curve. While it isn’t any fun to work in such a state, or to work with people so challenged, it is decidedly less fun to be a patient expiring for want of any medical provider, let alone a tired one. American medicine used to be in such a place in the not so recent past, to hear the story told by my forbearers.

True grit

In case you haven't noticed, a hot new topic in education is "grit." In order to reduce the long-standing 20% attrition rate of surgical residents, some say we should select applicants who have more grit or conscientiousness.

A recent paper in Surgery reported residents who dropped out of programs had decreased levels of grit as measured by a short-form survey. But due to unexpectedly low attrition rates in the surgical programs participating in the research, the study was underpowered to show a statistically significant difference in outcomes of high- vs. low-grit individuals.

It's hard to argue with the premise that choosing applicants with high reserves of grit might lead to better retention and performance of residents.

I blogged about this three years ago in a post called "Harvard says train residents and medical students like Navy SEALs."

Unfortunately, identifying who has grit will be much more difficult than simply testing those applying for surgical residency training.

Below is the eight-item grit survey, which is scored on a 1 to 5 Likert scale.