Review courses and board exams

Four years ago, I wrote a post called "Hints for new residents." Among my 15 tips was this: "Read, read, read. This isn't like school. You can't cram for your boards. You can’t learn 4 or 5 years’ worth of material in a one-week review course. You have to learn it as you go along."

Just published online in the journal Surgery is a paper entitled "Review courses for the American Board of Surgery certifying examination do not provide an advantage" by four officials from the board.

They surveyed new surgeons who took the certifying (oral) exam, 1067 for the first time and 329 who had previously failed the test, during the time period from October 2012 through June 2013. The overall response rate was 90%.

The pass rate for first-time takers was significantly better than that of repeaters, 82.1% and 72.6% respectively, p < 0.001; 77.9% of all examinees took a review course—76.1% were first-time takers compared to 84.6% of those repeating the exam, p = 0.002.

From the paper: "The overall CE [certifying exam] pass rate did not differ between those who did and those who did not participate in a review course (82.7% vs. 78.9%; p = 0.22)."

The results were controlled for sex, US or international med school graduate, written board exam scale scores, average written board scores over the last five years for the candidate's program, and size and type of program.

The review courses were also analyzed, and the authors found that attending any one of the specific courses failed to predict passing the certifying examination. The only significant predictor of passing the certifying exam was the scale score of the candidate on the written examination.

The authors concluded: "On the basis of this survey, there was no evidence that participating in a board review course provided a benefit to passing the CE of the ABS."

The work was presented at a meeting, and the transcribed discussion was also published. The assigned discussant, Dr. Michael Nussbaum, said, "As a long-term program director, I really strongly believe that preparation for the CE is a 5-year process, not something that can be taught in a short course or crammed for."

I am pleased that the board and the discussant agree with what I said four years ago.

However, it is not clear from this paper that review courses are of no value when preparing for the written board examination. The paper said the score on the written examination correlates with passage of the oral examination. If a review course a helps candidates pass the written exam, then indirectly, a course might have a positive effect on the outcome of the oral exam.

Another problem with the paper is its failure to separate prep courses for the written and oral exams. One would not expect a course directed at the written exam, which is more of a multiple choice test about remembering facts, to help with the oral exam, which focuses on a candidate’s judgment and maturity in managing hypothetical patients.

The authors noted that only 29% of candidates for the boards in took review courses in 1990, and the courses are expensive.

Should the nearly 80% of candidates who took courses 2 years ago not have done so? Unfortunately, this paper does not conclusively answer the question.