How journal articles are peer-reviewed: Part 2

Based on my experience as an associate editor of a journal, I recently blogged about the current mechanism of peer-review that most medical journals use and pointed out that it doesn't work very well.

Here is a story that illustrates some of the problems with the peer-review process.

A journal editor sent a manuscript to three peer-reviewers.

Of the three reviews returned, one recommendation was to “accept,” one said “accept with minor revision” and one said “accept with major revision.” Two of the reviews were 2-3 sentences long. One simply complimented the authors on a “nice paper.” The other pointed out two trivial areas needing revision.

The third review raised a couple of major points questioning the methods used by the authors. These were significant issues suggesting that the paper was unsalvageable.

Conflicting opinions are not that unusual. I have seen three reviews returned which had three even more diverse opinions such as, accept, accept with revision and reject. Based on that input, how does an editor decide the fate of a paper?

Perhaps you have read a paper and wondered, as I have, how did this ever see the light of day? Heck, I’ve even written a few papers like that myself. There used to be a saying, "If you have enough stamps, you can eventually get anything published." Now you don't even need the stamps.

Bottom line for me is that the peer-review process is very weak at the level of the peer-reviewers themselves. The reviews are not standardized and the quality is inconsistent.

Coincidentally, the New York Times ran a piece today mentioning some other problems with conventional journals such as “Peer review can take months, journal subscriptions can be prohibitively costly, and a handful of gatekeepers limit the flow of information.” The new wave seems to be “open science.” It will be interesting to see how this evolves.

I may be out of a job as an unpaid associate editor soon.