How are journal articles peer-reviewed?

There is a possibly some misunderstanding among science journalists regarding the process that the term “peer-review” encompasses.

I am an associate editor (AE) of a medical journal with a respectable impact factor. I also am or have been a manuscript reviewer for five different journals. I feel qualified to describe how manuscripts are reviewed and published.

In 2012, authors submit a manuscript electronically to the journal. It is assigned to an AE who screens it for appropriateness, format and, on occasion, readability in the English language. Manuscripts are not blinded. AEs and reviewers are aware of the authors’ names and their institutions.

The AE emails prospective peer-reviewers asking if they are willing to review the submission. Reviewers are chosen based on their self-reported areas of interest. They become listed as peer-reviewers by demonstrating expertise, usually having submitted papers of their own. They may also be well-known experts through society memberships or familiarity with the journal’s editorial board members. I once became a peer-reviewer for a journal after writing a letter to the editor pointing out a statistical flaw in a published paper. Although seen by many as a career-enhancing, the jobs of AE and peer-reviewer are not compensated.

If all goes well, the peer-reviewers return their recommendations in a timely way. Unfortunately, being a peer, an expert or an author in a field related to the manuscript’s topic does not necessarily mean that one can review a research paper competently. Most journals have guidelines for reviewers but no way to tell if the reviewer has read them. We often receive “two-sentence” reviews of 25-30 page (double-spaced) manuscripts.

A manuscript would have to be quite extraordinary to elicit only a two-sentence review. The reviewer may have been too busy, disinterested, incapable or not motivated to do a thorough job. But then why would he have accepted the assignment? That’s one of life’s great mysteries. The AE may have to become a peer-reviewer at times.

Assuming the AE receives two or three adequate reviews, he decides to accept, accept with revisions or reject the paper and forwards it to the editor for a final decision.

Here is what we cannot do. We cannot verify that

1. the data are not fabricated;

2. all authors deserve to have their names listed on the paper;

3. no plagiarism has occurred;

4. the paper is not an attempted duplicate publication.

Journals have no resources to investigate any of these issues. We must accept the word of those submitting. Among other causes, pressure on faculty to publish and/or greed may promote scientific misconduct.

Is this a good system? No. What are the alternatives? I don’t know. I pointed out in a previous post that many more people have read my blog than ever read my research publications. One day, every paper may be posted and critiqued by the scientific public, a movement that has already begun on websites such as Faculty of 1000.

Meanwhile, expect to see more publications retracted as internet users discover and expose fraud and duplicate publications. For more on retractions, follow the blog Retraction Watch for interesting insights into the process.