Open versus Blind reviewing process

The 2008 study entitled Peer Review in Scholarly Journals - Perspective of the Scholarly Community: An International Study aimed "to measure the attitudes and behaviour of the academic community with regard to peer review." Some quotes from the summary:

Double-blind review was preferred. Although the normal experience of researchers in most fields was of single-blind review, when asked which was their preferred option, there was a preference for double-blind review, with 56% selecting this, followed by 25% for single-blind, 13% for open and 5% for post-publication review. Open peer review was an active discouragement for many reviewers, with 49% saying that disclosing their name to the author would make them less likely to review.

and

Double-blind review was seen as the most effective. Double-blind review had the most respondents (71%) who perceived it to be effective, followed (in declining order) by single-blind (52%), post-publication (37%) and open peer review (27%).

A 2008 article in Nature (and a correction) discusses the above study but the article is about double-blind review versus single-blind review, and not about blind review versus open review.


The funny thing is that on this issue, most people mention immediately the "honesty" side of the problem, not the "quality" side. For the latter, I think that this not blind/open reviews which is the pertinent question, but rather the public/private question. If reviews are always public, then my guess is that the quality will increase, because the general chair/editor in chief will push that way to ensure its conference/journal to have excellent reputation. Personally, I don't want/need to know who is reviewing my papers, but I want AND need quality reviews, and they are unusual those days. We all know why : too many papers, too many reviews to make, not that much time...


The more is revealed about the identity of authors and reviewers, the less honest the review process may be. Such openness may favour already-established scientists over newcomers.

In the ideal world a reviewer would raise the same concerns when reviewing papers from a Nobel-prize winner or from an undergraduate student. But as people are even afraid of asking possibly dumb questions in public, I would be really surprised if they could apply the same scrutiny regardless of who they are reviewing.

Even with the standard (single)-blind process, I heard that an already-famous scientist submitted papers under made-up names to receive honest reviews (just can't recall who).

An example from Herbert S. Wilf on a birthday speech for Donald E. Knuth (pointed out by Joel Reyes Noche):

In the 1980's, in the early days of the Journal of Algorithms, I was an Editor-in-Chief, and Don [Donald Knuth] submitted a paper to me, authored by himself under the pseudonym of Ursula N. Owens, ostensibly from some small college in some small nonexistent town in Kansas. The reason was that he really wanted to get a tough and substantive referee's report on the paper, and he had been finding that sometimes referees had pulled their punches because of his name at the top of a paper.

Double-blind process may be even more beneficial, but at the same time illusory (as topic, references and style may reveal the author). Moreover, the identity of the author may sometimes be beneficial (e.g., to compare if the new submission has something new).

None of it is a proof.

But instead of counting of lines of reviews, one can try to compare how softly (or harshly) are treated contributors, depending on their status (academic title, university name, fame/recognizability).