LINGUIST List 15.310

Wed Jan 28 2004

Disc: Re: Blind Peer Review

Editor for this issue: Sarah Murray <sarahlinguistlist.org>

Directory

1. Ronald Sheen, Re: 15.277, Disc: Re: Blind Peer Review
2. David Odden, Re: 15.276, Disc: Re: Blind Peer Review

Message 1: Re: 15.277, Disc: Re: Blind Peer Review

Lawrence Baron writes:

"The other thing is that I am told that reviewers are not paid to do
this; it is all done out of academic altruism. Is that generally the
case?"

Though there may be some altruisim involved, we cannot discount the
very natural desire to advance one's career. As one contributor to
this discussion, Kleanthes Grohmann, mentioned, CV's often include
details of reviews carried out for various journals. The assumption
here is that being invited to do such reviews constitutes a
recognition of some standing in the field.

This, however, brings us back once again to the lack of transparency
and accountability in the blind review process. Journals do, on
accasion, list those who have done blind reviews (One up for
transparency). What is worthy of note in such lists is the number of
names which one seldom, if ever, sees as authors of published work.
This raises the following question: What qualifications do journal
editors demand of reviewers. They clearly do not have to have
published either in the field in general or in the specific area of
the article to be reviewed. So what do they have to have done to
qualify? Can any List members offer any enlightenment on the process?

I ask this because of the many errors I have noted in reviews, in
general, and, in the ones I mentioned in my initial introduction to
this discussion. In that latter case, I have already detailed the
serious errors apparently made by the reviewers. (for details see my
weblog). I say "apparently" as TESOL Quarterly has still not provided
me with copies of the actual reviews.

However, it would appear that both my reviewers do not know the
difference between anecdotal and empirical evidence and use that
misunderstanding as one of their arguments to support rejection. How
can any such reviewer worthy of the name be invited to evaluate a
manuscript based on empirical evidence?

Is there any chance of any editor or ex-editor informing us as to the
procedure for inviting people to be reviewers? Or, for that matter,
being on the Board?

Ron Sheen

Message 2: Re: 15.276, Disc: Re: Blind Peer Review

Ronald Sheen urges some / any who support blind reviews to contribute
to this debate, so as a supporter, I will do so. The fundamental issue
is an ethical one -- a reviewer has the right to know in advance
whether his/her identity will be revealed. If a journal has an
announced policy of anonymous reviews, that constitutes an implicit
promise of privacy between the journal and the reviewer (or associate
editor). The same ethical principle dictates that it would be wrong to
publish the names of reviewers who accepted or rejected an article, or
even to list the names of reviewers used by a journal, without the
consent of the reviewer.

The solution to the peceived problem would be to persuade editors that
it is desirable to reveal reviewer identities as a matter of
policy. As editor of the journal Studies in African Linguistics, I
will not do this, because it would exacerbate the problem of getting
good reviewers to write honest reviews. It is not easy to find
reviewers, and abandoning anonymity would only decrease the size of
the pool of reviewers, which would be a seriously negative outcome.


-David Odden