LINGUIST List 15.415

Sun Feb 1 2004

Disc: Final Posting: Blind Peer Review

Editor for this issue: Sarah Murray <sarahlinguistlist.org>

Directory

1. Martha McGinnis, Re: Blind Peer Review
2. Paul Chapin, Re: Blind Peer Review
3. Ronald Sheen, Re: 15.355, Disc: Penultimate Posting: Blind Peer Review

Ronald Sheen asks what might be the possible consequences of someone
attaching their name to an honest review.

Given the reciprocal nature of peer review, success in an academic
career depends in part on having a harmonious (or at least neutral)
relationship with colleagues in the same field. In some cases, writing
an honest critical review of a fellow scholar's work could jeopardize
that relationship. This is a particular concern for junior scholars,
whose ability to get a job and/or tenure depends in part on the
goodwill of their colleagues. I think David Odden is right: without
the protection of anonymity, and faced with the decision of whether or
not to write an honest negative review, many potential reviewers would
very sensibly refuse to write a review at all.

Cheers,
Martha 

Some of the postings on this thread have referred to funding agencies
as well as to journals. As a former program officer at one such agency
(NSF), let me endorse and echo Sally Thomason's very cogent comments,
which apply with equal force to the review process for grant
proposals.

 
Paul G. Chapin
pchapinalum.mit.edu
 

Message 3: Re: 15.355, Disc: Penultimate Posting: Blind Peer Review

Sally Thomason's recent post along with that of other editors is
essential to this sort of discussion because they have first hand
experience of dealing with reviewers. Their opinion is clearly against
open peer review for a variety of reasons one of which is the harm it
may do to personal relationships. However, isn't this an odd priority?
If respected, it would entail not criticising some advocacy or other
(in terms of teaching approaches, for example) for fear of not being
liked by someone or other. However, the failure to hold such
advocacies up to open scrutiny leads to far more serious consequences.

Krashen's advocacy of the Input Hypothesis and then The Natural
Approach was largely untouched by trenchant critique (with notable
exceptions). Because we do not know what submissions journals rejected
at that time, we do not know if this lack of critical comment in the
literature was due to the applied linguistics community falling en
masse for the Krashen advocacy or whether this was the impression
given by a publishing policy which tended to be unwelcoming to
articles critical thereof. I tend to believe it was both and this,
partly because of what occurred in Quebec in the 80's.

There, applied linguists such as Lightbown and Spada advocated a
teaching approach largely based on the Input Hypothesis and were
successful in doing so. This has resulted in students in Quebec
schools being deprived of any systematic teaching of grammar for the
last two decades. In my view, this was allowed to happen because of
the intellectual and pedagogical climate partly created by the absence
in the literature of any close scrutiny of the advocacy. Once a
bandwagon builds up momentum, it is difficult for saner possibilities
to be given journal-room. I find it difficult to understand how any
applied linguists with classroom experience could have fallen for the
Krashen advocacy. However, there was published in 2002 a rare touch of
the confessional when Lightbown, much to her credit, admitted that at
that time she believed that grammar should not be taught in the
classroom because, in the right conditions, it is acquired naturally.
Many others, unfortunately, thought similarly. Those who did not,
either kept silent or did not manage to get into print.

If open peer review is supposed, as Sally Thomason suggests, to lead
to a less friendly relationships between fellow scholars, would it
necessarily be a bad thing. It may lead to considerably less mutual
back-scratching and fewer exclusionary tactics in terms of various
aspects of applied linguistics.

In this regard, does anyone have any feedback on those psychology
journals which appear to have no problem with publishing open peer
reviews as part of publications. Do the participants not have friendly
releationships and if they do not, what are the consequences?

Actually, on occasion, some journals in applied linguistics do
practice a sort of open peer review. I'm thinking particularly of
Second Language Research and even more particularly of a debate
between Kevin Gregg and Fred Eckmann. I don't suppose they were the
best of pals afterwards but is that of any relevance. The exchanges
provided an excellent insight into the issues discussed. The same
applies to the other journals which allow for the publication of
response articles. Aren't transparency and accountability more
important than effects on personal relationships?

Perhap, one of the most important suggestions coming from the recent
messages is that of Franca Ferrari who proposes that there be an
independent review body which can address appeals objectively. As my
previous posts have indicated, journal officers' reaction to appeals
appears to be a knee-jerk closing of ranks and an automatic rejection
of them without addressing their substance. A body is need which
considers transparency and accountability as far more important than
the possibility of hurting someone's feelings

Finally, Kevin Gregg rightly raises the natural fear of reprisals on
the part of untenured reviewers. That may be even extended to editors.
Recall the example I gave of the "roasting" given to the two editors
for publishing articles of mine. One of them also added that she knew
she need not apply for jobs where the "roaster" had any say in who was
to be hired. Kevin also mentions horror stories he might relate. Pity
he doesn't. It might be useful to have on record particularly
egregious abuses of the blind peer review system. It is for that
reason that I did so in the case of my appeal where the response was
to agree to undertake a proper enquiry but then simply ended up in the
passing of the buck and implicitly rejecting the appeal without
addressing its substance.

Ron Sheen