LINGUIST List 15.118

Fri Jan 16 2004

FYI: New Website to Discuss Blind Peer Review

Editor for this issue: Sarah Murray <sarahlinguistlist.org>

Directory

1. Ronald Sheen, New Website to Discuss Blind Peer Review

The problems created by the blind peer review process as practised by
prominent applied linguistics journals are well-known but little
discussed openly in the literature. In fact, so little that that no
article devoted to those ills has, to my knowledge, been published in
those same journals. The reason for this absence is self-evident. Any
article on the subject will necessarily entail the discussion of
examples of the process in action and, almost certainly, of cases
where the process has putatively resulted in the unjustified rejection
of an article. As this will inevitably result in a critique of the
process as applied in the journal to which the article is submitted or
in other journals, no editors will countenance its publication.
However, not all fields are so shy of self-criticism. De Beaugrande
(see below), for example, cites Peters, Douglas, Stephen Ceci, et
al. 1982. Peer-review practices of psychological journals: The fate of
published articles, submitted again. Behavioral and Brain Sciences
5/2, 187-255.

It is not my intention here to discuss the various ills of the
process. For such a discussion, readers are invited to access Robert
De Beaugrande's article, "Peer Review" available on his website. The
crucial point to be made here is the following: the history of applied
linguistics as practised in issues concerning foreign and second
language teaching is not one which inspires confidence in its
influence in promoting more effective language teaching. One has only
to bring to mind doctrinaire principles it has supported and then
abandoned and of the tired but relevant clich� of "the baby and the
bathwater" to find support for this contention. The inevitable
conclusion to draw from this is that articles have either been
published or rejected on grounds which would not withstand scrutiny
based on empirically-verifiable criteria.

One the fundamental problems of the blind peer review process results
from the lack of transparency and accountability therein. Thus, one
does not know the identity of the reviewers; nor can one hope to
oblige them, through the editors, to provide an explanation and
justification of their decisions on the contentious issues. This lack
results in a veil of secrecy concealing much of what occurs between
those who submit articles, the reviewers and the editors.

This is not to say that the process does not always work effectively.
There are certainly many cases where articles have either been
rightfully published or rejected. At the same time, however, given
what has been said above concerning the history of applied linguists
and given the inevitable errors which reviewers may commit, it is
certain that some articles have been unjustifiably rejected.

This brings me to the purpose of this post. It is to announce the
creation of a website for the discussion of this issue. More
specifically, it is to offer a website which enables anyone to discuss
the pros and cons of both published and rejected articles. Even more
specifically, it is to invite authors of rejected articles to make
available to the readership those same articles, the comments of the
reviewers, any relevant correspondence between the author and the
editor(s) and the arguments the authors wish to advance to critique
the grounds for rejection. Those authors would do so, however, on the
assumption that they are making themselves, their articles and
arguments susceptible to criticism and this because it may well
transpire that contributors to the discussions may agree with the
rejection.

Authors should also realise that this website's purpose is NOT to
provide a means of "journal-bashing" or the unchallenged expression of
"sour grapes". It is to provide a transparent and accountable
discussion of the grounds on which were based the rejection of an
article.

To start the ball rolling, I will offer as "a guinea pig" all the
necessary documentation surrounding the rejection of an article I
submitted to TESOL Quarterly's Brief Reports last July. In my view,
this case will be a good illustration of the ills of the blind peer
review process as anyone who cares to visit the website can discover.
At the same time, however, I hope that contributors will feel free to
hold my arguments up to scrutiny and, should they find them wanting,
say so.

Here is a brief review of facts of that case. The article is entitled:
"An examination of the validity of the principles incidental learning
and developmental sequences". These two principles have become tenets
of the "focus on form" movement which has preoccupied many applied
linguists for the last fifteen years or so. Those tenets basically
contend that exposure to understood language use in the classroom will
result in incidental learning without pedagogical intervention and
that that learning will result in learners' passing through
development sequences on the way to acquisition. Unfortunately,
however, even though these claims have now become part of contempoary
wisdom, there is a dearth of empirical evidence to support them
derived from the necessary longitudinal and/or cross-sectional
studies.

On the other hand, the submitted article in question derives data from
a cross-sectional study covering eight years of elementary and
secondary learning of English in the Quebec school system using a
strong communicative approach (no systematic teaching of grammar
allowed) and demonstrates that contrary to published claims concerning
the acquisition of third person interrogatives, the vast majority of
learners in the initial stages acquire non-native forms (e.g., "Where
your father work?") and continue to use them until the end (and
beyond) of their secondary schooling thus demonstrating in such cases
an absence of incidental learning following the initial stages and a
failure to pass through subsequent developmental sequences.

This article was rejected on the basis of various seriously erroneous
claims contained in the summary of the reviewers' reports as provided
by the two co-editors of the TQ Brief Reports section. In spite of my
providing textual proof of the erroneous nature of these claims, the
co-editors refused to address the issue and forwarded my appeal to the
Chief Editor who, though initially agreeing to entertain the appeal,
supported the rejection without even reading the reports of the
reviewers and without demonstrating any flaws in the arguments
supporting the appeal. Subsequently, I have made numerous requests to
be sent those reports but have failed to get even a commitment from
the editors concerned to send them to me. I am now in the process of
appealing to a higher TQ body to which I was referred by the Chief
Editor. However, in spite of sending to it several letters including
an official appeal, I have so far received no response.

In my view, it is such cases which need to be held up to scrutiny in
order to bring about a more transparent and accountable approach to
the review of submitted manuscripts. I should add, however, that I
have no illusions about the potential of the proposed website. Any
informed discussion of the relevant documentation concerning a
rejected article will require investing a good deal of time which many
contributors may not be prepared to do. The website may not,
therefore, serve the purpose for which it is intended. Nevertheless,
it is worth making the effort even if it only results in failure. At
the very least, it will permit rejected authors to put on some sort of
public record what they see as the facts of their cases.

The weblog site is called "Articlereview" and may be accessed by going
to the following url:
http://www.20six.co.uk/articlereview

If you would like to offer comments, click on 'comments' and follow
the on-screen directions. However, whatever you wish to post, it is
important to bear in mind that the purpose of the site is to promote
academic argument and respect for the norms of such discourse.


If you would like to post directly onto the weblog, rather than just
making a comment, email your contribution to this address:
articlereview.343720six.co.uk 

Avoid using formatting in your email as the weblog program might not
be able to read this and could reject your post. Also, note that
emails sent to the above address will instantly appear on the weblog
for all to see.

To check for new posts to the weblog, go to the url and hit your web
browser's 'refresh' or 'reload' button.



Ronald Sheen
American University of Sharjah