LINGUIST List 15.276

Sun Jan 25 2004

Disc: Re: Blind Peer Review

Editor for this issue: Sarah Murray <sarahlinguistlist.org>

Directory

1. Ronald Sheen, Re: 15.204, Disc: Re: Blind Peer Review

Message 1: Re: 15.204, Disc: Re: Blind Peer Review

Geoff Pullum's idea (as reported by Joe Tomei, Linguist 15.204) would
go a little way to solving the problems of the blind review process.
Unfortunately, however, without a list of the submissions rejected by
a particular reviewer, it would mean little. If, as seems plausible
given the history of applied linguistics (see previous post),
reviewers are susceptible to the prejudices of the contemporary
mind-set, this would only be revealed by a list of the submissions
both accepted and rejected. It would, for example, be revealing if a
reviewer systematically rejected submissions questioning the bases of
contemporary wisdom given that the latter is founded on such flimsy
evidence.

It is, for example, remarkable that the applied linguistics literature
is replete with assumptions of the validity of the principles of
incidental learning (without pedagogical guidance) and associated
developmental sequences. Yet, that same literature is devoid of
empirical evidence derived from longitudinal studies demonstrating
these two principles successfully "at work" in the classroom in terms
of the acquisition of anything resembling native-like competence.
Further, and I speak from experience, it is extremely difficult to
have published articles which hold these principles up to close
scrutiny.

What has happened to the obligation on the part of authors, and, in
turn, reviewers and editors to insist on there being empirical support
for crucial claims as part of the advocacies of teaching approaches
based on the putative validity of these two principles?

Furthermore, on the rare occasions when there is published an account
of a longitudinal study (six years) (See Lightbown et al. 2002 in the
Canadian Review of Modern Languages) where concrete evidence of the
principles at work (if valid) should have appeared, there is
absolutely no discussion of the obvious absence of the necessary
evidence in the data collected during the study.

Now, given the strong support of Lightbown for the two principles,
it's surprising that the reviewers and editors involved did not insist
that the study address the absence of relevant empirical evidence and
draw the inevitable conclusions.

What is particularly noteworthy in this question of developmental
sequences is that one well-known journal, Applied Linguistics,
apparently had no compunction about publishing Lightbown's advice
(2002:533) to teachers to be patient while waiting for developmental
sequences to take effect - when there is, in fact, absolutely no
empirical evidence to support such advice . In fact, what evidence
there is indicates that the "patient wait" will result in
fossilisation rather than progress in learning.

Here again, what has happened to the obligation on the part of
reviewers and editors to show some circumspection when publishing
unsupported advocacies of teaching approaches?


Martin Haspelmath's (Linguist 15.182) addition of other issues in the
blind review process is welcome as it adds to the expression of
dissatisfaction felt by many at the way in which journals fail to
respect their implicit mandates. It is of interest in that it
criticises some journals for the length of time it takes from
submissions to rejections or publication. Of interest, because in a
world in which transparency and accountability counted for something
and in which journals had to answer for their handling of submissions,
this List would be used by journal editors to explain the many
problems they face which may account for their less than satisfactory
performance.

Unfortunately, judging from the past, critiques of various aspects of
the world of applied linguistics published on this List are simply
ignored by those responsible for those aspects. I, therefore, fully
expect that editors will not respond to the criticisms.

Should this turn out to be the case, it will be remarkable in itself
given the many thousands of members of this List for it means that
those in some sort of power in this field can simply ignore criticisms
even though those criticisms have become public knowledge, so to speak

As other examples of those criticised on this List failing to respond,
take reviews of books published here. The publishers, editors and
authors of the books clearly profit from the appearance of the reviews
on this List. Given this, I would have thought that those editors and
authors, at least, would feel some ethical obligation to respond to
critiques appearing in those reviews. In very large part, this is not
the case. (Illustrative examples provided on request.)

To come back to this particular thread of the blind review process, I
would argue that the bandwagon syndrome results in those in its thrall
gaining power in the field as editors and reviewers. As such, they act
in tandem in reacting negatively to any criticisms of contemporary
received wisdom. Editors can do this quite easily by submitting
critical manuscripts to reviewers they know will reject them. This
will remain impossible to substantiate whilst this endemic problem of
the blind review process is allowed to continue to exist.

Though it may have advantages, its disadvantages in terms of a marked
absence of transparency and accountability far outweigh them. Further,
though the word "blind" is supposed to work both ways, it patently
often does not. Though the potential author may be deprived of the
identity of the reviewer, the reverse is often not the case because
the submitted manuscript often inevitably reveals the identity of the
author.

There are certainly many who continue to support the blind review
process. Would that some of them would contribute to this debate by
attempting to respond to the argument that first, the abuses of the
process far outweigh its advantages and second, a transparent review
process would bring a much-needed accountability to the reviews
produced.

Ron Sheen



References:

Lightbown, P.M. (2002) "The role of SLA research in L2 teaching: Reply to
Sheen". Applied Linguistics, 23-4: 530-536.

Lightbown, M. P., Halter, H. R., White, J. L. & Horst, M. (2002)
"Comprehension-Based Learning: The Limits of 'Do It Yourself' ". CMLR, 58:
427-464.