LINGUIST List 5.745

Mon 27 Jun 1994

Sum: Native speaker judgements

Editor for this issue: <>

Directory

1. , summary: native speaker judgements

SUMMARY: NATIVE SPEAKER JUDGEMENTS

About two weeks ago, I posted a request for views on and experiences in dealing
with native speaker judgements, highlightling in particular the fuzzy nature of
the judgements. Here are the replies. There has been a good variety of
opinions. First there were those that alerted me work already done in the
area, to whom I am grateful. Then there are those of a more mainstream
persuasion who believe that there are a number of non-linguistic factors that
make judgements ambiguous, and that if we only filter these factors out, what
is left must be clear-cut. This viewpoint I feel less inclined to agree with,
 the reason being that having filtered out all the ambiguating factors, what is
 left, still, need not be discrete. But I nevertheless appreciate the very
 constructive discussions I've had with many of the respondents holding this
view. Finally, there are those that shared
their personal experiences in dealing with these fuzzy judgements, and some
that offer explanations as to while this might be so. Thanks to all who
contributed, and enjoy the messages!

Wen-Chao Li
Lady Margaret Hall, Oxford University

******************************************************************************


Perhaps *context* is the key to both your native speaker questions.
Linguists have a habit of asking, "Is X right or wrong?", when X has not
been given in any context. The native speaker who is being asked the
question is then obliged to try and invent a context in her/his head, and in
doing so may be looking at X from a different point of view from the
questioner.

Context might even help with you poor/paw question. Try presenting the words
in complete sentences. It would be nice if you could present a sentence in
which /pO/ is ambiguous -- being able to be interpreted as either "poor" or
"paw", but I'm afraid I can't think of such a sentence.

I became aware of the importance of context when I started investigating
aspectual particles in Shanghainese. I found that asking native speakers
whether they would use X, Y or Z after the verb in a particular sentence
produced very contradictory results because of the lack of context. Now I'm
trying to pursue the subject further, taking a different approach.

Lance Eccles
Macquarie University
lance.ecclesmq.edu.au

In my last message to you I said it might be useful to find a sentence in
which poor/paw could be used ambiguously. Just an hour ago I heard a
sentence on the radio which almost met the criteria:
There's one law for whites and another lore for blacks.
The speaker had to spell "lore" after uttering the sentence so that
listeners would know his meaning.

*******************************************************************************


Your request for references about instances where native speaker judgements
about their language are demonstrably fallible brings to mind the notion of
"near" or "apparent" mergers. In such instances, native speakers believe that
they are pronouncing two (minimal pair) words identically, but reliable
acoustic differences can be discerned using instrumental phonetic techniques.
These situations have been studied extensively by Labov, starting with the
study of sound change in progress (with Malcah Yaeger and Rich Steiner) in
1972. The two most recent references are his new book on sound change
(Blackwell, 1994) and an article with Mark Karen and Corey Miller in the
newish journal Language Variation and Change in 1991. In addition, I have a
paper (with Marianna Di Paolo), also in Language Variation and Change (1990).
John Harris in his 1985 book on Hiberno-English (CUP) has some discussion of
residual differences between the MEAT and MATE classes in Belfast vernacular;
words in these classes rhyme with each other in popular poetry, but
nonetheless are phonetically different.

Another, related strand, concerns final devoicing in German. Over the past
decade or so, there has been an ongoing debate, conducted mostly in Journal of
Phonetics, regarding the possibility of residual differences between
underlying voiced and voiceless obstruents in absolute final position. The
paper that I consider the most sound methodologically is by Bob Port and Penny
Crawford in 1989 (I believe); they show that there are differences in at least
some speech styles.

If these clues aren't enough for you to track things down, let me know and
I'll upload parts of the bibliography for a paper I'm currently revising on
how such near mergers could be acquired by speakers of a language.

Alice Faber
FaberYalehask.Yale.edu

******************************************************************************


Most people who have given this matter any thought realize that judgments
are fuzzy. It's discussed quite widely in the sociolinguistics
literature, and was much discussed in the 70s by R Quirk and associates
when they were setting up the Survey of English Usage.

When forced to make a judgment respondents may have recourse to notions
gleaned from school grammar, spelling, or just random prejudices.
Getting judgments is particularly difficult in non-standard varieties of
languages (see for example a paper by D Bickerton. 1977. Some problems of
grammaticality and acceptability in pidgins and creoles. In S Greenbaum
(ed) _Acceptability in Language_. The Hague: Mouton, 27-37). Fuzzy
judgments are responsible for the "not in my dialect" arguments that
circulate in theoretical linguistics. Again, all this has been discussed
long ago. You might like to look at R B Le Page & A Tabouret Keller
(1985) _Acts of Identity_ Cambridge:CUP, for a more recent review (start
with p9).

I have never understood why people expect judgments always to be sharp.
In some areas they are, but in areas where there is variation or where
there is ongoing change (as in your poor/paw example) there *is*
fuzziness. Without fuzzy edges there could be no variation or change.
Our acceptance of variation allows us to understand varying structures,
produce more than one variant ourselves, and allows for the possibility
of change in the individual and across individuals.

Language is not a rigid logical system but a part of human behaviour
which has variation and disorderliness, like all aspects of human behaviour.

_________________________________________________________________________
Anthea Fraser GUPTA

English Language & Literature
National University of Singapore
Kent Ridge e-mail: ellguptaleonis.nus.sg
Singapore 0511 telephone: (65) 772 3933
________________________________________________________________________

*****************************************************************************


I have just read your posting on native speaker judgements which I
find is a very important issue in linguistic methodology. You may want
to read an article by Chet Creider on constituent gapping in Norwegian
which is very explicit with respect to native speaker judgements. The
reference is

Creider, Chet A. 1986. Constituent-gap dependencies in Norwegian: An
acceptability study. In D. Sankoff (ed.), _Diversity and Diachrony_.
Amsterdam: John Benjamins. 415-424.

In my thesis on the use of space in Danish Sign Language, I have taken
up the same construction in (spoken) Danish:

Engberg-Pedersen, Elisabeth. 1993. _Space in Danish Sign Language:
The Semantics and Morphosyntax of the Use of Space in a Visual Language_.
Hamburg: Signum. Section I.5.2: Judgements of grammaticality, 25-30.

If you get other references, a summary on the Linguist List would be
very helpful, I think.

Sincerely,
Elisabeth Engberg-Pedersen

*******************************************************************************


 >Much of linguistic theory seems to be built upon the assumption that native
 >speakers can make clear-cut right/wrong judgements about their language.
 ...
 >One way to treat the problem [of unclear judgments-AH] would be to
 >relegate it to performance and sweep it under the carpet.

I'm not sure how many share that view about clear-cut right/wrong judgments.
Within the competence-performance view, every native speaker judgment is of
course a performance datum. You never have direct introspective judgments
about the grammar in your head. One can only extrapolate from the
performance data to whatever competence system it reflects. Informal
speaker interviews are "mini-experiments" (rather primitive by experimental
standards) and it's up to the researcher to interpret speaker responses as
"raw" data--always filled with more or less noise (depending on how good
the "experiment is controlled) from the performance system.

-Arild Hestvik

eepcphling.dk
IAAS
University of Copenhagen
Njalsgade 80
DK-2300 Copenhagen S.
Denmark

*****************************************************************************


Although I agree with you that grammaticality judgements have their
problems, I wonder if your example re poor/paw is ideal. In
particular, the speaker in question is typically aware of the spelling
difference, and you bringing the question of pronunciation to his/her
conscious attention. Have you read Labov on sociolinguistics? He has
all sorts of methodological tricks.

 -s

****************************************************************************


I saw your message on LINGUIST and felt compelled to respond. I work in
syntax/semantics/discourse analysis of Bulgarian and Russian and do a lot
of fieldwork. Native speakers are indeed funny things. My own methodology
is to use naturally occuring data (taken from contemporary prose) and fiddle
with it (i.e. change adverbs, etc) and offer a choice of two forms. I have
the speaker read the passage out loud and simply select the form that "sounds"
best. I have found that reading the passage out loud helps to derail the
metalinguistic censor (and thus get a speaker to accept a form that if asked
directly, he/she would reject). I also include a fair amount of context.
The difficulty with linguists simply presenting one with a sentence, out of
context is that the native speaker relies heavily on intonation in order to
make sense of it. Often a sentence isn't "wrong", but "unusual", or "funny".
Typically, "funny" sentences violate semantic or pragmatic constraints, rather
than a syntactic constraint. I follow Alan Timberlake in using the following
gradations: acceptable/preferred versus acceptable/not preferred versus
marginally acceptable/somehow odd versus unacceptable. These rankings also
follow a frequency axis since there is always a speaker who will accept a
form that everyone else has rejected.
Best regards,
Grace Fielder
University of Arizona
GFIELDERCCIT.ARIZONA.EDU

*****************************************************************************


Hi, there was a Master's thesis on the topic at the University of
Toronto, by Carson Schutze. You should be able to get hold of him by
e-mail at MIT. Chet Creider at the University of Western Ontario has
also done a study on Norwegian, showing that context influences speaker
judgments significantly. As for phonemic contrasts, luckily you don't
have to rely on speaker judgments, but can do measurements which show
whether or not a phonemic contrast has been neutralized. Regards, Regine.

****************************************************************************


I have a friend who holds degrees in physics and computer science. She is very
 interested in linguistics and at a certain point took at least one course in
it. Towards the end of one such course the instructor commented on the quality
 of her work and invited her to pursue a career in linguistics. My friend
begged off, giving as one of her reasons what might be called an insecurity
about grammaticality judgments. A native speaker of American English, she had
found during the course that too often when presented with a string of English,
 whether it was supposed to be acceptable or not she could convince herself
that it was at least in the sense that she could conceive of a discourse
context in which it would be perfectly acceptable, even unremarkable.

Speaking for myself, i consider myself a native speaker of English with a wide
variety of experience with different dialects and registers. And when asked
to make an acceptability judgment i find i have to distinguish between not two
categories 'acceptable' and 'unacceptable' but between three: 'actively
acceptable', meaning 'there are circumstances in which i would say that myself,
 even if on "my best linguistic behaviour"'; 'passively acceptable', meaning
'i don't know that i would use that construction myself, but i've come across
it in the usage of others'; and 'unacceptable', meaning 'i can't imagine any
native speaker of English, of any dialect known to me, ever allowing such a
construction'. Needless to say, the third category covers a relatively narrow
range of the strings thrown at me, since it includes a lot of 'self-evidently
ungrammatical' strings like '*Sam frightened Terry to Phil' (an example i used
in one of my introductory lectures), while the 'grey area' in the middle is
pretty broad, having to include Black English Vernacular and other
'substandard' usages.

In short, i am very much in sympathy with the concern you raise, and am
entirely in favour of it's being investigated.

Sincerely,
Steven
--
Dr. Steven Schaufele fcoswsnytud.hu
Room 119
Research Institute for Linguistics (Department of Theoretical Linguistics)
Hungarian Academy of Sciences (Eotvos Lorand University)
P. O. Box 19
1250 Budapest
Hungary

*** O syntagmata linguarum liberemini humanarum! ***
*** Nihil vestris privari nisi obicibus potestis! ***
*******************************************************************************


>(1) Being surrounded by linguists of all types, I am often given strange
>sentences in English or Mandarin Chinese (I am a bilingual speaker of these two
>languages) and asked if they would be acceptable in the language concerned.
>Very often I cannot answer yes or no, the sentences being strange yet not
>really wrong. I am able to give a rating for each sentence, or compare two
>sentences and say which is better, but to draw a line and say some are right
>and some are wrong, I believe, would be an abuse of intuition.

Same here. And on this topic I have an anecdote. That was around 1970-71
when I was doing fieldwork in Espiritu Santo. I was using the usual method
to try and figure out the phonology, grammar, syntax, whatever, of the
three or four languages on which I was gathering data. I was quizzing
Hilaire Chalet, who despite his French-sounding name, was full-blooded
native of Malakula, on this two native languages, when, suddenly,
he said to me: "Listen, Jacques, I am going to tell you: you must not
quiz me as you do because you confuse me. I no longer know. You must
listen to what I say the first time. If you ask me again, I no longer
know." He was perfectly right. If only I had been taught that principle,
I would have learnt a lot more of those languages a lot sooner.

Like Hilaire, I find that I get totally confused by certain marginal
sentences. Worse: having been asked about marginal sentences, I tend
to mark as incorrect sentences which I would normally mark correct
without the slightest hesitation. And vice versa. This practice is
indeed utterly unsound.

(2)
>

I have thought about it overnight and this is my explanation. Those test
sentences are stored in short memory, and processed there by a self-
learning "algorithm", a sort of neural net, perhaps. The odd-ball
sentences cause the algorithm to build putative ad-hoc models to
account for them. The run-of-mill correct sentences are also
processed in the same way, so that the correct sentences proposed
later are parsed by the tentative parser built during the
processing of the "dodgy" sentences. Accordingly, the dodgy parser
rejects the straight sentences. If you fed an informant enough dodgy
sentences for long enough, some of the temporary parser would
eventually be moved into long-term memory and provoke a collapse
of his grammar, and possibly a catastrophic (in the mathematical
sense of the term) mutation in his syntax.

******************************************************************************


I think that the proper object of linguistic research, or at least the
kind that I want to read and learn something from, is the vast body of
everyday, non-monitored usage. People's attitudes towards their usage,
e.g. judgments about their own pronunciation, are something I find about
as interesting as whether or not a person likes a certain movie or what
did they eat for breakfast. - I have lived in Seattle for almost 20 years
now but it still irks my Midwestern ears to hear people here saying "lahst"
(lost) and "kaht"(cot) = "kaht" (caught). When I try to make someone aware
of the difference, or when I quiz them on some (potentially) minimal pairs,
I find that some of them become hopelessly confused and keep changing
their minds---often moreso with the more literate types. I think it is the
job of a linguist to OBSERVE, not to put people on the spot.
---Jakob Dempsey, Asian Linguistics, UW.

*******************************************************************************


I agree with you that the notion that native speakers are infallible on
intuitive judgements of grammaticality is unsupportable, for a number of
reasons. 1. Labov has shown that often linguists make judgements that
fit their own theories, that substantiate what they wish to prove. 2.
Often linguists relegate the judgements of other people to the fact (?) of
their speaking "another dialect" without any notion of how dialects are
related, or what kidn of theory of dialectology they might have in their
heads. (I teach dialectology, and find that theoretical linguisics has
not served the field well; it raids it for judgements when it suits
people, and ignores dialects the rest of the time) 3. From the burgeoning
literature on GRAMMATICALIZATION we can see that often some construct is
in the process of being grammaticalized, so it is variable until it
becomes fixed; thus speakers can't make reliable judgements, but feel they
must. Linguists therefore make judgements based on their own intuition at
the moment; laypersons make judgements based on prescriptive grammar, and
judge the innovation wrong. 4. Language, especially non-standard
language, is inherently variable. Theoretical Linguists speak an
invariable idealized language, thinking of themselves as the ideal
speaker/hearer, for whom there are no fuzzy judgements. Who can believe
such people?

Hal Schiffman

******************************************************************************


Your query addresses a subject which is a standard lecture early in my
first year syntax course. (I don't have anything to say about phonetic
judgements specifically.)

There are a lot of issues here. First, as Chomsky admits, people do not
have judgements about grammaticality per se. All they know is what they
like and what they don't like. It is up to the linguist to ask questions
that will rule out all non-syntactic reasons (like truth, sensicality,
elegance, subversiveness, etc.) before concluding that a
negative judgement reflects ungrammaticality.

Second, when speakers "judge sentences" they are not judging abstractions
on purely formal criteria; they are judging the reasonableness of someone
uttering that sentence with some communicative intention. Even when
speakers think they are making that judgement in a "normal", "neutral", or
"null" context, they will differ on how they define that term. The rest of
the time they will vary even more widely, because they will vary, as
individuals, in how imaginative they are in constructing POSSIBLE contexts
in which uttering that sentence might make sense.

Third, once you go beyond the easy ("The farmer killed the duckling.") parts
of a description, distinguishing among competing hypotheses just about
necessarily involves you in getting judgements about unusual, often marginal
sorts of sentences. It should not be surprising that people have difficulty
judging these, and vary widely, and may be inconsistent. One
possibility is that their (internal) grammars do not give them any guidance.
(I think this possibility and some others are discussed in an article on
verb agreement by Jerry Morgan in the 1972 CLS volume.)

Georgia Green

******************************************************************************


There was a very thorough M.A. thesis written on
the psychology of linguistic judgments by someone
who was at the University of Toronto, about 5 years ago.
I've lent my copy to a friend who is now in Berlin,
and I'm not sure of the exact name of the author but I think
it was Carson Theodore R. Schutze or something like that.
Aha-- I see his name on the Linguist address list:
cschutzeathena.mit.edu.
He demonstrates a number of problems with the methodology
of using native speaker judgments as one's only data.
It's the only literature I know of on this topic, so I
think it's an important document.

Joyce Tang Boyland (jtangcogsci.berkeley.edu)

*******************************************************************************


I agree with you completely. Forced binary choices in grammaticality
inevitably falsify the picture of what real speakers actually do in
the real world.

Tom Cravens
cravensmacc.wisc.edu
cravenswiscmacc.bitnet

*******************************************************************************