LINGUIST List 5.855

Fri 29 Jul 1994

Sum: Linguists vs normals

Editor for this issue: <>

Directory

1. Richard Hudson UCL, Sum: linguists vs normals

Judgements by linguists vs normals

Many thanks to everyone who sent me comments and/or data:

 Paul Bennett, David Birdsong, Kittredge Cowlishaw, Lee
 Davidson, Ed Finegan, Michael Kac, Lynne Murphy, Colin
 Phillips, Michael Picone, Dennis Preston, Mary Ellen
 Ryder, Carson Schutze, William Snyder, Joyce Tang
 Boyland, Larry Trask

THE ISSUES

I haven't yet finished reading the long list of references
which I've collected (and listed below, with such comments as
I've been able to supply myself or by hearsay); but I have the
impression that the following are the main issues:

1. Are linguists special? The evidence for systematic
variation among judges is overwhelming, but is it related to
(a) specific training in linguistics, (b) experience of
working with language (e.g. by studying English literature),
(c) general level of education, (d) some kind of psychological
`set' such as +/- `language-bound' (as suggested by Labov
1979), or (e) verbal IQ (my contribution - not discussed as
far as I know), or .....?

2. If linguists are special (even if it's only because of
being highly educated), should we take their judgements more
seriously, or less seriously, than those of normals? The
obvious danger is of being biased by theory, but if this can
be avoided (how?) the case for taking linguists' judgements
*more* seriously than those of non-linguists seems pretty
strong. (I've changed my mind on this one since getting these
replies.) The literature is full of evidence that linguists'
judgements are much more sensitive to non-semantic, non-
pragmatic constraints than normals are (e.g. refs by Gleitman
& Gleitman and Ryder). Here's a nice example from a primary
school language lesson, by courtesy of Kittredge Cowlishaw who
saw it on a UK Open University programme. The children were
trying to construct sentences using phrase cards. Re /at night
/ two cats / played / with a kite/, the (non-linguist!)
teacher says: "That's obviously not a sentence; cats don't
play with kites."

3. A related issue is the effect of `satiation' (semantic or
syntactic), i.e. repeated thinking about the same examples,
which is obviously an occupational hazard of linguists. I got
four references on this from Joyce Tang Boyland, but haven't
included them here - no doubt she'd be happy to share them
with anyone interested (jtangcogsci.berkeley.edu); but I
couldn't resist the poster-summary that William Snyder sent
because it seems so directly relevant to my general worry
about how far I can trust my own judgements.

THE REFERENCES

Carden, G. (1970) On post-determiner quantifiers. Linguistic
Inquiry, 1. 415-428.

 An early attempt to show that judges vary
 systematically - NEG-Q dialects vs NEG-Q dialects re
 meaning of "All the boys didn't leave" - distinguished by
 judgements on "All the boys didn't leave until six" and
 "All the boys didn't leave, did they?" (see Labov 1972 below).

Elliott, D, Legum, S, and Thompson, S. (1969) Syntactic
variation as linguistic data. (Proceedings of) Chicago Ling
Soc 5, 52-59.

 Judgements by 21 linguists and 6 non-linguists on 2 sets
 of related sentences. For each set there was an
 implicational hierarchy for sentences x judges, but the
 judges were ranked differently for each set so there's no
 single rank of tolerance. Linguists and non-linguists are
 lumped together.

Fillmore, C, Kempler, D and Wang, W. (1979) Individual
Differences in Language Ability and Language Behavior.
Academic Press.

Gleitman, H. and Gleitman, L. (1970) Phrase and paraphrase:
some innovative uses of language. New York: Norton.

 Judgements on 3-word phrases as in G & G 1979, by 3
 levels of education (just school, just first degree,
 graduate students). Correlation between "illegal"
 syntactic parsings and lower levels of education.

Gleitman, H. and Gleitman, L. (1979). Language use and
language judgment. In Fillmore et al 1979, 103-126.

 Survey of evidence that judging sentences is harder than
 using them, and the lower the linguistic level, the
 harder the judgement is. Judgements by PhD students are
 "massively" different from those by clerical workers on
 judging e.g. "eat house-bird": PhD paraphrase it as `a
 house-bird who is very eat', clerical workers as
 `everybody is eating up their pet birds'[!].

Greenbaum, S. (1977) Acceptability in Language. Mouton.

------- and Quirk, R. (1970) Elicitation Experiments in
English: Linguistic Studies in Use and Attitude. Longman.

 More details about the tests reported in Quirk and
 Svartvik (below).

Labov, W. (1972) Sociolinguistic Patterns. Univ of
Pennsylvania Press.

 Pp. 193-8, critique (and rejection) of Carden's 1970
 conclusions re quantifier-dialects: different judgements
 are not consistent but are heavily influenced by context,
 plus evidence that linguists' theories influence their
 judgements.

------- (1975) What is a Linguistic Fact? Lisse: Peter de
Ridder.

 In general a (predictably) excellent survey of data-
 collection methods. Pp 15-16, enthusiastic summary of
 Spencer 1973, and survey of other related work which
 apparently shows that judgements, by linguists or by
 others, are not to be trusted.

------- (1979) Locating the frontier between social and
psychological factors in linguistic variation. In Fillmore et
al, 327-340.

 Nothing specifically to do with linguists vs normals, but
 interesting on individual variability (which may be
 related to what makes people become linguists?).

Lehrer, A. (1985) Markedness and antonomy. Jnl of Lingx 21,
397-429.

 A short appendix: 14 sentences containing measure phrases
 like "30 decibels loud" or "4 chroma bright" judged by 14
 physicists and 10 linguists; linguists were much more
 tolerant than physicists. Does this show that linguists
 are more liberal than physicists, or just that physicists
 know how to talk about measurements, and linguists are
 just guessing?

Newmeyer, F. (1993) Grammatical theory: its limits and its
possibilities. Univ of Chicago Press.

 p. 64-5, critique of Spencer 1973.

Paradis, C. and Deshaies, D. (1990) Rules of stress assignment
in Quebec French: evidence from perceptual data. Language
Variation and Change, 2.

 Judgements by both phonetically informed and non-informed
 students on perceived syllable stress in Quebec French.

Preston, D. = 22709MGRmsu.edu (1975) Linguist vs non-linguist
and native speaker vs non-native speaker: a study in
linguistic acceptability. Biuletyn Fonograficzny XVI, 5-18.

 Compares judgements of these four groups on sentences
 marked ungrammatical and questionable in Quirk et al
 1972.

------- (1989) Perceptual Dialectology. Foris.

 Summarises work on differences beteen folk and
 professional views of dialectology.

------- (1993) [title not known] International Jnl of Applied
Lingx, 3.2, 181-259.

 On folk views of US Black English based on discoursal
 evidence, plus mention of other work he has been doing
 recently on folk linguistics.

Quirk, R. and Svartvik, J. (1966) Investigating Linguistic
Acceptability. Mouton.

 `Elicitation experiments' with English and Geography
 undergraduates, who performed operations (e.g. change
 past to present) on 50 sentences, as well as judging
 their goodness. English students had only studied a
 little linguistics, and were only slightly different from
 the Geography students - tended to give more `minority'
 judgements and to be more successful in operations.
 (Contrast Snow and Meijer's finding that linguists are
 more consistent!)

Ross, J.R. (1979) Where's English. In Fillmore et al 1979,
127-163.

 Grammaticality judgements by 15 linguists and 15 non-
 linguists on 12 sentences. Claims to show that linguists
 give higher grades, but are less confident than normals -
 but differences are small and no significances are given.
 Entertaining. Sentences are in the `core', on the
 `fringe', or in a `bog' somewhere in between.

Ryder, M. = renryderidbsu.earn (1994) Ordered Chaos: the
interpretation of English noun-noun Compunds. Univ of
California Press (UCPL Vol 123).

 An investigation of 40 undergraduate non-linguist native
 speakers' interpretation of 100 novel noun-noun compounds
 in English (cf. Gleitman and Gleitman 1979), using a
 model based on schema theory. Some S's allow real-world
 plausibility to override even strong syntactic patterns.
 Same results emerge with two-word compounds e.g.
 "sweater-car" = `some sort of sweater' for 11/40 S's.

Schutze, C. = cschutzemit.edu (forthcoming) Stars and Gripes
Forever (?): Grammaticality judgments and linguistic
methodology. Univ of Chicago Press.

 General book on grammaticality judgements, including
 critique of Spencer for only using one linguist.

Snow, C, and Meijer, G. (1977) On the secondary nature of
syntactic intuitions. In Greenbaum 1977, 163-177.

 Intuitions are secondary in development, in pragmatics
 and in methodology, but they must be used and we need to
 understand their validity better. Experiments with 48
 Dutch sentences judged by 25 native non-linguists, 8
 native linguists and 8 non-natives. Linguists' judgements
 were much more reliable (consistent and uniform) than
 non-linguists', but they agreed on relative rankings.
 (Contrast Quirk and Svartvik.)

Snyder, W. = snyderpsyche.mit.edu (1994) Poster presented at
CUNY Human Sentence Processing Conference.

 Ungrammatical sentences involving classical subjacency
 violations ("Who do you wonder whether Mary likes?" "Who
 do you believe the claim that Mary likes?") become
 increasingly acceptable with repeated exposure, but
 various other (severe to mild) violations (that-trace,
 left-branch constraint) show no consistent tendency to
 improve or worsen with repeated exposure. Subject islands
 ("What do you think a bottle of fell on the floor?") show
 a weaker tendency to improve with repeated exposure,
 which is consistent with the view that these sentences
 combine a subjacency violation with some other type of
 island violation.

Spencer, N. (1973) Differences between linguists and non-
linguists in intuitions of grammaticality-acceptability. Jnl
of Psycholinguistic Research 2. 83-98.

 150 sentences that had been given as examples in
 linguistics papers were judged (in interviews) by 20
 linguistics graduate students, and by 43 psychology
 undergraduates. If more than half of a group give a
 different judgement from that of the original author,
 there is `disagreement'; 40% of sentences get
 `disagreement' from some group, 20% from both groups.
 Judgements are claimed to show that "naive and even non-
 naive [= non-linguist students or even linguistics
 students] can trust their intuitions, but linguists
 should not"! Very *small* differences between linguistics
 and psychology students. (I've only skimmed this paper,
 so I may have missed the point ...)

Valian, V. (1982) Psycholinguistic experiment and linguistic
intuition. In T. Simon and R. Scholes (eds.), Language, Mind
and Brain, Erlbaum.

 Why it's reasonable to use "expert" judgements instead of
 naive ones.

Dick Hudson
Dept of Phonetics and Linguistics,
University College London,
Gower Street,
London WC1E 6BT
(071) 387 7050 ext 3152