LINGUIST List 13.1577

Mon Jun 3 2002

Sum: Gestures and Speech

Editor for this issue: Marie Klopfenstein <marielinguistlist.org>

Directory

1. julia nikolaeva, gestures and speech

I am very grateful to all that have answered the
question about gestures.

Here is the summary for all interested in the
question:

>1) What are lexical correlates for beats (or batons,
formless gestures without an independent meaning): is
it simply a simultaneous word (for example a
conjunction at the beginning of a clause or an
adverb), or it's a phrase (NP or VP for example), or
in some cases perhaps a whole clause (when such a hand
movement precedes a clause)?

According to "Nonverbal Communication in Human
Interaction" by Mark L. Knapp and Judith A. Hall
(1997) (Holt, Rinehart &Winston ISBN 0-03-018023-6)
batons are representative of "nonverbal acts that have
a direct verbal translation or dictionary definition,
usually consisting of a word or two or a phrase." (p.
253).
Timo Sowa (University of Bielefeld) suggests that
there is no strong correlation between syntagmatic
structures and beat gestures, since beats usually
correspond (temporally) to one syllable and they may
be regarded as the gestural correlates to emphasis in
language. However, the emphasis may be expressed with
respect to a complex idea that unfolds in language in
a VP/NP or a whole clause.
Gale Stam (National-Louis University) thinks, that
beats are complex. They can occur with words, pauses,
and phrases. It depends on the kind of function they
are performing.

References:
E.A. Schegloff: "On some gestures' relation to talk".
In J.M. Atkinson, J. Heritage (eds.): "Structures of
social action", pp. 266-296. Camdridge: Cambridge
University Press, 1984.
U. Hadar: "Gesture and the Processing of Speech:
Neuropsychological Evidence". Brain and Language 62,
pp. 107-126, 1998.
These guys question the differentiation between
illustrative (iconic) and batonic gestures: P.
Feyereisen, M. Van de Wiele, F. Dubois: "The meaning
of gestures: What can be understood without speech?".
European Bulletin of Cognitive Psychology 8, pp. 3-25,
1988.

>2) Is there any explanation for the timing errors
between gesture and speech: very often the gesture
precedes affiliated speech, or a correct gesture may
be accompanied by speech errors?

As far as timing goes, it's important to use slow
motion with the sound intact to get exact
speech/gesture timing.
A gesture phrase must have a stroke(the part of the
gesture with meaning), but it can also have a
preparation, pre-stroke hold, post-stroke hold, and
retraction.
Shuichi Nobe in his article by "Where do most
spontaneous representational gestures actually occur
with respect to speech?� (in Language and gesture,
2000, ed. by David McNeill) argues that the claim
about gesture preceding affiliated speech is false,
and he "demonstrates that the seeming contradiction is
caused by methodological differences, not actual ones.
When the same methodology is applied to two different
data sets, the contradiction disappears. In fact, the
claim that gestures are more frequently initiated
during pauses, arises from an analysis comparing
gesture-to-pause and gesture-to-word rations. However,
this result was not corrected for the pause-to-word
ration, and thus gives a distorted view of
frequencies. In fact, in both data sets, only
approximately 1/4 of all gestures originated during
pauses." (quoted from the review, written by Fay
Wouk).
There's an article in the same book which discusses
the role of gesture in accessing lexical items. From
the same review: "two hypotheses have been presented
in the past to explain the role of gesture in
facilitating speaking, the Image Activation Hypothesis
and the Lexical Retrieval Hypothesis. Both theories
assume a gesture-to-language flow of information.
Sotaro Kita, in "How representational gestures help
speaking", presents a third hypothesis, the
Information Packaging Hypothesis, which argues that
two modes of thinking, analytic and spatio-motoric,
must be coordinated in order to produce an utterance.
Analytic thinking is then represented in speech, and
spatio-motoric thinking in gesture, and the
coordination between the two represents the successful
matching of the two types of mental representations.
The author shows how evidence given for the two
earlier hypotheses is compatible with IPH, and
presents two types of further evidence of the
interaction between the two modes of thinking,
mismatch between gesture and language among children
who are in the process of acquiring the concept of
conservation of liquid quantity, and the production of
gestures without vocalization and their later
recycling with appropriate vocalization, and sometimes
with modification of the gesture, in descriptions of
complicated spatial events. Unlike the article by
McNeil and Duncan, which assumes an integration of
gestures and speech into a single conceptual unit,
this approach assumes two parallel cognitive processes
which interact and exchnage information with each
other." The entire second section of that book (6
articles) is focused on the relationship between
gesture and thought.

Recommendations for reading:
D. McNeill (ed.): "Language and Gesture". Cambridge:
Cambridge University Press, 2000. (in particular Part
3)
U. Hadar, B. Butterworth: "Iconic gestures, imagery,
and word retrieval in speech". Semiotica 115 (1/2),
pp. 147-172.

>3) Why do hands sometimes keep their position during
the next clause, even if it does not concern the
meaning of the gesture?

It may have the function of "holding" an important
idea i.e. keeping the gesture space a person has
created with his/her gestures. It may be possible that
this space is re-used in subsequent clauses - that's a
point one can speculate about.
Sometimes, a gesture is maintained for cohesion across
clauses.

This reference is about gesture phases and it shortly
addresses their hypothetical function (refers at least
to other references):
S. Kita, I. van Gijs, H. van der Hulst: "Movement
Phases in Signs and Co-Speech Gestures, and their
Transcription by Human Coders". In I. Wachsmuth, M.
Fr�hlich: "Gesture and Sign Language in Human-Computer
Interaction: Proceedings of the Gesture Workshop '97".
Berlin, New York: Springer, 1998.

Below are listed recommended referents concerning the
question about gestures as a whole.
The most important books in this field seem to be
David McNeill's 1992 book "Hand and Mind" (University
of Chicago Press)
and "Language and gesture", 2000, ed. by David McNeill

The other references are:

Cienki, Alan (1998). "Metaphoric Gestures and some of
their Relations to Verbal Metaphoric Expressions." In:
Jean-Pierre Koenig (ed.), _Discourse and Cognition.
Bridging the Gap_. Stanford, California: CSLI
Publications, 189-204.

Corts, Daniel P. & Howard R. Pollio (1999).
"Spontaneous Production of Figurative Language and
Gesture in College Lectures." Metaphor and Symbol, 14:
2, 81-100.

Messing, Lynn S. & Ruth Campbell (1999) (eds.).
�Gesture, Speech, and Sign�. Oxford/New York: Oxford
University Press, reviewed by Zouhair Maalej for
LinguistList (2001) in "Review of Gesture, Speech and
Sign" (1999), edited by Messing, Lynn & Ruth Campbell
(eds.). Sign Language Studies, 2: 1, 116-131.


"The Nonverbal Communication Reader" by Laura K.
Guerrero, Joseph A. DeVito, and Michael L. Hecht
(1999), Waveland Press, ISBN 1-57766-040-4. Especially
interesting in it is a chapter about Hand Movements
(Kinesics) written by Paul Ekman and Wallace V.
Friesen.

"Nonverbal Communication" by Judee K. Burgoon, David
B. Buller, and W. Gill Woodall (1996), McGraw-Hill,
ISBN 0-07-008995-7 has a bigger section about kinesics
like this, including theories of origin. Besides it is
loaded with references.

Also were recommended the works by David McNeill
(1985, 1987, 1989, 1992, 2001); Butterworth and Hadar
(1989); Hadar and Butterworth (1997); Beattie and
Coughlan (1998, 1999); Beattie and Shovelton (1999).

A thesis by Jan-Peter de Ruiter about gestures and
speech, at the MPI for psycholinguistics, Nijmegen. He
wasn't concerned with beats, however

Also those interested in this field can visit the site
for McGill University: http://www.mcgill.ca At the top left
of the page, under "In brief" and dated May 1, you
will find a link to a page about the recent research
on acquisition of Rachel Mayberry, who has studied
gesture in stutterers.

P.S. Unfortunately, I have found none of these books
in Moscow public libraries (I'm a student of Moscow
State University), so special thanks for detailed
answers.
Also I'd love to know, are such books available in the
I-Net.

Julia Nikolaeva
Moscow State University

lis_juliayahoo.com