Review of Multimodal Discourse Analysis |
|
|
| Review: |
Date: Tue, 1 Feb 2005 10:31:31 +1100
From: Len Unsworth
Subject: Multimodal Discourse Analysis: Systemic Functional Perspectives
EDITOR: O'Halloran, Kay L
TITLE: Multimodal Discourse Analysis
SUBTITLE: Systemic Functional Perspectives
SERIES: Open Linguistics Series
PUBLISHER: Continuum International Publishing Group Ltd
YEAR: 2004
Len Unsworth, School of Education, University of New England, Australia.
OVERVIEW
Multimodal Discourse Analysis is a book for the research community. It is
a collection of research papers concerned with developing the theory and
practice of the analysis of discourse and sites which make use of multiple
semiotic resources. New social semiotic frameworks are presented for the
analysis of a range of discourse genres in biology textbooks,
advertisements in hard copy and on television, film, websites, and three-
dimensional spaces such as museum displays, streetscapes, and buildings
such as hotels and the Sydney Opera House. In these frameworks the
analysis and interpretation of language use is contextualized in
conjunction with other semiotic resources that are simultaneously used for
the construction of meaning. The theoretical genesis of this work is
Michael Halliday's (1994) systemic-functional theory of language,
subsequently extended to images (Kress & van Leeuwen, 1996) and 'displayed
art' (O'Toole, 1994). The papers are organized into sections according to
the medium of the discourse: Part 1 is concerned with three-dimensional
material objects in space, Part II deals with electronic media and film
and Part III which contains investigations into print media.
CONTENTS
In Michael O'Toole's opening paper in Part I, 'Opera Ludentes: the Sydney
Opera House at work and play', a systemic- functional analysis of
architecture is used to consider in turn the Experiential, Interpersonal
and Textual functions of the Sydney Opera House and its parts, both
internally and in relation to its physical and social context. The paper
extends the usual definition of 'functionalism' in architecture beyond its
preoccupation with the Experiential components concerned with practical
purposes (theatre, stage, seats, lights etc) to the Interpersonal
(concerned with components reflecting power relations among groups of
users and its affective and interactive impact with viewers and users) and
the Textual (concerned with components that connect various aspects of the
building and also those that relate the building to its environment).
In chapter two, 'Making history in From Colony to nation: a multimodal
analysis of a museum exhibition in Singapore', Alfred Pang uses systemic
functional theory to outline a framework for the multimodal analysis of a
museum exhibition. The application of this framework is exemplified in
the critical analysis of particular displays in From Colony to Nation,
portraying Singapore's political constitutional history. Through his
analyses Pang shows how the multimodal representation of history in these
displays ideologically positions the visitor to a particular style of
imagining a nation.
Safeyaton Alias investigates, in the third chapter, the semiotic makeup of
the city in 'A semiotic study of Singapore's Orchard Road and Marriott
Hotel'. The author proposes a rank-scale framework for the functions and
systems in the three- dimensional multi-semiotic city. The analysis
presented here focuses on the built environment of Orchard Road and the
Marriott Hotel. Safeyaton discusses how these built forms transmit
messages that are articulated through choices in a range of
metafunctionally based systems. This paper discusses the intertextuality
and the discourses that construct Singapore as a city that survives on
consumerism and capitalism.
Chapter four by Anthony Baldry is the opening chapter in the second
section of the book dealing with film and electronic media. Baldry
describes the online multimodal concordancer, the Multimodal Corpus
Authoring (MCA) system, which provides new possibilities for the analysis
and comparison of film and videotexts. This type of concordancing
preserves the dynamic text, insofar as this is ever possible, in its
original form. The author shows the capacity of the concordancer to
facilitate the delineation of the multimodal features of types of the key
semiotic units of 'phase' and 'transition' in television advertisements
for cars.
In chapter five, 'Visual semiosis in film', Kay O'Halloran describes the
use of video-editing software, Adobe Premiere 6.0, to implement a systemic
functional analysis of the dynamic visual imagery of film and the
accompanying soundtrack. The technique is outlined using an account of the
analysis of the temporal unfolding of semiotic choices in the visual
images for two short extracts from Roman Polanski's (1974) film Chinatown.
The sixth chapter by Arthur Kok is entitled 'Multisemiotic mediation in
hypertext'. This paper formulates a working definition and outlines a
theoretical model of hypertext, which contains different orders of
abstraction. The author then provides a detailed analysis of Singapore's
Ministry of Education (MOE) homepage, using approaches based on previously
developed systemic-functional frameworks (Halliday, 1994; Kress & van
Leeuwen, 1996; O'Halloran, 1999; O'Toole, 1994). This includes some
examples and discussion of the process of intersemiosis, the interaction
of meanings across different semiotic instantiations.
In chapter seven Cheong Yin Yuen discusses 'The construal of ideational
meaning in print advertisements'. The author proposes a generic structure
potential for print advertisements that incorporates visual and verbal
components. Cheong also investigates lexicogrammatical strategies for the
expansion of ideational meaning that occur through the interaction of the
linguistic text and visual images. Through the analysis of five
advertisements, Cheong develops a new vocabulary to discuss the strategies
which account for semantic expansions of ideational meaning in these
texts; namely, the Bi-directional Investment of Meaning, Contextual
Propensity, Interpretative Space, Semantic Effervescence and Visual
Metaphor.
Chapter eight, 'Multimodality in a biology textbook' moves to the field of
education where Guo Libo investigates the multi- semiotic nature of
introductory biology textbooks. Drawing upon the work of sociological
studies of biology texts and following O'Toole (1994), Lemke (1998) and
O'Halloran (1999), this paper demonstrates the analysis of schematic
drawings and mathematical or statistical graphs in biology. The analyses
show how the various semiotic resources interact with each other to make
meaning. The paper concludes by emphasising the need to give due
attention to the visual as well as the linguistic in teaching English for
Academic Purposes to science and engineering students and to ensure that
students for whom English is a second language (ESL) are taught to
negotiate the comprehension and production of multimodal texts in English
in their discipline areas.
In the final chapter, 'Developing an integrated multi-semiotic model',
Victor Lim proposes a model that takes into account the independent
meanings made by language and images, and in addition, proposes a space of
interaction and integration where inter-semiotic processes for the
expansion of meaning take place. The discussion of mechanisms for
intersemiosis include 'homospatiality' and semiotic metaphor.
EVALUATION
This book's stated concern on page one of the Introduction is 'with
developing the theory and practice of the analysis of discourse and sites
which make use of multiple semiotic resources'. In this endeavour it is
innovative, illustrative, exploratory and challenging. The most
innovative aspects are the use of computer-based techniques for the
multimodal analyses of film and television advertisements described by
Baldry (chapter four) and O'Halloran (chapter five). Although work on
these techniques is in the early stages, what is particularly attractive
is their potential easy accessibility to the research community. Baldry's
Multimodal Corpus Authoring (MCA) system is an XML-based multimodal
concordancer specifically designed to identify recurrent patterns in films
and as an online tool so that the research and teaching community can
easily access it. O'Halloran's approach uses commercially available video-
editing software - Adobe Premiere 6.0. This software allows the user to
create multiple transparent mattes as overlays on the original film
footage so that text can be inserted and lines, vectors, figures, outlines
etc can be drawn 'on' the footage. In this way the researcher can mark
the semiotic choices and enter the image analysis through direct
engagement with the data. O'Halloran points out some of the practical
issues of density of analytic coding on the footage and discusses the
development of analysis protocols to ameliorate this.
The most interesting theoretically innovative aspects of the book are
those dealing with intersemiosis - the ways in which various semiotic
resources interact to make meaning. Pang explored the inter-semiotic
mechanisms between language and images in the museum display practice of
labelling. In so doing he applied Appraisal Theory (Martin, 2000) (which
is an account of the linguistic resources of English deployed to construct
evaluative stance in text) to a multimodal text, showing in some cases how
images functioned to influence judgment of the veracity of text
propositions, and correspondingly how in some cases the institutional
authority of source information in the text functioned to influence
judgment of the veracity of the images.
Cheong reasoned from systemic functional grammar to propose inter-semiotic
mechanisms based on transitivity and projection to account for aspects of
the 'bi-directional investment of meaning' between the images and text in
advertisements. The proposal that the image of a woman facing the viewer
is the sayer of the quoted product testimonial included above her picture
seems appropriate since the quote is not attributed to anyone else and
there is no image of any other likely potential sayer. However, the
mechanism based on transitivity entails grammatical analysis of a number
of inferred texts in order to show that a young woman depicted in an
advertisement is one of the people who already have a statement to make.
The text above her photo states that 'It (the car) doesn't make a
statement. It's for people who already have one.' It would seem desirable
to be able to base the account of the inter-semiotic mechanism on a more
direct analysis of the intermodal data, as above in the case with the
instance of projection. Here perhaps the possibility of an intermodal
cohesive hyponymic relationship between "people" and the image of the
woman might be entertained. There are number of other innovative aspects
of this chapter, including a significant contribution to the development
of descriptions of generic structure for print advertisements.
Guo provides an illuminating discussion of O'Halloran's (1999) concept of
semiotic metaphor through his account of the relationship between the main
text and a statistical graph in a biology textbook. He shows how the Head
and Postmodifier composite in the written text is transformed into two
separate participants in visual text, exemplifying the concept
of 'parallel semiotic metaphor'. He further indicates how the precise
shape of the curve of the visual text did not exist in the written text
and hence exemplifies a 'divergent semiotic metaphor'.
This aspect of intersemiosis was usefully extended by Lim's demonstration
of the concept of 'homospatiality'. This involves the expansion of
meaning through reinforcement when two different semiotic systems share
the same spatial coordinates on the Expression plane. His example is an
image where 'the linguistic text 'Snaaap', realized through the system of
Font in the Typography, shares the same spatial coordinates as the visual
image realized through the system in Graphics of the word breaking in
two.' Lim also discusses semiotic metaphor, exemplifying parallel
semiotic metaphor, but unfortunately without any examples of divergent
semiotic metaphor.
These and other chapters are also richly illustrative of applications and
adaptations of the work of O'Toole (1994) Kress and van Leeuwen (1996) and
others. A number of chapters importantly offer theoretical explorations,
often with the caveat that more work is required in their pursuit, hence
challenging the research community to engage with the issues. At the same
time readers will find some ideas in the book that need to be
problematized further. Kok, for example, asserts that 'hypertext is not a
semiotic resource' and excludes CD ROMs for standalone workstations from
his definition, and Lim includes a discourse semantics of visual images in
this 'integrative multi-semiotic model' without any comment about the lack
of theorization of this stratum. In a few cases superfluous information
might have been deleted. For example, Kok's introduction of the
term 'ergodist' for 'this choice-making individual who may follow
predetermined paths suggested by hypertext links which connect one webpage
to another, or alternatively, may forge his or her own path' (p.132), or
his apparently gratuitous description of what HTML is. Readers will also
find somewhat repetitious the rehearsal of the same point in the
introduction to many of the papers that a logocentric approach to
discourse analysis is no longer theoretically or practically viable in the
multimodal textual habitat in which we operate. While this remains an
important point to establish, it might have been useful to do this in a
framing introduction and then the chapters could simply acknowledge this.
The book also raises the very significant problem faced by writers about
multimodality when publishers determine that economic considerations in
production of the book preclude colour images.
REFERENCES
Halliday, M. A. K. (1994). An introduction to functional grammar (2 ed.).
London: Edward Arnold.
Kress, G., & van Leeuwen, T. (1996). Reading Images: A grammar of visual
design. London: Routledge.
Lemke, J. (1998). Multiplying meaning: Visual and verbal semiotics in
scientific text. In J. R. Martin & R. Veel (Eds.), Reading science:
Critical and functional perspectives on discourses of science (pp. 87-
113). London: Routledge.
Martin, J. R. (2000). Beyond Exchange: APPRAISAL Systems in English. In S.
Hunston & G. Thompson (Eds.), Evaluation in text: authorial stance and the
construction of discourse (pp. 142-175). Oxford: Oxford University Press.
O'Halloran, K. (1999). Interdependence, Interaction and Metaphor in
Multisemiotic Texts. Social Semiotics, 9(3), 317-338.
O'Toole, M. (1994). The language of displayed art. London: Leicester
University Press.
|
| |
ABOUT THE REVIEWER:
ABOUT THE REVIEWER
Len Unsworth is Professor in English and Literacies Education at the
University of New England in Armidale, Australia. A key interest is the
role of functional social semiotic analyses as a resource for literacy
research and pedagogy. Publications in this area include 'Literacy
Learning and Teaching' (Macmillan, 1993), 'Researching Language in Schools
and Communities' (Continuum, 2000/2005) and 'Teaching Multiliteracies
Across the Curriculum' (Open University Press, 2001). Two further books
are in press: 'e-literature for Children and Classroom Literacy Learning'
(Routledge) and [with Angela Thomas, Alyson Simpson and Jenny
Asha] 'Teaching Children's Literature with Information and Communication
Technologies' (McGraw-Hill/Open University Press).
|
|
| Versions: |
| Format: |
Hardback |
| ISBN: |
0826472567 |
| ISBN-13: |
N/A
|
| Pages: |
256 |
| Prices: |
U.K. £
75.00
|
|
| |
|
|
