Most people modify their ways of speaking, writing, texting, and e-mailing, and so on, according to the people with whom they are communicating. This fascinating book asks why we 'accommodate' to others in this way, and explores the various social consequences arising from it.
Date: Tue, 1 Feb 2005 10:31:31 +1100 From: Len Unsworth Subject: Multimodal Discourse Analysis: Systemic Functional Perspectives
EDITOR: O'Halloran, Kay L TITLE: Multimodal Discourse Analysis SUBTITLE: Systemic Functional Perspectives SERIES: Open Linguistics Series PUBLISHER: Continuum International Publishing Group Ltd YEAR: 2004
Len Unsworth, School of Education, University of New England, Australia.
OVERVIEW
Multimodal Discourse Analysis is a book for the research community. It is a collection of research papers concerned with developing the theory and practice of the analysis of discourse and sites which make use of multiple semiotic resources. New social semiotic frameworks are presented for the analysis of a range of discourse genres in biology textbooks, advertisements in hard copy and on television, film, websites, and three- dimensional spaces such as museum displays, streetscapes, and buildings such as hotels and the Sydney Opera House. In these frameworks the analysis and interpretation of language use is contextualized in conjunction with other semiotic resources that are simultaneously used for the construction of meaning. The theoretical genesis of this work is Michael Halliday's (1994) systemic-functional theory of language, subsequently extended to images (Kress & van Leeuwen, 1996) and 'displayed art' (O'Toole, 1994). The papers are organized into sections according to the medium of the discourse: Part 1 is concerned with three-dimensional material objects in space, Part II deals with electronic media and film and Part III which contains investigations into print media.
CONTENTS
In Michael O'Toole's opening paper in Part I, 'Opera Ludentes: the Sydney Opera House at work and play', a systemic- functional analysis of architecture is used to consider in turn the Experiential, Interpersonal and Textual functions of the Sydney Opera House and its parts, both internally and in relation to its physical and social context. The paper extends the usual definition of 'functionalism' in architecture beyond its preoccupation with the Experiential components concerned with practical purposes (theatre, stage, seats, lights etc) to the Interpersonal (concerned with components reflecting power relations among groups of users and its affective and interactive impact with viewers and users) and the Textual (concerned with components that connect various aspects of the building and also those that relate the building to its environment).
In chapter two, 'Making history in From Colony to nation: a multimodal analysis of a museum exhibition in Singapore', Alfred Pang uses systemic functional theory to outline a framework for the multimodal analysis of a museum exhibition. The application of this framework is exemplified in the critical analysis of particular displays in From Colony to Nation, portraying Singapore's political constitutional history. Through his analyses Pang shows how the multimodal representation of history in these displays ideologically positions the visitor to a particular style of imagining a nation.
Safeyaton Alias investigates, in the third chapter, the semiotic makeup of the city in 'A semiotic study of Singapore's Orchard Road and Marriott Hotel'. The author proposes a rank-scale framework for the functions and systems in the three- dimensional multi-semiotic city. The analysis presented here focuses on the built environment of Orchard Road and the Marriott Hotel. Safeyaton discusses how these built forms transmit messages that are articulated through choices in a range of metafunctionally based systems. This paper discusses the intertextuality and the discourses that construct Singapore as a city that survives on consumerism and capitalism.
Chapter four by Anthony Baldry is the opening chapter in the second section of the book dealing with film and electronic media. Baldry describes the online multimodal concordancer, the Multimodal Corpus Authoring (MCA) system, which provides new possibilities for the analysis and comparison of film and videotexts. This type of concordancing preserves the dynamic text, insofar as this is ever possible, in its original form. The author shows the capacity of the concordancer to facilitate the delineation of the multimodal features of types of the key semiotic units of 'phase' and 'transition' in television advertisements for cars.
In chapter five, 'Visual semiosis in film', Kay O'Halloran describes the use of video-editing software, Adobe Premiere 6.0, to implement a systemic functional analysis of the dynamic visual imagery of film and the accompanying soundtrack. The technique is outlined using an account of the analysis of the temporal unfolding of semiotic choices in the visual images for two short extracts from Roman Polanski's (1974) film Chinatown.
The sixth chapter by Arthur Kok is entitled 'Multisemiotic mediation in hypertext'. This paper formulates a working definition and outlines a theoretical model of hypertext, which contains different orders of abstraction. The author then provides a detailed analysis of Singapore's Ministry of Education (MOE) homepage, using approaches based on previously developed systemic-functional frameworks (Halliday, 1994; Kress & van Leeuwen, 1996; O'Halloran, 1999; O'Toole, 1994). This includes some examples and discussion of the process of intersemiosis, the interaction of meanings across different semiotic instantiations.
In chapter seven Cheong Yin Yuen discusses 'The construal of ideational meaning in print advertisements'. The author proposes a generic structure potential for print advertisements that incorporates visual and verbal components. Cheong also investigates lexicogrammatical strategies for the expansion of ideational meaning that occur through the interaction of the linguistic text and visual images. Through the analysis of five advertisements, Cheong develops a new vocabulary to discuss the strategies which account for semantic expansions of ideational meaning in these texts; namely, the Bi-directional Investment of Meaning, Contextual Propensity, Interpretative Space, Semantic Effervescence and Visual Metaphor.
Chapter eight, 'Multimodality in a biology textbook' moves to the field of education where Guo Libo investigates the multi- semiotic nature of introductory biology textbooks. Drawing upon the work of sociological studies of biology texts and following O'Toole (1994), Lemke (1998) and O'Halloran (1999), this paper demonstrates the analysis of schematic drawings and mathematical or statistical graphs in biology. The analyses show how the various semiotic resources interact with each other to make meaning. The paper concludes by emphasising the need to give due attention to the visual as well as the linguistic in teaching English for Academic Purposes to science and engineering students and to ensure that students for whom English is a second language (ESL) are taught to negotiate the comprehension and production of multimodal texts in English in their discipline areas.
In the final chapter, 'Developing an integrated multi-semiotic model', Victor Lim proposes a model that takes into account the independent meanings made by language and images, and in addition, proposes a space of interaction and integration where inter-semiotic processes for the expansion of meaning take place. The discussion of mechanisms for intersemiosis include 'homospatiality' and semiotic metaphor.
EVALUATION
This book's stated concern on page one of the Introduction is 'with developing the theory and practice of the analysis of discourse and sites which make use of multiple semiotic resources'. In this endeavour it is innovative, illustrative, exploratory and challenging. The most innovative aspects are the use of computer-based techniques for the multimodal analyses of film and television advertisements described by Baldry (chapter four) and O'Halloran (chapter five). Although work on these techniques is in the early stages, what is particularly attractive is their potential easy accessibility to the research community. Baldry's Multimodal Corpus Authoring (MCA) system is an XML-based multimodal concordancer specifically designed to identify recurrent patterns in films and as an online tool so that the research and teaching community can easily access it. O'Halloran's approach uses commercially available video- editing software - Adobe Premiere 6.0. This software allows the user to create multiple transparent mattes as overlays on the original film footage so that text can be inserted and lines, vectors, figures, outlines etc can be drawn 'on' the footage. In this way the researcher can mark the semiotic choices and enter the image analysis through direct engagement with the data. O'Halloran points out some of the practical issues of density of analytic coding on the footage and discusses the development of analysis protocols to ameliorate this.
The most interesting theoretically innovative aspects of the book are those dealing with intersemiosis - the ways in which various semiotic resources interact to make meaning. Pang explored the inter-semiotic mechanisms between language and images in the museum display practice of labelling. In so doing he applied Appraisal Theory (Martin, 2000) (which is an account of the linguistic resources of English deployed to construct evaluative stance in text) to a multimodal text, showing in some cases how images functioned to influence judgment of the veracity of text propositions, and correspondingly how in some cases the institutional authority of source information in the text functioned to influence judgment of the veracity of the images.
Cheong reasoned from systemic functional grammar to propose inter-semiotic mechanisms based on transitivity and projection to account for aspects of the 'bi-directional investment of meaning' between the images and text in advertisements. The proposal that the image of a woman facing the viewer is the sayer of the quoted product testimonial included above her picture seems appropriate since the quote is not attributed to anyone else and there is no image of any other likely potential sayer. However, the mechanism based on transitivity entails grammatical analysis of a number of inferred texts in order to show that a young woman depicted in an advertisement is one of the people who already have a statement to make. The text above her photo states that 'It (the car) doesn't make a statement. It's for people who already have one.' It would seem desirable to be able to base the account of the inter-semiotic mechanism on a more direct analysis of the intermodal data, as above in the case with the instance of projection. Here perhaps the possibility of an intermodal cohesive hyponymic relationship between "people" and the image of the woman might be entertained. There are number of other innovative aspects of this chapter, including a significant contribution to the development of descriptions of generic structure for print advertisements.
Guo provides an illuminating discussion of O'Halloran's (1999) concept of semiotic metaphor through his account of the relationship between the main text and a statistical graph in a biology textbook. He shows how the Head and Postmodifier composite in the written text is transformed into two separate participants in visual text, exemplifying the concept of 'parallel semiotic metaphor'. He further indicates how the precise shape of the curve of the visual text did not exist in the written text and hence exemplifies a 'divergent semiotic metaphor'.
This aspect of intersemiosis was usefully extended by Lim's demonstration of the concept of 'homospatiality'. This involves the expansion of meaning through reinforcement when two different semiotic systems share the same spatial coordinates on the Expression plane. His example is an image where 'the linguistic text 'Snaaap', realized through the system of Font in the Typography, shares the same spatial coordinates as the visual image realized through the system in Graphics of the word breaking in two.' Lim also discusses semiotic metaphor, exemplifying parallel semiotic metaphor, but unfortunately without any examples of divergent semiotic metaphor.
These and other chapters are also richly illustrative of applications and adaptations of the work of O'Toole (1994) Kress and van Leeuwen (1996) and others. A number of chapters importantly offer theoretical explorations, often with the caveat that more work is required in their pursuit, hence challenging the research community to engage with the issues. At the same time readers will find some ideas in the book that need to be problematized further. Kok, for example, asserts that 'hypertext is not a semiotic resource' and excludes CD ROMs for standalone workstations from his definition, and Lim includes a discourse semantics of visual images in this 'integrative multi-semiotic model' without any comment about the lack of theorization of this stratum. In a few cases superfluous information might have been deleted. For example, Kok's introduction of the term 'ergodist' for 'this choice-making individual who may follow predetermined paths suggested by hypertext links which connect one webpage to another, or alternatively, may forge his or her own path' (p.132), or his apparently gratuitous description of what HTML is. Readers will also find somewhat repetitious the rehearsal of the same point in the introduction to many of the papers that a logocentric approach to discourse analysis is no longer theoretically or practically viable in the multimodal textual habitat in which we operate. While this remains an important point to establish, it might have been useful to do this in a framing introduction and then the chapters could simply acknowledge this. The book also raises the very significant problem faced by writers about multimodality when publishers determine that economic considerations in production of the book preclude colour images.
REFERENCES
Halliday, M. A. K. (1994). An introduction to functional grammar (2 ed.). London: Edward Arnold.
Kress, G., & van Leeuwen, T. (1996). Reading Images: A grammar of visual design. London: Routledge.
Lemke, J. (1998). Multiplying meaning: Visual and verbal semiotics in scientific text. In J. R. Martin & R. Veel (Eds.), Reading science: Critical and functional perspectives on discourses of science (pp. 87- 113). London: Routledge.
Martin, J. R. (2000). Beyond Exchange: APPRAISAL Systems in English. In S. Hunston & G. Thompson (Eds.), Evaluation in text: authorial stance and the construction of discourse (pp. 142-175). Oxford: Oxford University Press.
O'Halloran, K. (1999). Interdependence, Interaction and Metaphor in Multisemiotic Texts. Social Semiotics, 9(3), 317-338.
O'Toole, M. (1994). The language of displayed art. London: Leicester University Press.
ABOUT THE REVIEWER:
ABOUT THE REVIEWER
Len Unsworth is Professor in English and Literacies Education at the University of New England in Armidale, Australia. A key interest is the role of functional social semiotic analyses as a resource for literacy research and pedagogy. Publications in this area include 'Literacy Learning and Teaching' (Macmillan, 1993), 'Researching Language in Schools and Communities' (Continuum, 2000/2005) and 'Teaching Multiliteracies Across the Curriculum' (Open University Press, 2001). Two further books are in press: 'e-literature for Children and Classroom Literacy Learning' (Routledge) and [with Angela Thomas, Alyson Simpson and Jenny Asha] 'Teaching Children's Literature with Information and Communication Technologies' (McGraw-Hill/Open University Press).
