Publishing Partner: Cambridge University Press CUP Extra Publisher Login

New from Cambridge University Press!

ad

Revitalizing Endangered Languages

Edited by Justyna Olko & Julia Sallabank

Revitalizing Endangered Languages "This guidebook provides ideas and strategies, as well as some background, to help with the effective revitalization of endangered languages. It covers a broad scope of themes including effective planning, benefits, wellbeing, economic aspects, attitudes and ideologies."


New from Wiley!

ad

We Have a New Site!

With the help of your donations we have been making good progress on designing and launching our new website! Check it out at https://linguistlist.org/!
***We are still in our beta stages for the new site--if you have any feedback, be sure to let us know at webdevlinguistlist.org***

Review of  Multimodal Epistemologies


Reviewer: Kunkun Zhang
Book Title: Multimodal Epistemologies
Book Author: Arianna Maiorani Christine Christie
Publisher: Routledge (Taylor and Francis)
Linguistic Field(s): Discourse Analysis
Pragmatics
Sociolinguistics
Cognitive Science
Issue Number: 26.2535

Buy
Discuss this Review
Help on Posting
Review:
Review's Editors: Malgorzata Cavar and Sara Couture

SUMMARY

“Multimodal Epistemologies”, edited by Arianna Maiorani and Christine Christie, is a new volume in the “Routledge Studies in Multimodality” series edited by Kay L. O’Halloran. The book aims to create “a new, comprehensive, more flexible and adaptable epistemology” (p. 1) for discourse analysts and scholars in communication studies to respond to fresh challenges of new media and technologies. This volume explores diverse frontier issues and presents new analytical frameworks and methods in the study of multimodal discourse, that is, discourse that combines more than one semiotic mode (including but not limited to written language, speech, gesture, image, colour, music, sound, and typography) to make meaning. It is divided into three sections (16 chapters) apart from an introduction and a conclusion. The three main sections are based on “three ways of interpreting multimodality as an approach to knowledge: (1) multimodality as a semiotic perspective; (2) multimodality as a tool for cultural research; and (3) multimodality as a way to analyse contemporary narrative processes” (pp. 1-2).

Section 1, “Multimodality as a Semiotic Perspective,” presents the first six chapters, in which different approaches and methods are drawn upon to investigate multimodal semiotic discourse. In Chapter 1, “An Eye-Tracking Account of Reference Points, Cognitive Affordance and Multimodal Metaphors,” Luna Bergh and Tanya Beelders measure 21 participants’ movement of eye and duration of fixation when they view 30 print advertisements to test their perception of reference points in multimodal texts, illustrating the importance of conceptual archetypes, reference points, mental spaces, and blending in viewers’ understanding of multimodal metaphors. Chapter 2, “Demotivators as Deprecating and Phatic Multimodal Communicative Acts” by Krzysztof Ozga integrates multimodal, semantic and pragmatic analyses to explore demotivators. A demotivator is a combination of a picture or photo with a caption that tends to ironically comment on the picture or photo. The author analyses the socio-communicative functions of demotivators and the visual-verbal relations in demotivators. In Chapter 3, “Legitimation in Multimodal Material Ensembles,” Giulio Pagani combines the multimodal and material analysis framework with the critical discourse analysis of legitimation (Van Leeuwen, 2007) to see how Norwich Bus Station is represented and legitimized, considering the social functions of the Station. The analysis covers the material items and forms of the building itself as well as the documentary texts recording the process of its construction. The congruity theory, which approaches the meaning of a text in terms of its pragmatic functions, is adapted in Chapter 4, “A Pragma-Semiotic Analysis of Advertisements as Multimodal Texts: A Case Study” by Sabrina Mazzali-Lurati and Chiara Pollaroli, to do a multimodal analysis of a Nike Air print advertisement. The paper investigates the rhetorical strategies this advertisement uses to persuade consumers and analyses the language, images and layout of the advertisement. Chapter 5, “Analysing Pictures: A Systemic-Functional Semiotic Model for Drawing” by Howard Riley, takes Halliday’s systemic functional approach and analyses how different resources for making meaning are chosen in drawing and fine arts to represent things, communicate with viewers and organize compositional elements. In Chapter 6 “Multimodal Advertisement as a Genre within a Historical Context,” Sonja Starc merges the text pattern theory (Hoey, 2001), social semiotic visual grammar (Kress and Van Leeuwen, 2006) and systemic functional Appraisal Theory (Martin and White, 2005) to probe into the advertisements in the early 20th century Slovene newspapers. The focus is on the interpersonal function of the advertisements, and their text structures as culturally shaped patterns.

Section 2, “Multimodality as a Tool for Cultural Research”, features five articles that discuss various cultural issues. In Chapter 7, “A Multimodal Analysis of the Metonymic Indexing of Power Relations in Novel and Film,” Christine Christie draws upon the Relevance Theory (Sperber and Wilson, 1995) and pragmatic theorisation of indexicality to do a multimodal analysis of power relations between characters in Kazuo Ishiguro’s fiction ‘The Remains of the Day’ and the film adapted from it. The author particularly illustrates the role of “banter,” a phenomenon of mock impoliteness that involves using acts of impoliteness to index intimacy and equality in communication. Chapter 8, “Re-Bombing in Memento: Traumata of Coventry, Belgrade and Dresden in Multimodal Collective Memory” by Jan Krasni, looks into the public memories of the bombing of three cities in different countries during the Second World War through multimodal analysis of websites in memory of these bombing events, combining the frameworks of Key Visuals (Kramer and Ludes, 2010) and Multimodal Discourse Analysis (Kress and Van Leeuwen, 2006). According to this article, collective remembrance serves as a common ground for identity construction in a given community. Chapter 9, “Argumentation, Persuasion and Manipulation on Revisionist Websites: A Multimodal Rhetorical Analysis” by Michael Rinn, takes a rhetorical approach to explore the multimodal revisionist websites that try to deny the Holocaust and restore the Nazi ideology. These websites use such strategies as argumentative manipulation, polyphonic communication (using pseudo-diversity of opinions to stir up controversy), and effective infotainment to deny the Holocaust and change the audience’s worldviews. Chapter 10, “A Corpus Approach to Semantic Transformations in Multisemiotic Texts” by Aleksandar Trklja, investigates the semantic changes of 500 modified versions of the American recruitment poster “I want you for U.S. Army” in terms of experiential, interpersonal and textual meanings. The author tries to provide a new approach for analyzing meaning transformations in multimodal texts. Chapter 11, “Multimodality and Illustrations: A Comparative Study of the English and Italian Illustrated First Editions of The Jungle Book by Rudyard Kipling” is written by Monica Turci. Using Kress and Van Leeuwen’s (2006) model of multimodal analysis, the author identifies the similarities and differences between two illustrations separately from the “Multimodal Source Text” (the English illustrated edition) and the “Multimodal Target Text” (the Italian illustrated edition) in the aspects of image composition, ideational meaning, and visual-verbal relationship. Then she connects the domestication of illustrations in the target text to historical and political contexts in Italy of the time when the translated text was published.

The final five chapters in Section 3, “Multimodality as a Way to Analyse Contemporary Narrative Processes,” study the narrative processes as well as textual cohesion and coherence in multimodal discourse. Using English-Italian parallel filmic dialogues as a corpus, Maria Freddi and Chiara Malagori in Chapter 12, “Discourse Markers in Audiovisual Translation,” investigate how pragmatic markers, including “well”, “so”, “now”, “you know/see”, “look/listen” and “I mean”, in English films are translated into Italian and what roles these pragmatic markers play. The contribution finds that translation by omission is a regular pattern in dealing with pragmatic markers in film dubbing. In Chapter 13, “Filmic Narrative Sequences as Multimodal Environments: A New Perspective on the Effects of Dubbing,” Arianna Maiorani takes systemic functional and Multimodal Discourse Analysis approaches to make a corpus-based comparative study of English filmic dialogues and their Italian dubbed versions, focusing on the space in film as a semiotic dimension. Transitivity patterns and perspective shifts are analysed and compared in the source text and target text to see whether the Italian audience is in fact watching a different sequence. In Chapter 14, “Multimodal Analysis of the Textual Function in Children’s Face-to-Face Classroom Interaction,” Roberta Taylor investigates the pupil-to-pupil conversations in a Year Five class in a British primary school. The pupils make use of different modes such as speech, gesture, and noise to make cohesive and coherent meaning through the strategies of repetition, omission, reference, intertextuality and so on. Chapter 15, “The Contribution of Language to Multimodal Storytelling in Commercials” by Sabine Wahl, is a multimodal investigation of a “mini-drama” television commercial, in which the product (a Mercedes-Benz engine) is promoted through a story. It first presents how the story is told without written language or speech but using the modes such as image, music, and sound effects. Then the author analyses the role played by the on-screen language. The language does not contribute to the storytelling but is indispensable for the advertisement, as it identifies the product by its name, for example. Finally taking a filmic extract as example, Janina Wildfeuer in Chapter 16, “Coherence in Film: Analysing the Logical Form of Multimodal Discourse,” analyses the process of filmic comprehension and interpretation based on textual qualities and structure on the one hand and knowledge of the world and film conventions on the other hand. The interpretation process is illustrated combining the formal, logical analytical framework (Asher and Lascarides, 2003) and multimodal film analysis approach (e.g. Bateman and Schmidt, 2011).

EVALUATION

This volume enriches the approaches and analytical methods in the study of multimodality and multimodal communication by presenting studies with both practical analysis and theoretical considerations. These chapters draw upon a variety of theoretical approaches (semiotics, discourse analysis, semantics, pragmatics, rhetoric, corpus linguistics, systemic functional linguistics, cognitive linguistics, film studies, and so on) to explore multimodal discourse in different areas (advertisements, websites, social media, films, architecture, drawing, literature, translation, face-to-face interaction, and so forth). Although the book is divided into three parts, the titles of the sections do not cover all the theoretical aspects of these chapters. For instance, Chapter 1 is included in the section on “Multimodality as a Semiotic Perspective”, yet the experiment and analysis in the chapter adopt a cognitive rather than semiotic perspective.

Indeed the book covers a wide range of fields of research. On the one hand, it enriches the studies of multimodality from the perspective of social semiotics, which is one significant theoretical foundation for multimodal analysis (Van Leeuwen, 2005; Jewitt, 2009; Kress, 2010). On the other hand, the book promotes the application of multimodal analysis in several new fields. Firstly, it continues the tradition of the multimodal approach to literature and narrative, which could be traced back to Page (2010). Secondly, it features the multimodal analysis of translation, including both traditional literary translation (Chapter 11), and audiovisual translation and film dubbing (Chapter 12 and Chapter 13). As audiovisual discourse involves multiple semiotic modes such as written language, speech, image, colour, sound, and music, it can be said that the translation of audiovisual “texts” provides a ready-made sample for cross-cultural multimodal discourse analysis.

This volume is innovative not only in terms of the research fields that it covers, but also in terms of integrated theoretical frameworks. The combination of multimodal analysis with critical discourse or cultural studies is displayed in Chapter 3, Chapter 6, and Section 2 “Multimodality as a Tool for Cultural Research” of this book. This theoretical integration responds to Van Leeuwen’s (2013) call for the “critical analysis of multimodal discourse” and Djonov and Zhao’s (2014) critical first step in establishing “critical multimodal discourse analysis” as a distinctive branch of discourse analysis. Particularly Chapter 3, “Legitimation in Multimodal Material Ensembles”, meets the trend of a multimodal legitimation (e.g. Van Leeuwen, 2014).

Another theoretical innovation of this volume lies in the editors’ effort in combining multimodality and pragmatics. This tendency is obviously demonstrated in Chapter 2, Chapter 4, Chapter 7 and Chapter 12. Christie (Chapter 7) remarks that the chapter also aims to “initiate a debate about what pragmatic approaches in general can bring to the analysis of multimodal texts” (p. 111). It seems, however, that this goal of combining pragmatics in general and multimodality has not been reached so far. Nevertheless, the debate promoted in this volume could prove beneficial to the combination of the two approaches.

In conclusion, with theoretical and methodological innovations, this edited volume offers its reader a range of recent studies in multimodal discourses of diverse kinds. These studies renew our knowledge, or in the editors’ term, “epistemologies” in multimodality and the rules played by multimodality in contemporary communication.

REFERENCES

Asher, N. and Lascarides, A. (2003). Logics of conversation. Cambridge: Cambridge University Press.

Bateman, J. A. and Schmidt, K-H. (2011). Multimodal film analysis: How films mean. London and New York: Routledge.

Djonov, E., and Zhao, S. (Eds.). (2014). Critical multimodal studies of popular discourse. New York: Routledge.

Hoey, M. (2001). Textual interaction: An introduction to written discourse. London and New York: Routledge.

Jewitt, C. (Ed.). (2009). The Routledge handbook of multimodal analysis. London and New York: Routledge.

Martin, J. R. and White, P. R. R. (2005). The language of evaluation: Appraisal in English. Basingstoke and New York: Palgrave MacMillan.

Kramer, S. and Ludes, P. (2010). Networks of culture: The world language of key visuals. Berlin: Li.

Kress, G. (2010). Multimodality: A social semiotic approach to contemporary communication. London: Routledge.

Kress, G. and Van Leeuwen, T. (2006[1996]). Reading images: The grammar of visual design. London and New York: Routledge.

Page, R. (Ed.). (2010). New perspectives on narrative and multimodality. New York: Routledge.

Sperber, D. and Wilson, D. (1995[1986]). Relevance: Communication and cognition. Oxford: Blackwell.

Van Leeuwen, T. (2005). Introducing social semiotics. New York: Routledge.

Van Leeuwen, T. (2007). Legitimation in discourse and communication. Discourse and Communication 1(1): 91-112.

Van Leeuwen, T. (2013). Critical analysis of multimodal discourse. In C. Chapelle (Ed.), Encyclopedia of applied linguistics (pp. 1-5). Oxford: Wiley-Blackwell.

Van Leeuwen, T. (2014). Multimodality and legitimation. Plenary speech delivered at the 2014 Australian Systemic Functional Linguistics Association Conference, University of New South Wales, Sydney.
 
ABOUT THE REVIEWER:
Kunkun Zhang is a PhD candidate at Macquarie University, Sydney, Australia. His research interests include multimodality, (multimodal) discourse analysis, (social) semiotics, literary stylistics, and systemic functional linguistics. Currently he has been engaged in exploring children’s literature across media, particularly the relations between narrative, media, multimodality, and literacies.

Versions:
Format: Hardback
ISBN-13: 9780415825238
Prices: U.S. $ 130.00