LINGUIST List 25.2394
Mon
Jun 02 2014
Diss: Text/Corpus Ling:
Hiippala: 'Modelling the Structure of a
Multimodal Artefact'
Editor for this issue:
Danuta Allen <danutalinguistlist.org>
Date: 02-Jun-2014
From: Tuomo Hiippala
<tuomo.hiippala
gmail.com>
Subject: Modelling the
Structure of a Multimodal Artefact
E-mail this message to a
friend
Institution: University of Helsinki
Program: Department of Modern Languages
Dissertation Status: Completed
Degree Date: 2014
Author: Tuomo Hiippala
Dissertation Title: Modelling the Structure of
a Multimodal Artefact
Dissertation URL:
https://helda.helsinki.fi/handle/10138/41736
Linguistic Field(s): Text/Corpus
Linguistics
Dissertation Director:
Eija Ventola
Dissertation Abstract:
This dissertation studied the structure of
multimodal artefacts, or how language, image
and other semiotic modes combine and interact
in documents. This places the study within the
emerging field of multimodal research, which
uses linguistic methods to study the
interaction of multiple semiotic modes. Despite
the growing amount of multimodal research, the
structure of multimodal artefacts has not
received the attention it warrants. Previous
studies have been either very detailed or
exceedingly abstract, leaving a significant gap
between data and theory, which this
dissertation attempted to bridge. To do so, the
dissertation adopted a data-driven approach to
multimodal analysis, addressing the structure
of multimodal artefacts, the factors that shape
the artefact structure, and the role of
structure in the recognition and interpretation
of the artifacts. The data consisted of tourist
brochures produced by the city of Helsinki
between 1967 and 2008, which allowed a
longitudinal perspective to their multimodal
structure. A total of 58 double-pages were
annotated for their content, visual appearance,
layout and rhetorical organisation, and
compiled into an XML-based multimodal corpus.
To study the corpus, the dissertation developed
visualization methods that combined information
from multiple analytical layers of the corpus
to represent the multimodal structures in the
data. The study revealed the functional
motivation behind the structure of the tourist
brochures, identifying patterns in their
hierarchical and rhetorical organisation, which
were used to fulfil specific communicative
tasks. The configuration of these patterns, in
turn, signalled how the brochure was to be
interpreted. The results also showed that after
the year 1985, which marked the introduction of
desktop publishing software, the organising
principles of the tourist brochures have
shifted towards a more fragmented and
non-linear structure.
Page Updated: 02-Jun-2014