LINGUIST List 16.840
Sat Mar 19 2005
Review: Applied Ling/Cognitive Science: Karreman (2004)
Editor for this issue: Naomi Ogasawara <naomilinguistlist.org>
What follows is a review or discussion note contributed to our Book Discussion Forum. We expect discussions to be informal and interactive; and the author of the book discussed is cordially invited to join in. If you are interested in leading a book discussion, look for books announced on LINGUIST as "available for review." Then contact Sheila Collberg at collberglinguistlist.org.
Use and Effect of Declarative Information in User Instructions
Message 1: Use and Effect of Declarative Information in User Instructions
From: Umarani Pappuswamy <umarani+pitt.edu>
Subject: Use and Effect of Declarative Information in User Instructions
AUTHOR: Karreman, Joyce
TITLE: Use and Effect of Declarative Information in User Instructions
SERIES: Utrecht Studies in Language and Communication 18
Announced at http://linguistlist.org/issues/15/15-1633.html
Umarani Pappuswamy, Research Associate, Learning Research and Development
Center, University of Pittsburgh, USA.
BOOK'S PURPOSE AND CONTENTS
The book 'Use and effect of declarative information in user instructions'
is a PhD thesis by Joyce Karreman, promoted by Michaël Steehouder and
Peter Jan Schellens at the University of Twente, Netherlands. This study
investigates the effects of different information types in instructions
The work of Ummelen (1997) serves as the starting point for many
researchers studying reading and learning from texts and Karreman is not
an exception. According to Ummelen (1997), information in instructive
texts can be either procedural or declarative. Information types that are
directly related to the functioning of the product, for example, "actions"
that must be performed to get a product working, conditions for actions
and results from these actions are "procedural" whereas all other
(explanatory) information about the internal workings of the device,
trouble-shooting tips etc. are "declarative". Previous studies have shown
that adding declarative information in an instructive text helps to
improve the task performance (Kieras and Bovair 1984; Smith and Goodman
1984; Payne 1988) in a number of ways. The types of declarative
information used in these studies were: information about how a system
works; metaphorical and information about the interface (p.43). Karreman
extends research in this area with the goal 'to investigate whether
particular types of declarative information lead to specific positive
effects during the process of learning to work with a device' (p.3).
The study is presented in the form of five chapters.
Chapter 1 presents an overview of the literature on using instructive
texts. It describes two general theories on reading text: Van Dijk and
Kintsch (1983) and ACT* theory of Anderson (1983). Van Dijk and Kintsch
(1983) distinguish three models of mental representation, viz., surface
representation, propositional representation and situational model. After
a careful analysis of these three representations of text comprehension,
the author concludes that this theory cannot be directly applied for
describing the process of operating devices by using instructive texts.
However, the author finds the notion 'situational model' to be more useful
to the topic of her research. Karreman discusses ACT* theory on the
description of procedural learning in detail and finds that it is also not
an appropriate theory for explaining effects of instructive texts. In the
next three sections, the author provides various other theoretical
constructs that were essentially outgrowths of the two reading theories
mentioned above. Kieras and Bovair (1984) assume that instructive texts
are read as in Van Dijk and Kinstsch (1983). They also provide a
characterization of how suitable declarative representations are formed
based on the instructions read (which was not clear in Anderson's ACT*
theory). Karreman discusses their model of procedure acquisition and some
experimental results that threw light on the effect of declarative
information on task performance in depth. Payne (1988, 1991 and 1993)
criticized Kieras and Bovair (1984)'s model and Karreman finds some of his
comments to be a valuable contribution to the topic of effects of
declarative information in texts (p.22). Lastly, the author discusses
Dixon (1987, 1993)'s model of hierarchical planning framework with the
central assumption "mental plans have a hierarchical structure, in which
the task is described in general terms at the top of the hierarchy and
specified more precisely at lower levels" (Dixon 1993, p.377).
The author also presents a comparison of all the five theoretical
constructs in the latter part of the chapter with a reference to the
analytical data from experiments concerning declarative information in
instructive texts. The experiments are about the effects of adding
declarative information to procedural instructions; the effects of
declarative information compared with the effects of examples or results
of practicing. The chapter concludes with a table that neatly summarizes
the results of these experiments.
Chapter 2 describes an experiment which differs from Ummelen (1997) in
three aspects. Firstly, it tests if Ummelen's results could be extended to
another new system. Secondly, unlike Ummelen whose experiments focused on
the effects of declarative information in general, Karreman investigates
two particular types of declarative information: system and utilization.
System information is about the internal working of the product whereas
utilization information is about the reason, circumstances under which a
particular function of the system can be used. Thirdly, Karreman's
experiment uses 'reading times' in addition to Ummelen's 'using times'
(p.47). The methods, designs and results of the investigation are
discussed in detail. Justifications for selecting Ummelen's 'Click and
Read' method over the others are also provided. The results of this
experiment showed that system and utilization information are read during
the orientation phase but unfortunately they did not have any effect on
the task performance and knowledge. Hence, Karreman devised a follow-up
experiment that uses a combination of 'Click and Read' and 'Think Aloud'
methods. The results of this experiment also leave the question of absence
of any effect of declarative information on task performance.
Chapter 3 discusses another experiment and a follow-up study both of which
are designed differently from the studies described in chapter 2. The goal
here is to investigate the possible effects of cognitive information on
cognitive load, confidence and the appreciation of the instructions and
the device. The results demonstrate that declarative information in user
instructions does not affect the appreciation of the instructions or the
device but has a negative effect on the cognitive load and the confidence
of the users. This experiment also showed that declarative information
does not have clear effects on task performance. Liberal use of figures
and tables makes the ideas clear and interesting.
Chapter 4 investigates the effect of system information on the "transfer
of knowledge". The author presents a brief overview of previous research
on the theory of 'structure mapping' that discusses the role of analogy
and similarity in human cognition along with a review of some empirical
studies on this concept. Karreman prefers to use the terms 'structural
similarities and superficial similarities' (p.135) for her study to
investigate if system information about one device will result in a higher
degree of transfer when learning to operate structurally similar devices.
She conducts three experiments which are extensions to the series of
experiments conducted by O'Reilly and Dixon (1999, 2001 and 2002). Each
experiment is based on previous research on problem solving and
transferring knowledge to new tasks. The main finding is that order of
devices affected the degree of transfer (p. 167). Karreman reports the
observations of these experiments at length.
Chapter 5, the final chapter, summarizes the results of her experiments,
conclusions and discussions. It also provides answers for the three
research questions raised in the previous chapters. The questions are:
1. Are system and utilization information read?
2. Do system and utilization information affect knowledge, cognitive load,
confidence and appreciation?
3. Do system and utilization information affect task performance?
Her main conclusions are that declarative information has a weak effect on
task performance and had some unexpected effects on user's confidence in
performing a task. She outlines some directions for future research in
Lastly, the author presents a neat compilation of the references running
into seven pages.
This book is a good starting point for researchers interested in
information in instructional texts. The author provides a good
presentation of various methods and techniques used in investigating the
use and effect of declarative information. The language is clear and
effective and the text material is supported by tables and graphics
wherever needed. The chapters are arranged in such a manner that the
findings of one chapter serve as a stepping stone for the next one. Most
of the experiments reported in the chapters 2, 3 and 4 have been also
previously published by the author and her co-authors Steehouder and Dixon
(see the book for references). As pointed out by the author herself in
many places, her findings provoke several interesting hypotheses that need
further investigation from a cognitive point of view.
Though it was not within the scope of the study, the author could have
written a short section on potential guidelines as how to write useful and
effective instructive texts in general and what kind of system and
utilization information should be included in an instructive text. Such
guidelines could be of great help to user-manual writers.
The book has a minor flaw in typesetting which changes the meaning of the
sentence the author intended to convey: on page 132, in line 22, the
word "cannot" should have been " can not" ('These users cannot only
execute tasks that are explicitly formulated ... ; they can also ...").
Besides this, there are other minor presentation issues, for example,
repetition of hypotheses (shared between two experiments) on p. 152 could
have been avoided by simply providing reference to the list given on p.
142. The section 4.2 , titled 'First Experiment', goes on to describe
O'Reilly and Dixon's experiments before she sets hers own hypotheses for
the desired experiment (p. 138-142). This spoilt the readability of the
text at least for this reviewer who feels that this could be avoided by
moving this portion of the text to the subsection on literature review on
p. 137 and Karreman could have provided justifications for her motive to
build upon O'Reilly and Dixon's hypotheses. The book also lacks a subject-
index for reference purposes to concepts, techniques and domain-specific
vocabulary mentioned in the book.
Overall, this book should appeal to those interested in research on
learning from texts. It could also serve as a course material for students
designing experiments on information texts.
Dixon, P. (1987). The processing of organizational and component step
information in written directions. Journal of Memory and Language, 26, 24-
Dixon, P., et al. (1993). Effects of sentence form on the construction of
mental plans from procedural discourse. Canadian Journal of Experimental
Psychology, 47, 375-400.
Kieras, D. E. and Bovair, S. (1984). The role of a mental model in
learning to operate a device. Cognitive Science, 8, 255-273.
O'Reilly, T., and Dixon, P. (1999). Procedures are only skin deep: The
effects of surface content and surface appearance on the transfer of prior
knowledge in complex device operation. Proceedings of the Cognitive
Science Society Conference, 21, 486-489, Hillsdale, NJ:Erlbaum.
Payne, S. J. (1988). Metaphorical instruction and the early learning of an
abbreviated-command computer system. Acta Psychologica, 69, 207-230.
Payne, S. J. (1993). Memory for mental models of spatial descriptions: An
episodic-construction-trace hypothesis. Memory and Cognition, 21, 591-603.
Smith, E. E. and Goodman, L. (1984). Understanding written instructions:
The role of an explanatory schema. Cognition and Instruction, 1, 359-396.
Ummelen, N. ( 1997). Procedural und declarative information in software
manuals. Effects on information use, task performance and knowledge.
Van Dijk , T. and Kintsch, W. (1983). Strategies of discourse
comprehension. New York: Academic press.
(For other references mentioned in this review, please see Karreman's
ABOUT THE REVIEWER
Dr. Umarani Pappuswamy is a research associate at LRDC, University of
Pittsburgh, USA. She has a Ph.D. in Linguistics with specialization in
computational linguistics. Her main areas of research interests are:
computational semantics, intelligent tutoring systems, machine learning,
discourse analysis, typology, and corpus linguistics.
Respond to list|Read more issues|LINGUIST home page|Top of issue