LINGUIST List 15.280
Sun Jan 25 2004
Review: Language Acquisition: Porte (2003)
Editor for this issue: Naomi Ogasawara <naomilinguistlist.org>
What follows is a review or discussion note contributed to our Book
Discussion Forum. We expect discussions to be informal and
interactive; and the author of the book discussed is cordially invited
to join in.
If you are interested in leading a book discussion, look for books
announced on LINGUIST as "available for review." Then contact
Sheila Dooley Collberg at collberg
linguistlist.org.
Directory
Louise Manga, Appraising Research in Second Language Learning
Message 1: Appraising Research in Second Language Learning
Date: Sun, 25 Jan 2004 19:38:19 -0500 (EST)
From: Louise Manga <louise.mangasympatico.ca>
Subject: Appraising Research in Second Language Learning
Porte, Graeme Keith (2002) Appraising Research in Second Language
Learning: A Practical Approach to Critical Analysis of Quantitative
Research, John Benjamins Publishing Company, Language Learning and
Language Teaching 3.
Announced at http://linguistlist.org/issues/13/13-2818.html
Louise Manga, Canadian Forces Language School, CFB Borden.
PURPOSE
This book is designed for students of applied linguistics and teaching
English as a foreign language (TEFL) and for practising language
teachers. It is intended to be the main textbook or a supplement to a
''research-techniques'' book for undergraduate and graduate students
so they can critically read and evaluate research. (The book assumes
the reader knows the basic principles of research and common
statistical techniques.)
The book examines the different components of a research paper and
demonstrates how to critically analyze a research paper using examples
of research in learning a second language. Readers are then helped to
do their own critical analyses by the guided samples in the workbook
section. Porte hopes that ''the experience of appraising in this way
will help the reader better to present their own work for publication
and peer evaluation.''
CONTENTS
The book consists of a Textbook (pp. 1-149) with four chapters:
Introduction, Method and procedures, Results, Discussion and
conclusions; and a Workbook (pp. 151-229) with worked out sample
appraisals and samples for guided appraisal. The book begins with a
Preface which introduces the aims of the book, presents the
methodology used in the book, and outlines the organization of the
book. It ends with a Glossary of key quantitative research terms and
three Appendices.
Text
The division of the Text into four chapters parallels the four basic
parts of a research paper. To help the reader evaluate the logic and
consistency in a research paper, Porte uses an ''awareness-raising
methodology'' whereby he asks readers to question, reflect on, predict
and summarize what they have read. Porte often summarizes what he has
said and relates it to material already covered or to be covered.
Throughout he introduces a topic by asking questions to get the reader
to think about the important issues and then he answers the questions.
Porte also provides a variety of examples from second language
learning.
Chapter 1 Introduction lays the foundation for the reader to better
evaluate the completed research. It covers: what to look for in an
abstract; how to find and evaluate the background to the problem, the
problem statement and the variables chosen; whether the review of the
literature is adequate theoretically and empirically for the
hypotheses and/or research questions formed; and what to look for in
research questions and hypotheses, the appropriateness of the
variables and if operational definitions are given.
Chapter 2 Method and Procedures deals with ''the nuts-and-bolts of the
research design'', i.e., the appropriateness of the methodology and
what kind of confidence it gives to the results. This chapter
considers subjects and materials, procedures, and research design and
data analysis in order to appraise the reliability and validity of the
results, and the replicability of the study. Porte discusses internal
and external validity, and reliability of the subjects and materials.
Issues of subject selection, group assignment, observation of subjects
and instructions given are discussed. Different types of research
design are described and as the researcher does not usually mention
what type was chosen, the reader must be able to determine the type
and its suitability. Porte also discusses the assumptions behind the
different types of statistical analyses.
Chapter 3 Results looks at how to assess the results of the research.
The chapter covers how the data might be presented, what the reader
should look for in the presentation of results to determine how much
confidence to place in the results of the study and the statistical
measures used. Porte describes when to use and how to interpret
commonly used tests in second language research: correlation,
regression, t-tests, analyses of variance and chi-squared.
Chapter 4 Discussion and Conclusions covers the quality of the
discussion and conclusions in the research paper. It shows how to
check that any conclusions are consistent with the results, and how to
evaluate any interpretations, generalizations or stated implications.
Workbook
The Workbook uses two fictitious sample research papers based on
quasi-experimental studies which would be representative of the type
of research in second language learning. Both papers are separated
into sections which correspond to the sections in the Text. The
section from sample 1 research paper has a full analysis by the author
and is followed by the corresponding section from sample 2 research
paper for the reader to analyze.
TEXT WORKBOOK
1. Introduction
1.1 Abstract Ii Abstract 1, 2
1.2 Problem Iii Problem 1, 2
1.3 Literature review Iiii Literature review 1, 2
1.4 Research questions Iiv Research questions 1, 2
2. Method and procedures
2.1 Subjects, materials IIi Subjects,materials 1, 2
2.2 Procedures IIii Procedures/Design
2.3 Design and analysis and analysis 1, 2
3. Results
Nature of findings III Nature of findings 1, 2
Correlation
Regression
T-tests
Analyses of variance
Chi-squared
4. Discussion and conclusions IV Discussion, concl. 1, 2
Beside each sample text there is a column on the left for the reader
to summarize the gist of each paragraph and a column on the right for
the reader to record any thoughts or questions that arise during the
reading of the text. (Note that only I and IV require the reader to
summarize the gist of the text.) The reader should then re-read the
text and answer a series of leading questions which are closely
related to the material in the corresponding textbook section.
Certain words or sentences in the sample text have been numbered and
there are observation questions related to these for the reader to
answer. Sample 1 in each part has been fully analysed: paragraphs are
summarized, thoughts recorded, questions answered and observations
answered. Sample 2 in each section is for the reader to complete.
Porte provides guidance by asking relevant questions with prompts for
helping to answer them, and with observation questions about words or
sentences in the text that he has numbered.
Appendices
Appendix I is a flow chart comparing what the research is trying to
discover, the related tests used and how to interpret the
results. This flow chart is ''reprinted from Hatch, E., and Lazaraton,
A. 1991, The Research Manual. New York: Newbury House Publishers,
pp. 544-545.'' Appendix II is a table comparing assumptions in
statistical tests. It is ''adapted from Brown, J.D. 1992. Statistics
as a foreign language: Part 2. Tesol Quarterly, 26, 4, pp. 629-664.''
Appendix III contains the statistical tables referred to in the text.
EVALUATION
I think this book fulfils its two fold aim of developing critical
readers and helping researchers do better research. It would be a
useful reference for classroom teachers who would like to do research
on teaching methods or materials.
Text
The Text has been organized to correspond to the major sections of a
research paper, which is an excellent way to present Porte's
methodology and to develop critical thinking about reading research
papers. The only problems I had with the Text were some typo errors
and the choice of grey for highlighting.
Errors
There are two serious typos. On page 51 line 17, ''Content validity''
is described as ''a more subjective and formal evaluation'' than
''Face validity''. It should read ''Content validity is a more
objective and formal evaluation''. On page 91 line 24, ''negative
correlation'' is described as ''(i.e., no linearity)''. It should be
described as ''(i.e., no positive relationship)'' or ''(i.e., one
variable decreases as the other increases)''. (Note that this error
is not in the Glossary.) Other typos are on page 48 line 10
''Materials - Internal and External Validity'' should be ''Materials -
Reliability''; on page 50 after line 21 and just before the questions
in bold, ''Materials - Validity'' should be inserted; and on page 41
line 17 ''be'' should be added after ''this kind can''.
Style
I question the use of grey highlighting for terms to be found in the
Glossary as grey is lighter than the rest of the text and doesn't
stand out. Grey highlighting was perhaps seen as an option as
sometimes words in the text are in bold (e.g., p. 110) and sometimes
underlined (e.g., pp. 16, 204). One glaring omission from
highlighting and the Glossary is in the section about the validity of
materials pp. 50-54. Four types of validity are described but only
three are in grey and in the Glossary. ''Predictive validity'' on
p. 52 line 1 should have been in grey and in the Glossary. There were
a few other errors with the use of grey highlighting. ''Kendall's
tau'' p. 109 was the only test mentioned that was not in grey nor in
the Glossary. There was one term, ''t crit'' (table on p. 115) that
was in grey but was not in the Glossary.
Workbook
This section is very thorough with good directions at the beginning.
The questions and answers reinforce the discussion in the Text and
help you to approach the research topic analytically. Each section of
the Workbook would best be done after reading the corresponding
section in the Text. It would be useful to have had the contents of
the Workbook listed in the Table of Contents so readers could see the
relationship of the Workbook to the Text before beginning. For those
interested in accessing more guided appraisals, Porte provides some on
his website (see Preface for his website). There are two typos:
p. 195, last line, first word should be ''due'' not ''down''; p. 196,
line 21, insert ''be'' after ''subjects are to''.
Glossary
This is a very useful part, however there is one glaring omission -
predictive validity. ''Predictive validity'' was not highlighted in
grey in the Text (see above) so was not included as a key term.
''Predictive validity'' should also be mentioned in the brackets at
the end of the terms ''Construct validity'', ''Content validity'',
''Face validity'', ''External validity'' and ''Internal validity''.
Also the key term ''Continuous data measurement'' should show
''(non-continuous)'' in the last sentence after ''A variable that is
not continuous'' since ''non-continuous'' is a key term.
Appendices
These are very appropriately included and will be very handy
references both when reading research papers and conducting one's own
research.
ABOUT THE REVIEWER
Louise Manga holds a Ph.D. in linguistics from the University of
Ottawa, Canada and is currently teaching EFL to adult foreign
nationals. Her interests include second language acquisition,
phonology, syntax, semantics and corpus linguistics. She has done
linguistic fieldwork on Inuktitut, including first language
acquisition. She has co-authored bilingual English-Mongolian books on
North American English pronunciation and on English grammar. She has
also prepared learning kits to help students with individual problems.