"Kissine offers a new theory of speech acts which is philosophically sophisticated and builds on work in cognitive science, formal semantics, and linguistic typology. This highly readable, brilliant essay is a major contribution to the field."
Date: Sat, 07 Jan 2006 22:52:16 +0000 From: Hiroshi Matsumoto <hiroshi.matsumoto@att.net> Subject: Second Language Research: Methodology and Design
AUTHORS: Mackey, Alison; Gass, Susan, M. TITLE: Second Language Research SUBTITLE: Methodology and Design PUBLISHER: Lawrence Erlbaum Associates YEAR: 2005
Hiroshi Matsumoto, Soka University of America
DESCRIPTION/SUMMARY
This volume aims to present an introductory and yet comprehensive survey of second language research methodology and design with novice researchers and graduate students as the main readers. It covers various data elicitation measures, research designs, and practical/logistical considerations essential for conducting well- controlled second language studies, including quantitative, qualitative, and classroom-oriented research. It also includes ethical issues for data gathering and a detailed checklist useful in submitting research for publication.
The book is intended to be used primarily as a textbook for introductory research methodology courses with graduate students as the main audience. However, it is useful as a resource book, as well, for more experienced researchers who want to enhance the quality of their research methods and designs.
Chapter 1, ''Introduction to Research,'' provides an overview of fundamentals in second language research, such as: (1) the definition and purpose of research (i.e., what is research?), (2) two major types of research (quantitative and qualitative studies) and their main characteristics, and (3) elements of standard research report format (title, abstract, introduction, literature review, statement of purpose, hypotheses, method, results, and discussion).
The chapter then emphasizes the importance of narrowing down and identifying research questions. After conducting extensive literature review, the research questions (except for so-called replication studies) need to be original and interesting, making sure that they have not been studied before.
In Chapter 2, ''Issues Related to Data Gathering,'' Mackey and Gass introduce important ethical issues related to the process of gathering date from human subjects. The main issues included in this chapter are (1) obtaining informed consent from the human subjects, (2) review process by the institutional review boards (IRBs), and (3) when withholding information toward the human subjects becomes necessary in second language research. Regarding (3), the authors explain that second language researchers can ''sometimes'' conceal their real interests and use some small deceptions. Informing the participants about the goals of the research, for instance, may change their actions during the research and thus lead to biased/unrepresentative research results. Withholding information may be permitted when all of the following three conditions are met: (a) incomplete disclosure is essential to the goals of the study; (b) no risks are undisclosed; and (c) participants will be provided with an opportunity to be debriefed when the study is completed.
Chapter 3, ''Common Data Collection Measures,'' focuses on presenting measures for eliciting and collecting second language data. The chapter shows seven major research paradigms in second language research and specific data elicitation/collection measures often used in each research paradigm. The seven major research paradigms the authors lay out are: (1) formal models of language research (that is, Universal Grammar/UG approach to studying the structure of linguistic forms; elicitation measures include acceptability judgments, elicited imitation, truth-value judgments and other interpretation tasks), (2) processing research (more psycholinguistic approach to investigating second language acquisition processes and mechanisms; its elicitation measures are sentence interpretation, reaction time, and moving window), (3) interaction-based research (that aims at studying learners' language interactions with others; measures include picture description tasks, spot the difference, and consciousness-raising tasks), (4) strategies and cognitive processes research (aiming at determining the strategies and cognitive processes second language learners tend to use; measures are observation and introspective measures), (5) sociolinguistic/pragmatics-based research (examining social and contextual variables that affect second language learning; its measures include naturalistic setting, elicited narratives, and role play), (6) questionnaire and survey research, and (7) existing date bases research. The authors explain that these paradigms are for the purpose of practical convenience and there is crossover with some measures used in more than one research paradigms.
Chapter 4, ''Research Variables, Validity, and Reliability,'' introduces basic concepts necessary for designing a second language research project, such as hypotheses, variable types, operational definitions (or operationalizations)of variables, validity, and reliability. Regarding variable types, the authors explain the notions of independent and dependent variables, moderator variables, intervening variables, and control variables. As for validity, fundamental concepts of content validity, face validity, construct validity, criterion-referenced validity, predictive validity, internal validity, and external validity are included.
The subsequent chapters, from Chapter 5 through 7, deal with various research designs and practical considerations in quantitative research (Chapter 5), qualitative research (Chapter 6), and classroom-based research (Chapter 7).
Chapter 5, ''Designing a Quantitative Study,'' explains practical considerations pertaining to three major design types in quantitative research: (1) correlational (associational) research design, (2) true experimental design (with random assignment), and (3) quasi- experimental design (without random assignment). The chapter shows that (A) a pretest/posttest design or (B) posttest-only design can be used for true experimental studies. For quasi-experimental research, (C) a repeated measures design can be used to overcome the problem of nonrandomization. When more than one independent variable is involved, whether in true or quasi-experimental studies, (D) a factorial design can be used to study the effects of multiple variables on a dependent variable. When the number of subjects is small, (E) a time-series design is useful. Finally, (F) a one-shot design is used in a quasi-experimental research to measure learner knowledge/behavior at one particular point in time.
Chapter 6, ''Qualitative Research,'' provides useful discussions on the nature/definition of qualitative research, major methods for collecting qualitative data, and practical considerations for conducting qualitative research. Regarding the nature of qualitative research, the authors explain that qualitative research is based on (a) descriptive data that does not make use of statistical procedures, (b) ''emic'' (or inside; vs. ''etic'' or outside) perspective, and (c) cyclical/open-ended (without prior hypotheses) processes.
The chapter stresses that the quantitative and qualitative data should be viewed as complementary means of investigating complex phenomena in second language acquisition. Then, the chapter explains the advantages and caveats for (1) ethnographies, (2) case studies, (3) interviews, (4) observations, and (5) diaries and journals. Then, it emphasizes the importance of cyclical data analysis (composed of the first round of data collection/data analysis without hypotheses, a hypothesis-formation stage, a second and a more focused round of data collection where hypotheses are tested, and a more refined stage in which a rich/full picture of the data can be obtained). The importance of methodological ''triangulation'' (using different research methods/measures to study a certain phenomenon) is also discussed.
Chapter 7, ''Classroom Research,'' addresses important research methods and practical/logistical considerations pertinent to classroom- based research. Classroom research takes place in the distinct context of ''classrooms'' (vs. laboratories). It requires judiciously selected and combined approaches rather than rigid adherence to one approach over another. Two methods useful for classroom research, which are explained in the previous chapters, are further elucidated with more detailed-information and examples: (1) observations and (2) introspective methods. For observation techniques, the chapter provides detailed descriptions of various observation schemes, such as the Target Language Observation Schemes (TALOS) and the Communicative Orientation of Language Teaching (COLT). For introspective methods, it explicates (a) uptake sheets, (b) stimulated recall, and (c) diary research in classroom contexts. Similar to Chapter 6, the chapter emphasizes the importance of methodological triangulation in carrying out classroom- oriented research.
The final three chapters of this volume discuss the issues of appropriate coding systems in processing research data (Chapter 8), analyzing quantitative data (Chapter 9), and reporting research (Chapter 10).
Chapter 8, ''Coding,'' explains about various aspects of data coding, including (1) transcribing oral data (for conversation analysis; transcribing conventions and technology), (2) coding for three different types of data (i.e., nominal, ordinal, and interval data), (3) coding systems, (4) interrater reliability, and (5) the mechanics of coding. Regarding coding systems, the chapter provides descriptions and examples of common/standard coding systems as well as more custom-made coding systems. For common coding systems, (a) t- units, (b) suppliance in obligatory context (SOC) counts, and (c) CHAT system are described. For custom-made systems, examples of systems made/used by different researchers for their own studies for coding (d) question formation, (e) negative feedback, (f) classroom interaction, (g) second language writing instruction, and (h) task planning are included.
Chapter 9, ''Analyzing Quantitative Data,'' presents an overview of introductory statistics commonly used for second language research, including (1) descriptive statistics (measures of frequency, measures of central tendency, and measures of dispersion), (2) inferential statistics (parametric and nonparametric statistics), and (3) correlation. Statistical techniques such as t-tests, Analysis of Variance (ANOVA), Analysis of Covariance (ANCOVA), Chi Square, and Mann- Whitney U/Wilcoxon Rank Sums are explained.
In Chapter 10, the authors show helpful tips for drafting sections that address discussion of results, limitations, directions for future research, and conclusions. They also provide a detailed checklist for researchers to consider in submitting research for publication. The checklist covers all the necessary sections of second language research, including the research problem and question, hypotheses, literature review, design, logistics, data gathering, data analysis, conclusions, references, final touches and formatting.
CRITIQUE/EVALUATION
As stated before, this volume is intended to be used as a textbook for introductory courses on second language research methodology with graduate students and novice researchers as the primary readers. To evaluate the volume's strengths/significance and limitations/issues, therefore, my critique/evaluation section takes a look at the following six aspects with reference to and some revisions of evaluation criteria provided by Brown (2001, p. 142). These components are all essential characteristics quality college and university level textbooks have to have:
(1) Scope (how thoroughly the intended scope is covered); (2) Sequence and organization (whether the chapters are organized and sequenced in a natural way); (3) Contents of each chapter, including currency of information/knowledge included and the quality of examples and illustrations; (4) Background of the readers and their needs (how much their needs are reflected in the volume); (5) Formatting (including general layout, tables of contents, chapter headings, glossary, and index,); (6) Goals and Overall quality.
First, regarding its scope, this volume presents a fairly comprehensive survey of second language research methodology and design, including a broad perspective on what research really is, ethical perspective on data gathering, common data elicitations measures, validity and reliability in research, practical considerations for well- controlled studies, data coding, data analysis, and tips for submitting research for publication. All these areas are covered extensively and in detail.
One important element or sub-area requiring a little more elaboration, however, seems to be a philosophical perspective on what research really is (Chapter 1). In Section 1.1 ''Different Types of Research,'' some more elaborated and yet relatively brief discussion might be added on the nature of second language research while focusing on its modes of inquiry.
The authors do show the major differences between quantitative and qualitative studies in Section 1.1. However, it seems that readers would benefit more by at first gaining a broad and philosophical perspective on modes of inquiry in all areas of research, not second language research alone.
Research or scholarly/scientific research needs to examine various phenomena, whether natural, social, or human phenomena, in a careful/systematic way. In natural sciences (i.e., physics, chemistry, biology, and meteorology), researchers seek new knowledge that can better ''explain'' natural phenomena by establishing causal relationships and/or ''predict'' by the notion of probability. In contrast, researchers in social sciences (i.e., psychology, sociology, anthropology, and ethnography), especially since 19th century, have been pursuing new knowledge that serves to not only explain but also help us ''understand'' various human/social phenomena. Social scientists had been attempting to examine various social/human phenomena in the same way as natural scientists did. However, Max Weber, well-know 19th century sociologist, advocated that social scientists need to ''verstehen'' (that is, to interpretively/hermeneutically understand) the contexts/reasons of social and human phenomena, as well. He emphasized that social/human phenomena are occasionally stimulus free and intrinsically different from natural phenomena
Again, some concise discussion as above would certainly help graduate students recognize the place of second language research within the entire realm of ''research'' and, thus, understand why the area of second language research requires not only quantitative but also qualitative studies.
Second, regarding its sequence and organization, the way many chapters of this volume are organized/sequenced seems natural and logical. The book starts with a broad perspective on what second language research is, covers ethical issues, elucidates methodological considerations in various data collection measures, further moves on to data analysis and coding, and in the final chapter revisits a broad (and yet more elaborated) perspective on what second language research is composed of. All the chapters seem to be organized with a relatively natural flow from one chapter to another.
Third, many chapters of this volume provide detailed discussions on the pertinent topics and components. In addition, various studies cited in the book are most recent and helps to provide readers with updated information/knowledge about the topics. Chapters 3, 6, and 7 on common date elicitation measures, qualitative research, and classroom-oriented research respectively are especially excellent chapters providing in-depth discussions and clear examples/illustrations.
Regarding the quality of examples of illustrations, however, the quality of Chapter 8, ''Coding,'' may be rated as average. Section 8.3.2 shows several examples of custom-made coding systems, including question formation, negative feedback, classroom interaction, second language writing instruction, and task planning. Descriptions of some examples and coding systems included in the section are not necessarily sufficient. It is a little difficult for readers to fully understand these interesting examples only with the book's descriptions.
Regarding the background of the readers and their needs, as a whole this volume provides adequate amount and level of discussions on second language research methodology well suited for the needs of graduate students and novice researchers. For example, Chapter 6 presents qualitative research methods, such as (1) ethnographies, (2) case studies, (3) interviews, (4) observations, and (5) diaries and journals. While explicating both the advantages and caveats of each method, the chapter helps readers become able to choose the most appropriate method that can address their specific research topic/questions well. Questions and skill-building exercises at the end of each chapter are also helpful for many novice researchers.
Fifth, regarding its formatting (including general layout, tables of contents, chapter headings, glossary, and index,), the volume is in general user-friendly and well-formatted (except very one small inaccuracy found on p.305, last line). Again, the readers' needs are well considered.
With all the above five aspects considered, the main goal of this volume to present a comprehensive survey of second language research methodology and design to novice researchers and graduate students appears to be accomplished fairly well. In addition, especially because of the fairly comprehensive scope coverage and in- depth/quality discussions in many chapters, I recommend that this book might be read by more experienced researchers, as well, as a source of new ideas and inspirations as they always need insights and clues for polishing their research further.
As Long (1980, 1985) and Matsumoto (1998; Chapter 3) emphasized, to show the directions second language research should pursue further, the maturity of one area of scholarly/scientific research (like psychology, sociology, and physics) may be measured by the extent of its methodological growth and rigor. From this point of view, I think many second language researchers are very pleased to see this volume and to realize that the quality of second language research methodology has reached this level and will continue to be refined further.
REFERENCES
Brown, D. (2001). Teaching by principles: An integrated approach to language pedagogy. White Plains, NY: Addison Wesley Longman.
Long, M. (1980). Inside of the ''black box'': Methodological issues in classroom research on language learning. Language Learning, 1, 1- 42.
Long, M. (1985). Input and second language acquisition theory. In S. M. Gass & C. G. Madden (Eds.), Input in second language acquisition (pp.377-393). Cambridge, MA: Newbury House.
Matsumoto, H. (1998). The relationship between various types of teachers' language and comprehension in the acquisition of intermediate Japanese. Lanham, MD: University Press of America.
ABOUT THE REVIEWER:
ABOUT THE REVIEWER
Hiroshi Matsumoto is an associate professor of second language
acquisition and pedagogy at Soka University of America, California.
His research interests include the relationship between various types
of teachers' language (or teacher talk) and comprehension, error
analysis and corrective feedback techniques for enhancing students'
speaking skills, Peak Learning Experiences and intrinsic motivation,
and teaching culture.
