LINGUIST List 31.1819

Tue Jun 02 2020

Review: Applied Linguistics: Rose, McKinley, Briggs Bafoe-Djan (2019)

Editor for this issue: Jeremy Coburn <jecoburnlinguistlist.org>



Date: 06-Mar-2020
From: Andrew Jocuns <jocunsagmail.com>
Subject: Data Collection Research Methods in Applied Linguistics
E-mail this message to a friend

Discuss this message


Book announced at https://linguistlist.org/issues/30/30-4526.html

AUTHOR: Heath Rose
AUTHOR: Jim McKinley
AUTHOR: Jessica Briggs Bafoe-Djan
TITLE: Data Collection Research Methods in Applied Linguistics
SERIES TITLE: Research Methods in Linguistics
PUBLISHER: Bloomsbury Publishing (formerly The Continuum International Publishing Group)
YEAR: 2019

REVIEWER: Andrew Jocuns, Assumption University

SUMMARY

This book is the latest in Bloomsbury’s series Research Methods in Linguistics and, as the authors note, this work is meant to be complementary to Paltridge and Phakiti's (2015) Research Methods in Applied Linguistics. The difference between them being that this work focuses upon data collection not research design. Each chapter is structured similarly with sections on: pre-reading tasks, key concepts, procedures and practicalities, improving validity and reliability, examples of studies that use that method, implications for researchers, post-reading activities, and resources for further reading.

Chapter 1, An Introduction to Research Methods covers some popular research design methodologies including: experimental and quasi-experimental research, field research, case study research, ethnographic research, survey research, and corpus and document research. The authors provide a nice discussion of quantitative and qualitative research including a discussion of mixed methods designs that incorporate qualitative and quantitative data collection. Table 1.1 on page 15 juxtaposes the differences of qualitative and quantitative research design, data collection and data analysis.

Data elicitation tasks are the focus of Chapter 2 and the authors note that data elicitation has been widely employed within applied linguistics research. Data elicitation tasks are divided into language production tasks (discourse completion, role play, storytelling, oral proficiency interviews) and experimental elicitation tasks (lexical decision, semantic categorization, grammaticality judgement). Key concepts include: task naturalness, task utility, task essentialness, modality, stimuli, fillers, counter-balancing and priming. The authors identify a number of issues that researchers who implement data elicitation tasks should keep in mind. For example, time constraints can be a factor in collection because usually only one participant can be interviewed at a time. In a subsection on stimuli for experimental elicitation the authors offer a number of websites that host word generators. The discussion on reliability and validation offers some suggestions for both language production and experimental elicitation. The chapter concludes with a discussion of seven applied linguistics studies which have used data elicitation with a focused discussion on Grey et al. (2015) and Galante and Thomson (2017), both of which implemented data elicitation in novel ways.

Introspective and retrospective protocols are taken up in Chapter 3, where introspection involves participants looking inward on their own behaviors (e.g. think aloud protocols) and retrospection involves a participant thinking about their behavior during a past activity. While the authors argue that such introspection is a valid protocol, they also suggest that one issue inherent in this method is memory deterioration. To overcome this, they offer some suggestions such as planning the data collection to occur as soon as possible after the event. Think aloud protocols involve the participant voicing their thoughts while performing a task and the authors note that a think aloud is useful for writing research. Retrospective think aloud protocols are used immediately after a participant is involved in a research task. Simulated recall protocols are when stimuli are used to enhance the participants recollection of events. Time constraints, researcher interference, reactivity of the protocol, eye-tracking and keystroke logging are discussed in terms of validity and reliability. Five sample studies are discussed at the end of the chapter as well as implications which focus upon how in order to accurately employ these types of methods researchers need to keep in mind factors that can affect the protocol in terms of the task being studied.

Chapter 4 covers the use of validated tests and measures in applied linguistics research where validated refers to the fact that there is an ample body of research that the test or measure does what it claims to do. Key concepts are broken down into three subcategories: types of tests and measures, validity, and reliability. The authors also discuss procedures and practicalities of choosing validated test and measures with an additional dialogue on whether or not to adopt or adapt them into one’s study. The data analysis is mentioned, in particular the focus on validity and reliability is a very thorough and worthwhile read for graduate students or novice researchers. Seven different studies are discussed with a nuanced focus upon Briggs’ (2015) study of vocabulary acquisition in study abroad which used Vocabulary Levels Test; and Saito et al. (2019) which used aptitude tests in a study of language acquisition.

The focus of Chapter 5 is on observations as a form of data collection and a number of important issues including the Hawthorne effect are discussed. Other important discussions here include: the role(s) of the researcher, self-observation, the importance of taking good field notes that involve thick descriptions, as well as observation types and schemes. Reliability and validity in observations are mentioned, which is on the surface dicey as most observations are conducted by lone researchers. The authors suggest overcoming this through having more than one observer. This is followed by discussions of the Hawthorne effect when participants alter their behavior due to being cognizant of being watched, as well as ways of improving validity and reliability. Seven studies are showcased in the case studies section, all of which used observation in different capacities, of note is Curdt-Christiansen's (2016) study of family language policy which used participant observations to examine and obtain detailed descriptions of family language activity in practice.

Interviews are the topic of Chapter 6 where an emphasis is placed on the need for researchers to take a reflexive stance in their conducting of interviews. Reflexivity in interviews is derived from Talmy's (2010) discussions of the interview as a form of social practice in which researchers refer to themselves and their social influences during interviews. A discussion of structured, semi-structured and unstructured interviews follows with discussion of the effects of time (follow-up interviews) and interviewer bias. The authors note that one of the biggest threats to interview reliability is a lack of reflexivity on the part of the researcher, noting also that social constructionism as a theory is another means of overcoming the lack of reflexivity. Eight case studies that involved different types of interviews were discussed but not in more nuanced detail. An interesting discussion develops at the end of the chapter with regard to how interviews are better perceived as discussion or conversations.

Chapter 7 draws our attention to the use of diaries, journals and logs as a form of data where the participants are asked to keep track of certain topics with regard to applied linguistics in their daily lives. This form of data, the authors also note, has been largely underutilized but allows participants to have agency and voice within the data. Different types of journals are discussed as well as how they should be designed for the participants. The discussion of reliability and validity emphasizes time and memory deterioration. Another point of not is how participants often have difficulty in writing in an L2, and to overcome this, researchers should allow participants to write in their L1 and to allow for flexibility in terms of entries in the journal through translanguaging and multimodality. Eight case studies are noted emphasizing Galloway and Rose's (2014) use of 1092 listening journal entries and Casanave's (2012) use of a journal that documented eight years’ worth of studying and learning Japanese.

Questionnaires are the focus of Chapter 8. The authors point out early on how novice researchers often align themselves with questionnaires under the mistaken belief that this method is simple, noting of course the complexity that is embedded with the design and implementation of questionnaires. The chapter covers a lot of information that we typically see in methods books on questionnaires (sampling, constructs, item writing, administration, piloting and data analysis) with a focus on applied linguistics. Of those I found the piloting discussion of particular importance for novice researchers. The discussion on validity and reliability covers some important topics such as: order effects, recall bias, and response rates. Six applied linguistics case studies are highlighted with an emphasis on the previously noted Briggs (2015) research on vocabulary learning in study and Teng and Zhang's (2016) study which was focused upon validating a questionnaire.

Chapter 9, Focus Groups, discusses the various ways of designing, sampling, constructing the participant makeup of, and moderating focus groups. Emphasized here is how often in text books on methods focus groups are lumped in as another interview type, when in reality they are a nuanced means of gaining understanding of the depth and breadth of opinions and practices among a group of people. The chapter includes an important discussion of choosing a moderator who should not be the researcher, as well as dealing with participants who are either too silent or the opposite; everyone’s voice needs to be heard and dominant voices need to be handled early on so that can happen effectively. Six applied linguistics studies that used focus groups are discussed with a focus on Galloway's (2017) study, which used focus groups to study the teaching of Global English to students in Japan, and Lanvers' (2018) study, which used focus groups as the primary data such that the data were representative of the target population. The authors provide a list of six good practices for using focus groups on page 192.

Chapter 10 is on the use of documents as a form of data collection from student writing to language policy documentation. One point of emphasis is on how documents are socially constructed. The authors mention systematic reviews which seek to evaluate the detailed findings of previous research and meta analyses which reanalyzes data. These are juxtaposed with narrative reviews which lack a clear research methodology. One interesting point the authors make is that quantitative analyses of documents often require qualitative analysis in order for the documents to have meaning. Document selection, quality, and types of methodology (quantitative, qualitative and mixed method) are also discussed. Eight case studies are discussed but none are focused exclusively in this chapter.

Constructing and using corpora is the subject of Chapter 11 where the authors effectively give us a crash course in corpus-based research. As someone who is mildly familiar with corpus-based research, I found this chapter helpful in thinking about and evaluating research that uses corpora. Types of corpora are discussed as well as representativeness, balance and size as key concepts. As far as procedures, the authors mention access as a key component of analyzing previously constructed corpora. The chapter also includes discussion on: transcription, processing text, software, marking up corpora, and how corpus analysis is conducted using statistical means. Six studies are offered as sample case studies that use corpora.

Chapter 12 is the final chapter and focuses upon strengthening data collection. Triangulation is emphasized where using two or more sources to validate data in order to have a comprehensive and accurate perspective on the object of inquiry. One of the most interesting parts of the book was the discussion in this chapter of replication and why studies are not replicated, noting that replication of studies or instruments is really an open avenue for applied linguistics research. It has never really occurred to me that there was a lack of replication of studies and I am glad the authors brought this up here. The chapter also offers a discussion of transparency in research noting how it is often the case that how they were conducted and problems faced, are often left out of journal articles or shelved in appendices. There are five case studies presented in this chapter and they focus on transparency and reflexivity in research as well as how technology was used in new ways, for example audio recording using wristwatches (Seals & Kreeft Peyton, 2017).

EVALUATION

The audience for this text would be graduate students in applied linguistics as well as novice and professional researchers. The book would be good for use in an applied linguistics methods course. One point that the authors make that is worthy of further discussion, and implementation, is the replication of previous studies. One of the issues here is that many qualitative studies such as open-ended interviews or ethnographic research, are research designs which are not really replicable to the degree that experimental or clinical studies are; however the methods used in data collection are. Overall this is a very detailed book that covers data collection in applied linguistics. The only weakness I note is where the authors discuss multimodality, which is it not always as straightforward as it seems. I have found this book helpful in thinking about how to advise graduate students on data collection in their research and plan to use some of its chapters in a methods course.

REFERENCES

Briggs, J. G. (2015). Out-of-class language contact and vocabulary gain in a study abroad context. System, 53, 129–140. https://doi.org/10.1016/j.system.2015.07.007

Casanave, C. P. (2012). Diary of a Dabbler: Ecological Influences on an EFL Teacher’s Efforts to Study Japanese Informally. TESOL Quarterly, 46(4), 642–670. https://doi.org/doi.org/10.1002/tesq.47

Curdt-Christiansen, X. L. (2016). Conflicting language ideologies and contradictory language practices in Singaporean multilingual families. Journal of Multilingual and Multicultural Development, 37(7), 694–709. https://doi.org/10.1080/01434632.2015.1127926

Galante, A., & Thomson, R. I. (2017). The Effectiveness of Drama as an Instructional Approach for the Development of Second Language Oral Fluency, Comprehensibility, and Accentedness. TESOL Quarterly, 51(1), 115–142. https://doi.org/10.1002/tesq.290

Galloway, N. (2017). Global Englishes and Change in English Language Teaching: Attitudes and Impact. Routledge. https://www.routledge.com/Global-Englishes-and-Change-in-English-Language-Teaching-Attitudes-and/Galloway/p/book/9780415786201

Galloway, N., & Rose, H. (2014). Using listening journals to raise awareness of Global Englishes in ELT. ELT Journal, 68(4), 386–396. https://doi.org/10.1093/elt/ccu021

Grey, S., Cox, J. G., Serafini, E. J., & Sanz, C. (2015). The Role of Individual Differences in the Study Abroad Context: Cognitive Capacity and Language Development During Short-Term Intensive Language Exposure. The Modern Language Journal, 99(1), 137–157. https://doi.org/10.1111/modl.12190

Lanvers, U. (2018). ‘If they are going to university, they are gonna need a language GCSE’: Co-constructing the social divide in language learning in England. System, 76, 129–143. https://doi.org/10.1016/j.system.2018.05.010

Paltridge, B., & Phakiti, A. (2015). Research Methods in Applied Linguistics: A Practical Resource.

Saito, K., Suzukida, Y., & Sun, H. (2019). Aptitude, experience, and second language pronunciation proficiency development in classroom settings: A longitudinal study. Studies in Second Language Acquisition, 41(1), 201–225. https://doi.org/10.1017/S0272263117000432

Seals, C. A., & Kreeft Peyton, J. (2017). Heritage language education: Valuing the languages, literacies, and cultural competencies of immigrant youth. Current Issues in Language Planning, 18(1), 87–101. https://doi.org/10.1080/14664208.2016.1168690

Talmy, S. (2010). Qualitative Interviews in Applied Linguistics: From Research Instrument to Social Practice. Annual Review of Applied Linguistics, 30, 128–148. https://doi.org/10.1017/S0267190510000085

Talmy, S. (2011). The Interview as Collaborative Achievement: Interaction, Identity, and Ideology in a Speech Event. Applied Linguistics, 32(1), 25–42. https://doi.org/10.1093/applin/amq027

Teng, L. S., & Zhang, L. J. (2016). A Questionnaire-Based Validation of Multidimensional Models of Self-Regulated Learning Strategies. The Modern Language Journal, 100(3), 674–701. https://doi.org/10.1111/modl.12339


ABOUT THE REVIEWER

Andrew Jocuns, PhD, is a sociolinguist who presently holds
a position in the English Language Teaching Program at
Assumption University of Thailand. His most recent research has included a multimodal discourse analysis of how students
perceive the linguistic landscape and geosemiotics of their
schoolscape and their perceptions of Thai English.



Page Updated: 02-Jun-2020