Date: 12-Feb-2006
From: Fred L. Perry <flperryaucegypt.edu>
Subject: Re: 16.3460, Review: Applied Ling / Methodology, Perry (2005)
Date: February 9, 2006 From: Fred L. Perry, Jr. aucegypt.edu> Subject: Richard Todd's review of Research in Applied Linguistics: Becoming a Discerning Consumer.
I would like to thank Dr. Richard Todd for his review of my book in Linguist List, Dec. 04, 2005. His overview of the book was succinct yet complete. He highlighted the main points of the chapters, giving the reader a good understanding of what was covered. I applaud his clear presentation of what the book is about.
The next section of Todd's review, critical evaluation, is the part that interests me the most since it helps me to know where I might want to make changes or clarifications in future editions. First, he correctly recognizes that the book does not teach people how to do research but how to read and understand it. As emphasized in the preface of my book, this is exactly what I intended the book to do.
Unfortunately, many programs that include a research methodology course are forced to try to teach people both how to understand research and do research at the same time. Typically, due to time constraints, the former is crowded out by the latter. However, in our desire to cover all of the research methods - the list has become overwhelming - we leave our students a somewhat sketchy survey on how they are to carry out these various methods.
My question is, do our students need to know how to perform all of these various methods? Or, do they need to understand the principles underlying these different methods and how others take (or not) them into consideration when doing research? Achieving both would be nice, but we usually do not have sufficient time. When forced to choose, I prefer the latter, which was the primary motivation for writing the book.
My reasoning further goes that an individual graduate student usually needs only to focus on one or two research methods for his or her thesis. Having a working understanding of the foundations of good research, as presented in my book, allows him or him to become familiar with how a variety of research methods have been used in published research. Once s/he has chosen a specific area in which to do his or her research, s/he can select an appropriate method based on this familiarity and proceed to study the 'how to' for this method in greater depth.
Todd raises the issue of the book not mentioning some "less formal research approaches". However, for the reasons mentioned above, I did not think it necessary to give an exhaustive list of all the methodologies that are available. Rather, the issue is whether I have provided the needed foundations for reading research in applied linguistics with discernment, be it formal or less formal. If I have left out any foundational principles, I want to know about these to include in any future revision.
Another impression that Todd had was that the book focused too much on quantitative as compared with qualitative methodology. I can understand this to some degree because the typical research book makes clear divisions between these two research paradigms. However, this book does not draw these boundaries. Instead, I attempted to weave the material in such a way that foundations for both paradigms (qualitative and quantitative) are sufficiently represented for the novice reader to comprehend with discernment. Every chapter has material that relates to both paradigms. Research that is referenced in each chapter was carefully selected to illustrate both. Again, if there is any important foundational principle that has been missed for either paradigm which would improve a readers discernment across specific methodological procedures, I am eager to know for future editions.
Finally, due to the expressed purpose of the book, attention was given to some details that may appear to be 'nitpicking' to Dr. Todd. Nevertheless, the targeted reader is not an "experienced reader" but a novice and needs to be alerted to certain details that more advanced readers have automatized in their critical repertoire. Once they are aware of these, novice readers are able to move on to the big picture, only now with the ability to distinguish stronger from weaker research.
Todd continues by pointing out what he perceives as some minor annoyances (cf. p. 4 of 6). The first one relates to the study on page 83 which I used to illustrate exploratory research. Todd proposed that this study is not a good illustration of exploratory research because a hypothesis could be inferred in the study even though it was not explicitly stated. I agree with Todd that a hypothesis could most likely be inferred. However, the strict distinction that has been made in the research literature between exploratory vs confirmatory research is whether a hypothesis is explicitly stated in a study. If not, it is classified as exploratory. Yet, as pointed out in the book, research can be classified on a continuum between exploratory and confirmatory research. Where one plots a study along this continuum is determined by the degree a researcher insinuates there might be a hypothesis. On the other hand, all research, except for descriptive research, has an underlying hypothesis, i.e. the null hypothesis. The research question Todd referred to, "whether bilingualism influences the learning of a third language", has the underlying null hypothesis, "bilingualism does not affect learning a third language". In this sense, one might say that almost all research is confirmatory.
The second problem Todd addresses is on page 130. He correctly points out that correlation coefficients range from - 1.0 to + 1.0. This is clearly stated in pages 173 and 174. However, the subject of discussion on page 130 is reliability coefficients. Though they are often computed using some form of correlation coefficient, reliability coefficients range from 0.0 to + 1.0. The reliability of a data collection procedure ranges from no reliability ( 0.0 ) to perfect reliability (+ 1.0). There is no negative reliability.
The final problem that Todd perceives is that "Not all research using inferential statistics is trying to generalize" (p. 4 of 6). However, the word 'inferential' means that the results are being inferred (i.e., generalized) from a limited sample of data to a larger target population. Every statistics book I have ever read states this. However, as I mention in my book, what is being inferred related directly to the null hypothesis whether there is a stated research hypothesis or not.
Speaking of problems, there is a typo on page 211 of my book. The text should read 68% rather than 64%, which can be found in the middle of the page. I have alerted the publisher about this for future printings.
Having said the above, I want to thank Dr. Todd for doing something few reviewers do, that is giving the book to one of his MA students to evaluate. Though I value Todd's comments, the real litmus test of any book is the response of the targeted audience. My students have expressed similar impressions as did Nguyen Quang Tuyen. First, I apologize for the book binding. My students have had the same problem. I have notified the publisher, and I hope they can correct the problem. Second, regarding feedback on the assignments, I have been thinking of posting a website on which to post some of my students 'best' work for those like Tuyen to compare their work. Third, when writing the book, I pondered whether to discuss action research, but I decided at the time that since the book was not about 'how to do' but 'reading with discernment' I left it out. Again, I asked myself whether someone could read and critique published action research with the foundations I laid down and concluded that they could. Unfortunately, my literature search has located very little published action research for people to read. Fourthly, I apologize for building people's expectations about the amount of humor in the book. Unfortunately, I had some division between my reviewers regarding the humor, and my attempt to compromise left out some of the more humorous stuff. I need to play this down in coming editions. Lastly, I congratulate Tuyen for working through all of the exercises. This is no easy task; however, my students agree that they help apply the principles discussed at the time of reading.
Linguistic Field(s): Applied Linguistics
|