LINGUIST List 25.4834

Mon Dec 01 2014

Review: Applied Ling; Computational Ling: Green (2013)

Editor for this issue: Sara Couture <saralinguistlist.org>


Date: 23-Apr-2014
From: Elizaveta Tarasova < elizaveta.tarasova1gmail.com">etarasovaipc.ac.nz, elizaveta.tarasova1gmail.com>
Subject: Statistical Analyses for Language Testers
E-mail this message to a friend

Discuss this message


Book announced at http://linguistlist.org/issues/24/24-1923.html

AUTHOR: Rita Green
TITLE: Statistical Analyses for Language Testers
PUBLISHER: Palgrave Macmillan
YEAR: 2013

REVIEWER: Elizaveta Tarasova, International Pacific College

Review's Editor: Helen Aristar-Dry

SUMMARY

There is a large number of textbooks currently on the market that aim to explain the basics of statistical analyses, yet only few of them focus on readers from the field of language testing. Rita Green’s recent book “Statistical Analyses for Language Testers” (SALT) sets an ambitious goal of providing language teachers and test developers with the knowledge and practical tools that can enhance their understanding of what contributes to the validity and reliability of a language test. The textbook is a practical introduction to the most useful statistical analyses for language test developers and researchers based on the programs IBM SPSS, Winsteps and Facets. The first of these is one of the most popular statistics software packages mainly because it is so user-friendly. The latter two programmes (as persuasively demonstrated by the author) complement SPSS outputs by providing opportunities for more detailed and complex analysis of data.

The 16 chapters of SALT cover a wide range of topics from the most basic to quite advanced. The chapters have a very user-friendly outline: each chapter focuses on a particular type of analysis and begins with an introduction that explains the value of this particular analysis to the reader. This is followed by a (very) brief explanation of the key terms and concepts that are used in the chapter. The description of each analysis is very detailed, with step-by-step instructions and an exhaustive translation and deciphering of outputs. Nevertheless, the explanations are not overloaded with information and only cover the details that are essential for understanding the core principles of the analysis and the results it yields. Each chapter ends with a task for further practice of newly gained skills, where the reader is referred to one of the Appendices (2-13) that contains additional data sets to practice the analyses, questions for the reader and answers for self-check. The book has an adequate table of contents, a list of the symbols and most common abbreviations (which the author calls acronyms) used in the book, a list of appendices as well as appendices with additional data for further practice, a reference list, a list of recommended sources for further reading and an index. The data files used in the text can be found at http://www.palgrave.com/language/salt/.

The textbook can be logically divided into two parts, with the first one (Chapters 1-9) discussing SPSS software and the second one (Chapters 10-16) dealing with Winsteps and Facets. The level of difficulty of the information presented in each chapter is indicated by symbols (the author uses pictures of boots), the more boots – the more difficult the chapter is going to be. Chapters 1 and 2 gently introduce the reader to the Statistical Package for Social Sciences (SPSS) by explaining what different types of variables there can be, how to enter the data and variable details as well as how to identify and deal with errors in the dataset. Chapter 3 concentrates on the peculiarities of item analysis. It starts with a brief description of some of the main operational concepts that are used in statistics, e.g. facility value, measurements of central tendency (mean, mode and median), range of scores, etc. This is followed by detailed instructions on how to perform the test and what the obtained results mean. Chapter 4 looks into descriptive statistics, covering such stages of the analysis as computing a variable, creating and interpreting a histogram and a bar chart and obtaining information in a non-graphic form, as well as comparing two distinct groups in the dataset. Chapter 5 covers the peculiarities of dealing with the data obtained from test taker feedback questionnaires and comparing the responses of the test takers with their performance on the test. Chapters 6-9 explore the concept of correlation in detail. Chapter 6 goes over the ways of comparing performances of a test taker on two different tests to see if there is a correlation between the results. Chapter 7 moves from descriptive statistics to inferential statistics and t-tests in order explain how to make generalizations about the relationships or differences that can be found between two variables. Chapter 8 concentrates on the ANOVA (Analysis of Variance) test, the test that allows for investigating the differences that are not due to chance between groups of test takers.

In Chapter 9 the reader is introduced to factor analysis, which gives a researcher an opportunity to see what factors may underlie the correlations demonstrated by the data. Chapters 10-13 and 14-16 give the reader an opportunity to get acquainted with Winsteps and Facets respectively. Chapter 10 explains how to create a control file from an SPSS data file and how then to create a convergence table. The next three chapters concentrate on the concept of fit statistics and some of the analyses that Winsteps has to offer, including item and person statistics and distracter analysis. These allow the test developer to see whether the data obtained from the test accurately reflect test takers’ abilities and whether the level of difficulty of the items comprising a test is suitable for the level of the test taker. Chapter 14 introduces the statistical program Facets and demonstrates how to work with a file using an SPSS data set. The final two chapters of the book inform the reader about the possibilities of the analyses in Facets, a programme that allows to concentrate on specific variables. The concept of quality control fit statistics is re-visited, and the explanations of peculiarities of an inter-rater agreement analysis is discussed.

EVALUATION

It is often remarked that language teachers are usually not very good at calculations, and the word ‘maths’ may sometimes cause us a bit of anxiety. At the same time, the requirements of today’s world put a lot of pressure on language teachers to ensure that when our learners’ skills and abilities are tested, they correspond to international standards. Developing tests for learners is part of a teacher’s everyday job; therefore, the knowledge of how to improve the quality of the tests and how to check whether the tests provide valid and reliable data on the students’ progress and/or performance is of utmost importance. SALT by Rita Green provides a concise and focused yet exhaustive and practical introduction to the use of statistical software packages that will be appreciated by language teachers and test developers, especially those who have little or no experience of dealing with statistics.

From the very first pages, the textbook strikes the reader as being applied and experiential, with little or no prerequisite knowledge of statistics required for grasping the concepts discussed by the author. I also found it appealing that the amount of theoretical input is limited to the degree necessary for understanding how things work and why they work this way. The author measures the amount of theoretical information that needs to be fed to the reader very carefully, and deliberately avoids the use of statistical symbols and formulas, so that even complete beginners do not feel intimidated. The author does not go too deep into the discussion of theoretical prerequisites rather concentrating on the practical matters. The list of references will provide those who are interested with enough reading for further self-study. All this reflects the author’s overall approach to how the statistical analyses should be treated, i.e. in terms of their use in the field of language testing. All the data that is used in the textbook comes from real tests that were developed for real purposes, which makes the text accessible, relevant to the intended audience (language teachers and test developers) and easy to follow for anyone who has ever set out to develop a language test, even though they may not have training in statistical analyses.

The outline of the book is easy to follow. Information about the key terms is presented in the form of bullet points and is complemented by the detailed analysis of test data, snapshots of pop-up windows and of tables with outputs, together with a meticulously detailed explanation of what the reader sees on the screen when they perform this or that type of test. The “learn-by doing” approach adopted in the book resembles a language drill, where the learner attempts to produce the target structure in a safe environment for the very first time under the strict control of the teacher. This gives the reader a clear understanding of how exactly to perform the tests and how to interpret the data, and also creates a learning environment where nothing can go terribly wrong because the instructor has taken care of everything.

There are only a few minor points for criticism, mainly of a technical nature. One of these is that the list of recommended further readings contains only three sources; it would be helpful to have a larger choice.

Another issue concerns the data files that the author uses in the book, or, rather the lack of explicit instructions on where to find them. For example, in Chapter 2 (p. 13) readers are requested to open the data file “Reading test data with errors”; however, there is no explicit reference to the location of this file. It took me some time to realize that there may be a supporting web page and then I had to look for it online. Once the webpage was discovered, I had no problem downloading the files and working with them, but it would be helpful to inform the reader about it.

As I mentioned above, the limited amount of theoretical input and the strong focus on technical details is very advantageous on the one hand. On the other hand, it can be viewed as a weakness, since the book may often sound like a language teaching-focused manual for using the software. It would be good to hear the author’s voice more often.

These reservations are very minor, and I can strongly recommend the book to all teachers and test developers who are striving to improve the quality of their tests. The book is especially useful for self-study and is very effective in achieving its goal of providing test developers with practical skills and basic knowledge of SPSS, Winsteps and Facets in their work.


ABOUT THE REVIEWER

Dr. Elizaveta Tarasova is a TESOL co-ordinator/lecturer at IPC, New Zealand. Her research interests include a wide range of subjects in Theoretical and Applied Linguistics. Having recently completed her PhD project on regularities in the formation of English N+N compounds, she is now working on preparing her research for publication.


Page Updated: 01-Dec-2014