LINGUIST List 8.1602

Fri Nov 7 1997

Sum: ESL Proficiency testing

Editor for this issue: Anthony Rodrigues Aristar <aristarlinguistlist.org>

Directory

1. Laurie Zaring, Summary: ESL Proficiency testing

About a week ago I posted query concerning easy-to-implement testing
procedures for determining the approximate level of proficiency (oral and/or
written) of non-native speakers of English. I'd like to thank those who
replied for their extremely helpful information:

Christine Klein-Braley, he201kbunidui.uni-duisburg.de
Rebecca Larche Moreton, mlrlmsunset.backbone.olemiss.edu
Carsten Roever, roeverhawaii.edu
Chad D Nilep, chad.nilepasu.edu
Claus Steiger, Claus.A.Steigertzm.uni-giessen.de

Carsten Roever and Christine Klein-Braley both directed me to a couple of
Webpages where information on the C-Test is available. This is a test derived
from the Cloze test, but much more reliable. The first URL below provides a
fairly extensive overview of the procedure (though I had trouble connecting to
the page for some reason), and the second gives an extensive bibliography:

http://www.uni-duisburg.de/FB3/ANGLING/FORSCHUNG/HOWTODO.HTM
http://www.slf.ruhr-uni-bochum.de/biblio/ctbib.html

Claus Steiger directed me to two sites, both of which are interactive
test-yourself sorts of deals:

http://www.livjm.ac.uk/language/engtest.htm (choose the "Test your English"
link)
http://www.faceweb.okanagan.bc.ca/toefl/ (choose the "TOEFL & PREP" link)

Chad Nilep says that the TABE (Test of Adult Basic Education) battery has
proven reliable for him. This is a publicly available set of tests used for
GED (General Equivalence Diploma) placement in the U.S. He uses only two
portions of the test- the reading section, and the applied math section (which
consists of "word problems")--and finds that the grade-level equivalency these
tests give is an excellent indication of language ability viz. literacy. The
tests are extremely easy to administer (multiple choice "bubble" tests), but
have the great disadvantage of only testing literacy, not speaking ability. 

Finally, Becky Moreton sent a detailed description of the SPEAK test (though
she's not sure how exactly how it's spelled), which she has found to be quite
reliable for gauging oral proficiency. I include the entire text of her note
below.

Laurie Zaring
Reed College
*********************************************************************

There is a test called the Speak Test (it is pronounced that way; I am no100%
sure at this moment how it is spelled or what the letters are an acronym for)
which is widely used for this purpose. It consists of a taped component, which
the testee must listen to and respond to (you have two tape recorers going at
one time, one to play the test tape, the other to record what the person says),
plus a part that requires storytelling from pictures. I do not remember whether
there is a detailed reading/writing part; this is mostly for assessing
comprehension and production of spoken English. 

When I first heard that our university was going to administer this test, I was
extremely skeptical, for I was told by our testing people (general testing,
academic, vocational, psychological, everything) that the test could be
administered by non-specialists and scored by them, as well, with
very little training. I did not see how the results could possible be valid. 
So I "took" the test myself, so to speak, and also scored a number of the tests
taken by non-native-speaking undergraduates and graduates. Much to my
surprise, I concluded that the test is about as good as it could be. (My
background is as a linguist at the Foreign Service Institute, where I
administered long, interview-type tests involving a linguist and a native
speaker of Lao language, to Americans who had
studied Lao either with us or at proprietary or military language schools;
these tests are the original inspiration for the ACTFL tests you mentioned. So
I know I know what a good test of spoken proficiency is.)

When you are giving an FSI-type test, you are looking at the person's speaking
globally at every moment, pronunciation, morphology, syntax, lexicon,
comprehension, pragmatics, everything; the trick that makes the Speak Test work
is that the evaluator, as she listens to the tape made by the testee, is
directed to only one small point at a time. Any native speaker can assess one
phoneme (the terminology of course is not given in linguists' form, but in
laymen's) at a time, or can say whether a given phrase is in the right
word-order, or is responsive to the meaning asked for in a question. The
assessor just has to read the directions given for scoring the test and follow
them very carefully, for getting that she is a linguist for the moment; in
fact, it is the case that any reasonable intelligent native English speaker who
is willing to carry out the procedures carefully will get a useable test score
for the testee. 

The Speak Test takes, if I remember correctly, about 20-30 minutes max for the
student, who is alone in the testing room and just has to mash the START
buttons on the machines. It takes about the same length of time for the
assessor to do her work. So this is pretty quick to do. As to whether it
gives reliable results over the long run for any particular student, you know
how notoriously hard it is to say if so-and-so "really" speaks X language. Let
us say that this is about as good as you are goingto find for your purposes.

The Speak Test can be ordered from:

	TOEFL Publications
	P.O.Box 6161
	Princeton, NJ 08541-6161

	Tel. 609-771-7243 (for credit card buyers)

The items available (this refers to the 1996 version) are:

	Speaker-Rater Training Kit	$200

	Speaker Test, versions A
		and B			$100 each

	Examinee Practice Set		$100

You have to have the Training Kit and one version of the Test, in order to be
able to use the thing. Next year, you get the other version. The Examinee
Practice Set is not really necessary, I should think.