LINGUIST List 14.2040

Wed Jul 30 2003

Disc: Re 'Celtic Found to Have Ancient Roots'

Editor for this issue: Karen Milligan <karenlinguistlist.org>

Directory

1. Larry Trask, Disc: Re: 'Celtic Found to Have Ancient Roots'

- On Friday, July 25, 2003, (Linguist 14.2012) Peter Forster wrote:

> With respect to our paper, Forster and Toth "Toward a phylogenetic
> chronology of ancient Gaulish, Celtic, and Indo-European", available
> at
>
> http://www.pnas.org/cgi/content/abstract/100/15/9079
>
> Larry Trask made a number of critical comments which we fear will
> cause considerable confusion for the potential readers of our work
> (Linguist 14.1876).

> Both core issues and peripheral issues are raised by Larry Trask. For
> the record, some of the peripheral claims made by Larry Trask are in
> error (e.g., concerning publication procedures at PNAS,

At the beginning of all this, I was told by a colleague in the States
that PNAS is not peer-reviewed. But I now learn that I am out of
date. Another colleague tells me that, until not long ago, PNAS was
indeed not peer-reviewed, but that it has recently introduced some
kind of reviewing procedure. So, I will apologize for that
misstatement.

However, as several other observers have remarked, PNAS's refereeing
procedure leaves a great deal to be desired. To start with, it seems
clear that none of the referees knew anything about historical
linguistics -- even though the paper is expressly directed at
historical linguists.

But there's more. Recall the authors' arithmetic: 35 minus seven
equals 29. I noticed that on my first reading, but none of the
referees saw it. And there's another one. The authors describe their
'SV' character as "binary", meaning that it has only two states -- yet
they assign three states to it. None of the referees picked that up,
either.

I don't have the impression that the referees read the paper with the
kind of scrupulous care that we regard as normal in linguistics. So,
PNAS is peer-reviewed -- sort of.

> citation of the Pennsylvania group,

The authors cite a single paper by *one* of the Penn group -- Warnow.
But they give it only the barest mention in passing. And they don't
cite it in connection with tree-drawing procedures. Instead, they
cite it *solely* in connection with the possible reality of an
Italo-Celtic grouping within IE.

No reader of the article would gain the slightest inkling that the
Penn group exists, or that it has ever done any work on tree-drawing,
or that *anybody* has ever done any work on tree-drawing before
Forster and Toth. Doctor Forster is being disingenuous in suggesting
that I was in error when I accused him and his co-author of failing to
acknowledge the work of the Penn group.

While I'm here, I might add that there exists a second group, until
recently based at Cambridge University, just down the road from the
first author. This group has published several articles on techniques
for drawing linguistic trees, and it includes a biologist as well as
linguists. But the existence of this group and its work is likewise
not acknowledged.

This abject failure to recognize the existence of any earlier work on
the subject is not an appealing characteristic of the article, and it
is not the sort of thing I regard as normal professional behavior.

> Celtic substrate in Tuscany,

Eh? Is Doctor Forster telling us now that there *was* a Celtic
substrate in Tuscany? Most interesting. Sadly, every reference book
I have ever consulted fails to mention these intriguing Tuscan Celts,
and all of the books insist that the language displaced by Latin in
Tuscany was Etruscan. What is Doctor Forster playing at?

> networks versus trees,

I don't know what Doctor Forster has in mind here. But I might
reiterate one point: though the authors claim that their procedure
does not force trees at the expense of networks, examination of their
work shows that they do everything in their power to force trees,
including throwing away all the data that fail to produce satisfactory
trees.

> impact of mutation rates, etc.),

I don't recall saying anything about this at all.

> in other cases he is right
> (e.g., concerning the typographical error of "29"). Rather than
> dealing with these details here, we would encourage interested readers
> to peruse the paper, the Supplementary Information and the website
> Tutorial. For those of you who do not have access to PNAS, note that
> all materials are routinely made available on the PNAS website six
> months after publication, free of charge.
>
> The core criticisms concern the issue of negative controls, and the
> issue of resemblance coding versus cognation coding. As we have
> explained in our article and in the Tutorial, cognation coding is
> unfortunately not advisable with the fragmentary corpus of ancient
> Gaulish, because we would inevitably run the risk of ascertainment
> bias.

Right. Let's talk about this.

First, the object of the exercise is to construct a genetic tree --
that is, a tree showing which languages share common ancestry. Since
common ancestry is the topic of interest, then, I submit, the only
evidence which is relevant to the tree is evidence of common ancestry
- in short, cognation. Nothing else is relevant.

Second, Doctor Forster keeps harping on this theme of giving Gaulish a
"level playing field" with the other languages. But this is absurd.
Gaulish is extinct and sparsely recorded, and there is no way on earth
we can treat it on a par with abundantly recorded languages like
English and Latin. Pretending otherwise is nonsense. The authors
might as well try to provide a "level playing field" for Thracian --
total corpus, two brief inscriptions, neither of which we can read.

Third, in their flight from the monster of ascertainment bias, the
authors have fallen into a much greater error than they would ever
have experienced from ascertainment. Scrambling to avoid cognation,
they have resorted to a perfectly preposterous procedure -- subjective
judgements of resemblance -- which gives them utterly meaningless
results. Out of the frying pan and into the Great Fire of London, I'd
say.

> Incidentally, this by no means implies that we reject cognation
> coding in general: it is a powerful procedure where applicable,

I'll go further: it's the only procedure that makes any sense at all
when we're trying to draw genetic trees. And it is perfectly
applicable here, in spite of the authors' protestations. If it turns
out to be true that Gaulish is so poorly known that we can't identify
cognates with confidence, then we can't work on Gaulish, and
pretending otherwise by making wild guesses is nonsense.

> as we
> demonstrated for example in our first linguistic network paper on
> Alpine Romance languages (Forster et al. 1998). Accepting that we had
> to resort to error-prone resemblance coding for the Gaulish analysis,

Why "had to"?

Anyway, "resemblance coding" is not just "error-prone": it's wholly
meaningless. "Resemblance coding" works like this: Peter Forster, on
a given day, decides, for reasons known to no person alive, that <forn>
resembles <horno> but not <sorn>, and that <e> resembles <eta> but not
<y>. But, if he tried it again six weeks later, he might reach
different conclusions.

As I remarked earlier: this is supposed to be science?

> we needed to test our error rate using a negative control, for which
> we chose Basque. As expected, Basque demonstrated that the resemblance
> coding entails a noticeable error rate (about 5 spurious identities
> out of 35 characters), and we can expect a similar, but invisible,
> error for other language pairs in the table. That is the function of a
> negative control.

But the Basque data are terrible. Of the 35 Basque items reported, at
least 18 are wrong, with errors ranging from the trivial to the
catastrophic, and one or two more can be called into question.

If half the Basque data are wrong, what exactly is the significance of
"five spurious identities"?

More generally, when the total number of characters is only 35, of
which ten are ordinal numerals, and when seven of these (not including
any ordinals) are thrown away and not used, because the authors don't
like the effect they produce, then just how meaningful can a few
Basque items be? Why should we believe that any results for these few
Basque data are statistically significant? After all, with just a few
different choices of items, they might have picked up a number of loan
words from Latin and Romance (Basque has thousands), and their results
would have looked very different.

> As concerns the coding procedure, we have no disagreement with the
> statement that resemblance coding is much more subjective and
> difficult than cognation coding;

I repeat: resemblance coding is useless. There is no room for
subjective judgements in work of this kind.

The whole point is that the procedure must be explicit, objective and
principled at every step. A "procedure" that relies critically on
somebody's subjective judgements is of no value to anybody.

> indeed we explicitly made this point
> in the Tutorial, and we listed borderline cases in the
> article. Unfortunately, Larry Trask seems to go much further than this
> by implying in his examples that the coding procedure needs to be
> identical between rows of the table (i.e., between different
> characters).

I said no such thing. I said only that the procedure needs to be --
once again -- explicit, objective and principled. But the authors'
procedure is not explicit, it is not objective, and it is not
principled.

> This is not the case, as it would amount to comparing the
> proverbial apples and pears. The coding must only be consistent within
> a row (i.e. within a character), regardless of decisions in other
> rows,

I see. So, now Doctor Forster is telling us this. We can assign
states in the first row by throwing darts at a dartboard. We can
assign states in the second row with an equally random method. And so
on.

Doctor Forster would have us believe that it makes no difference at
all if our coding procedures are nonsensical and preposterous, so long
as we are consistent in each row. Am I supposed to take this
seriously?

> and it is up to the researcher to decide at which resolution to
> perform the coding for each character (e.g. whether or not to
> distinguish the Chicago and London pronunciations of "herb"). For
> tree-generating characters, the level of resolution will have a
> bearing neither on the branching order of the tree, nor on the time
> estimates.

A bold assertion. I suggest that working at this level of resolution
will quickly make wave effects very prominent, leading Doctor Forster
to begin chucking out huge masses of inconvenient data.

(By the way, in my earlier posting I dozily assigned wave theory to
the wrong Schmidt -- apologies).

In sum, Doctor Forster has still not shown us an explicit, objective
and principled procedure for converting raw data into trees. We have
seen nothing but arbitrary assertions based on wholly subjective
guesses, with no trace of any identifiable criteria.

One last comment. It appears to me that Forster and Toth are trying
to construct linguistic trees *without using any linguistics at all*.
Everything that linguists have ever learned about the languages under
discussion is dismissed by these authors.

In almost all cases, we know which words are cognate. But the authors
refuse to recognize cognation.

In most cases, we know which words have been borrowed by which
languages from which other languages. But the authors refuse to use
this information.

We often know which words are chance resemblances, or which words are
mama/papa words and therefore useless, or which words are imitative
and therefore useless. But the authors refuse to take advantage of
any of this knowledge.

Forster and Toth clearly believe that two hundred years of work in
historical linguistics is of no relevance to the business of drawing
linguistic trees -- although, mysteriously, they are willing to take
our word for it which languages are Indo-European.


Larry Trask
COGS
University of Sussex
Brighton BN1 9QH
UK

larrytcogs.susx.ac.uk