Publishing Partner: Cambridge University Press CUP Extra Publisher Login

Discussion Details

Title: Re: 16.1251, Disc: A Challenge to the Minimalist Community
Submitter: Carson Schutze
Description: Following up on Peter's point
  So the P&P parser that Sproat and Lappin envision would accomplish
  much more than comparable statistical parsers, which makes the
  proposed accuracy metric a poor yardstick for comparison

In addition to capturing the distinction between learnable and unlearnable
languages, P&P has as an important goal capturing the distinction between
well-formed (grammatical) and ill-formed (ungrammatical) sentences
within a language. As I understand it, the challenge demands only correct
parsing of grammatical sentences, not correct rejection of ungrammatical
ones. This represents another case where the P&P system, by virtue of the
goals of the theory, is being subjected to greater demands than the
statistical parsers.

Comp Ling isn't my field either, but I gather it is a desideratum for at least
some statistical parsers that they be robust in the face of noisy input,
certainly during training but perhaps also during parsing, if they are to
avoid being completely thrown off by the occasional typo or unfamiliar
word. So it strikes me as an interesting empirical question whether such
robustness, if indeed the best statistical parsers have it, hinders them from
being able to detect ungrammaticality in general. Of course humans too
can "cope with" ill-formedness of various kinds (as Sproat and Lappin
note), but they mostly know when they are having to do so, i.e., ill-
formedness is still detected.

So, I would like to suggest a revised version of the challenge that
incorporates a second corpus consisting of ungrammatical sentences that
are to be identified as such. (Earlier P&P parsers such as Fong's were
designed to do this, but it's not obvious that this ability will easily scale up
with broader coverage, so I don't think this is a sucker's bet.) Furthermore,
since the computationalists got to choose the corpus of good sentences, it
would seem only fair that the theoreticians get to choose the corpus of bad
sentences :-)

P.S. The statistical parsers will still be getting off easy, in my view, because
the unfamiliar sentences they *are* supposed to parse as well-formed are
drawn from the same sample as the training set. The set of novel sentences
humans [and P&P parsers, we hope] parse as grammatical arguably
includes sentence types that do not occur in the language learner's input.


Carson T. Schutze Department of Linguistics, UCLA
Date Posted: 22-Apr-2005
Linguistic Field(s): Computational Linguistics
Discipline of Linguistics
LL Issue: 16.1288
Posted: 22-Apr-2005

Search Again

Back to Discussions Index