LINGUIST List 16.1364: Re: A Challenge to the Minimalist Community

Eastern Michigan University

Wayne State University

LINGUIST List 16.1364
Fri Apr 29 2005 Disc: Re: A Challenge to the Minimalist Community Editor for this issue: Michael Appleby <michaellinguistlist.org>
To post to LINGUIST, use our convenient web form at http://linguistlist.org/LL/posttolinguist.html.
Directory 1. Sean Fulop, Re: A Challenge to the Minimalist Community 2. Ash Asudeh, Re: A Challenge to the Minimalist Community 3. Stefan Müller, Re: A Challenge to the Minimalist Community Message 1: Re: A Challenge to the Minimalist Community
Date: 22-Apr-2005 From: Sean Fulop <sfulopcsufresno.edu> Subject: Re: A Challenge to the Minimalist Community I believe the challenge offered by Sproat and Lappin, for someone to build a successful Principles and Parameters parser that learns from treebanks, is mixed up in a number of respects. Let me point out some of the ways the original challenge doesn't take into account the goals of theoretical linguistics, and misses an important distinction between applied natural language processing and theoretical computational linguistics. First, let me say that I am not in favor of the Principles and Parameters framework as a linguistic theory, nor am I in favor of Minimalism (whatever that actually turns out to be). I promote type- logical categorial grammar as a framework for Universal Grammar, but whatever, my goals are broadly the same as those of P&P proponents, and Sproat & Lappin's challenge might equally be applied to type-logical grammar. There have been some spirited defenses of P&P posted already by various proponents, and I apologize if I overlap with what is said there a little bit in the sequel. P&P is an effort to describe and explain what the human language faculty is, and what it does during learning. Personally, I don't think there is much hope for its approach, but that remains debatable, so let's presume it's a good theory, meaning some future tweak of it will turn out to effectively describe, to every linguists' satisfaction, the fundamental grammars of every language and how those could be learned given the specific Universal Grammar that would also be needed. Now, who said anything about computational tractability? This is a complete theory, not an efficiency contest. I think Ed Stabler's work on GB and Minimalism over the years should be convincing enough, that a computational implementation is at least possible, and won't be tractable because it is bereft of performance limitations. But we have to be happy with our theories when they correctly describe the input- output relation of the human language acquisition process, never mind the actual process. It is hopeless to try to conform to the "actual process" used by the mind, we have no way of knowing what that is or whether our particular computers are even up for the job. Some cognitive scientists have even suggested that the brain is not constrained by computability at all, since mathematical computability is defined in reference to computational procedures that we now fathom. Worrying about tractability in P&P is like denigrating relativity theory because it makes it needlessly harder to calculate artillery ballistics. That's of course true, but that's also why no one invokes such a complete theory of relative motion simply to calculate artillery ballistics. Sproat and Lappin note that learning from treebanks is supervised, meaning the parser is trained by a subset of the right answers. They go on to reference work by Klein and Manning which induces grammars in an "unsupervised" fashion from text. Well first of all, it is still debatable whether anything can ever actually do this (see the algorithmic learning theory literature, summarized in Jain et al. 1999), and Sproat and Lappin also note that Klein and Manning's scheme uses part-of-speech tagged text, which is a far cry from text. This is a huge annotation, and could be taken as a component of Universal Grammar. It is a component that is argued for in P&P, as well. I agree with Sproat and Lappin, that P&P would be better off with an attempt to implement its general learning scheme---an analysis of the informational complexity of this problem is provided in Partha Niyogi's book. I disagree that this computational effort should be expected to succeed in practice, since that would require tractability, and that is too much to ask of a generally correct theory. The effort should be mathematically proven to succeed in theory, by defining just exactly what it would learn if you could wait long enough for it, and by proving that it would terminate if you waited long enough for it. This is the sort of learnability result that I've obtained for type-logical grammar, given a certain format for Universal Grammar and certain assumed mathematical properties of all human languages. What I don't require is the parts of speech - we learn those, surely. The learning scheme is outlined in my book and also some current papers (see references). Instead of the POS annotation, it requires annotation by skeletal semantic structures, but there is nothing wrong with requiring annotation. Children certainly receive "annotated" sentences, in that they get other clues to meaning when sentences are spoken. While I support the current efforts in statistical NLP, I don't believe it has any future as a linguistic theory. It is a computational methodology, inspired by the recognition that it would be too hard/slow to invoke the most complete form of linguistic theory to perform basic NLP tasks. As I've often told my comp ling students, "if you've made software that works well, it probably doesn't actually solve a theoretical problem, which is what makes it part of applied NLP; if you've made software that implements a full theoretical solution, it probably won't finish by the end of the day, which is what makes it theoretical computational linguistics." That doesn't mean the twain shall never meet. But not yet. References: Fulop, Sean A. (2005) "Semantic bootstrapping of type-logical grammar," Journal of Logic, Language and Information 14:49-86. Fulop, Sean A. (2004) On the Logic and Learning of Language. Victoria, Canada: Trafford. Fulop, Sean A. (2003) "Discovering a new class of languages," Proceedings of Mathematics of Language 8, available online at MOL website. Niyogi, Partha (1998) The Informational Complexity of Learning. Kluwer. Stabler, Edward (1992)? The Logical Approach to Syntax. Cambridge, MA: MIT Press. Linguistic Field(s): Computational Linguistics Discipline of Linguistics Syntax Message 2: Re: A Challenge to the Minimalist Community
Date: 24-Apr-2005 From: Ash Asudeh <asudehcsli.stanford.edu> Subject: Re: A Challenge to the Minimalist Community Some confusion has arisen in the subsequent discussion of the Sproat-Lappin challenge. Most of the subsequent posts discuss statistical parsing versus P&P parsing. However, the challenge has nothing to do with statistical parsers per se, although it's true that much of the original challenge discussed these parsers. There are, however, non-statistical or hybrid broad-coverage parsers in theories other than P&P, such as the PARC LFG parser mentioned in the original Sproat-Lappin challenge and the HPSG Lingo/LKB parser at Stanford. For theorists who for whatever reason think that statistical parsers are a different sort of thing from the P&P parser that the challenge focuses on (they are not, for the reasons sketched by Sproat and Lappin themselves and also by Ken Shan (LL16.1288)), these can perhaps serve as a reference point instead. There is also, in my opinion, a confusing aspect to the Sproat-Lappin challenge. They are not only asking for a P&P parser, but also asking for a large P&P grammar. Grammars and parsers are different things, at least for non-statistical "deep grammar" approaches (purely statistical methods instead induce the grammar), such as the HPSG and LFG ones noted above and, presumably, like the P&P one in question. For example, the PARC LFG parser has been applied to analyses of a large number of languages, although the English grammar is what Sproat and Lappin seem to have in mind in their message, given the citation of Riezler et al. Another example is the LKB system, which implements an HPSG parser. The English Resource Grammar (ERG) is used with this parser, but they are developed separately (although with strong interactions -- same goes for the PARC XLE parser and the ParGram grammars). Carson Schütze (16.1288) wrote: > In addition to capturing the distinction between learnable and > unlearnable languages, P&P has as an important goal capturing > the distinction between well-formed (grammatical) and ill-formed > (ungrammatical) sentences within a language. As I understand > it, the challenge demands only correct parsing of grammatical > sentences, not correct rejection of ungrammatical ones. This > represents another case where the P&P system, by virtue of > the goals of the theory, is being subjected to greater demands > than the statistical parsers. I don't understand the substance of this objection. All grammars, those used in statistical parsing or otherwise, attempt to reject ungrammatical sentences: Nobody wants their grammar/parser to overgenerate. Even if the claim is true of statistical parsers (I don't think it is), it certainly isn't true of the LFG and HPSG parsers and grammars noted above. Peter Hallman (LL16.1251) and Martha McGinnis (also LL16.1251), although positively inclined to the challenge, also raise objections. The substance of the objections are that P&P is attempting to do much more than just parse sentences (Hallman) and that the goals of P&P are different to those of computational linguistics (McGinnis). I think there is merit to both these statements, but they are ultimately non sequiturs to the challenge. P&P is not just concerned with how the adult grammar results from the inititial state (although this is a principal goal): it is also concerned with the state of the final grammar (Chomsky, 1986: "What constitutes knowledge of language?"). The requirement of capturing the adult grammar also means that it's insubstantial whether the goals of P&P are those of computational linguistics: P&P is still expected to capture adult grammatical competence in the end, even if this isn't a motivation for a lot of its practitioners. Lastly, I agree with Ken Shan (LL16.1288): "All other things being equal, poor (or unknown) parsing performance indicates failure at (resp. disinterest in) answering Q [What is a possible human language?]." It is no surprise that the objections to the challenge so far have sought to argue that all other things are not equal. Like Ken, however, I think that this challenge is a good thing for P&P and I too am optimistic for P&P. In any case, the attempt to meet the challenge will without doubt reap huge rewards, not just for computational linguistics, but especially for P&P theory. Ash Asudeh Linguistic Field(s): Computational Linguistics Discipline of Linguistics Syntax Message 3: Re: A Challenge to the Minimalist Community
Date: 25-Apr-2005 From: Stefan Müller <Stefan.Muellercl.uni-bremen.de> Subject: Re: A Challenge to the Minimalist Community Carson Schutze: > So, I would like to suggest a revised version of the challenge that > incorporates a second corpus consisting of ungrammatical > sentences that are to be identified as such. (Earlier P&P parsers > such as Fong's were designed to do this, but it's not obvious that > this ability will easily scale up with broader coverage, so I don't > think this is a sucker's bet.) Furthermore, since the > computationalists got to choose the corpus of good sentences, > it would seem only fair that the theoreticians get to choose the > corpus of bad sentences :-) This is a very important point and negative data has been collected and is used to evaluate deep linguistic processing. A nice software for evaluating systems and working with test suite databases can be found at: http://www.delph-in.net/itsdb/ Test suites for German, English, French, Spanish, and other languages are also available there. You may find test suites for German at: http://www.cl.uni-bremen.de/Software/TS/ These test suites contain (normalized) examples from the descriptive literature, P&P, HPSG, and other theoretical literature. With the [incr TSDB()] software it is possible to get a selection of sentences that is relevant for a certain phenomenon. Sentences are crossclassified according to the phenomena they are relevant for. The idea is to develop these collections further into a generally accepted benchmark for linguistic theories in general and for deep linguistic processing in particular. Of course the negative sentences can be used to check what statistical parsers have to say about them in comparison to the well-formed examples. So if somebody has a look at the German collection and wants to contribute, please send me the relevant examples and pointers to the publications in which the examples are discussed. Best wishes Stefan Müller Universität Bremen/Fachbereich 10 http://www.cl.uni-bremen.de/~stefan/ http://www.cl.uni-bremen.de/~stefan/Babel/Interaktiv/ Linguistic Field(s): Computational Linguistics Discipline of Linguistics Linguistic Theories Subject Language(s): German, Standard (GER) Respond to list\|Read more issues\|LINGUIST home page\|Top of issue

Print This Page

Please report any bad links or misclassified data

LINGUIST Homepage | Read LINGUIST | Contact us

While the LINGUIST List makes every effort to ensure the linguistic relevance of sites listed
on its pages, it cannot vouch for their contents.