LINGUIST List 9.64

Thu Jan 15 1998

Calls: Parsing Systems,Language Resources & Evaluation

Editor for this issue: Anita Huang <anitalinguistlist.org>

Directory

1. John Carroll, THE EVALUATION OF PARSING SYSTEMS - Workshop Call for Papers
2. Simone Saint Laurent, Language Resources and Evaluation

 THE EVALUATION OF PARSING SYSTEMS

 a workshop jointly organised by the CEC Language
 Engineering 1 projects SPARKLE and ECRAN

 to be held at the

 FIRST INTERNATIONAL CONFERENCE 
 ON LANGUAGE RESOURCES AND EVALUATION

 GRANADA, SPAIN, 26 MAY 1998

This workshop will provide a forum for researchers interested in the
development and evaluation of natural language grammars and parsing
systems, and in the creation of syntactically annotated reference
corpora.

Organisers: John Carroll, Roberto Basili, Nicoletta Calzolari,
 Robert Gaizauskas, Gregory Grefenstette


WORKSHOP SCOPE AND AIMS

The aim of this workshop is to provide a forum for discussion of
evaluation methods for parsing systems, and proposals for the
development of syntactically annotated language resources.

With increased attention to evaluation of component technology in
language engineering, evaluation of parsing systems is rapidly becoming
a key issue. Numerous methods have been proposed and while one, the
Parseval/Penn Treebank scheme, has gained wide usage, this has to some
extent been due to the absence of workable alternatives rather than to
whole-hearted support. Parseval/PTB evaluation has several limitations
and drawbacks, including a commitment to a particular style of
grammatical analysis, and oversensitivity to certain innocuous types of
misanalysis while failing to penalise other common types of more serious
mistake. Also, the original published description of the scheme -- and
the evaluation software widely distributed as a follow-up to it -- is
specific to the English language. It may be that there are currently no
alternative more workable schemes or proposals, but this needs to be
more fully discussed: this workshop will provide an opportunity for such
a debate.

This workshop is particularly timely given the large number of CEC
Language Engineering projects that involve parsing in one form or
another and which need to evaluate and share the results of their
efforts. Parsing is an essential part of many larger applications, such
as Information Extraction, which have gained in importance over the last
few years. Often in such systems, the strength of the parser and
grammar has a direct effect on the desired results, and thus achieving
good results rests on being able to determine and improve weaknesses in
the parser/grammar. Without a reliable parser evaluation method this
cannot be done effectively.

A parsing evaluation workshop is also appropriate at this time given the
imminent creation of large-scale syntactically annotated resources for
European languages. Contributions from those involved in such activities
are welcomed, so as to improve communication between the resource
construction and the resource utilisation communities. This should
ensure that the resources constructed are maximally useful to the
general language engineering community.

The organisation of this workshop brings together two European language
engineering projects which are closely related and whose partners share
similar research interests: SPARKLE and ECRAN.

The organisers solicit contributions from the general community on the
following topics:

- descriptions of generic syntactic annotation schemes
- methodologies and metrics for parsing system evaluation
- reports and analyses of the results of utilising particular parser
 evaluation schemes
- description/analysis/experience of language-dependent (especially
 for languages other than English) and task-dependent syntactic
 annotation schemes


PROGRAMME COMMITTEE

Roberto Basili Gregory Grefenstette
Ted Briscoe Mark Hepple
Nicoletta Calzolari Tony McEnery
John Carroll Maria Teresa Pazienza
Roberta Catizone Paola Velardi
Robert Gaizauskas Yorick Wilks


PAPER SUBMISSION

Papers should not exceed 4000 words or 10 pages. Submission may be in
either hard copy or electronic form. The submission deadline is February
15th, 1998.

Hard Copy Submission:

Three copies of the paper should be sent to:

Dr John Carroll
Cognitive and Computing Sciences
University of Sussex
Brighton BN1 9QH
UK

Electronic Submission:

Electronic submission may be in either self-contained Latex, Postscript,
or RTF formats, to john.carrollcogs.susx.ac.uk. For each submission --
whether hard copy or electronic -- a separate plain ascii text email
message should be sent to John Carroll, containing the following
information:

# NAME : Name of first author
# TITLE: Title of the paper
# PAGES: Number of pages
# NOTE : Any relevant instructions
# KEYS : Keywords
# EMAIL: Email of the first author
# ABSTR: Abstract of the paper
 . . . . . .


IMPORTANT DATES

Paper submission deadline (hard copy/electronic) February 15th
Notification of acceptance March 10th
Camera-ready papers due April 10th
Workshop May 26th


CONFERENCE INFORMATION

General information about the conference is at:
http://www.icp.inpg.fr/ELRA/conflre.html

Specific queries about the conference should be directed to:

LREC Secretariat
Facultad de Traduccion e Interpretacion
Dpto. de Traduccion e Interpretacion
C/ Puentezuelas, 55
18002 Granada, SPAIN
Tel: +34 58 24 41 00 - Fax: +34 58 24 41 04
reli98goliat.ugr.es

============================================================================
CALL FOR PAPER

Adapting Lexical and Corpus Resources to Sublanguages and Applications
Granada May 26, 1998

This workshop will be held in conjunction with the First International
Conference on Language Resources and Evaluation (LREC), to be held
in Granada, Spain on May 28 - 30, 1998.
The workshop will provide a forum for those researchers involved in
the development of methods to integrate corpora and MRDs, with the aim of
adding adaptive capabilities to existing linguistic resources.


Workshop Scope and Aims

Lexicons, i.e., those components of a NLP system that contain "computable"
information about words, cannot be considered as static objects. Words may
behave very differently in different domains, and there are language
phenomena that do not generalize across sublanguages.
Lexicons are a snapshot of a given stage of development of a language,
normally providedwithout support for adaptation changes, whether caused
by language creativity and development or the shift to such
a previously unencountered domain.

The divergence of corpus usage's from lexical norms has been studied
computationally at least since the late Sixties, but only recently
has the availability of large on-line corpora made it possible to establish
methods to cope systematically with this problem.
An emerging branch of research is now involved in studies and experiments
on corpus-driven linguistics, with the aim of complementing and
extending earlier work on lexicon acquisition based on Machine Readable
Dictionaries (MRD): data are extracted from texts, as embodiments of
language in
use, so as to capture lexical regularities and to code them into operational
forms. The purpose of this workshop will be to provide an updated snapshot
of current work in the area, and promote discussion of how to make progress.

Central topics will be (though this list is in no way exclusive):

* corpus-driven tuning of MRDs to optimize domain-specific inferences,
* terminology and jargon acquisition,
* sense extensions,
* acquisition of preference or subcategorization information from corpora
* taxonomy adaptation,
* staistical weighting of senses etc. to domains
* use of MRDs to provide explanations of linguistic phenomena in corpora
* what is the scope of "lexical tuning"
* the evaluation of lexical tuning as a separate task, or as part
 of a more generic task

Organizers: Roberto Basili (University of Roma "Tor Vergata"), Roberta
Catizone (University of Sheffield), Maria Teresa Pazienza (University of
Roma "Tor Vergata"), Paola Velardi (University of Roma "La Sapienza),
Yorick Wilks (University of Sheffield)

Preliminary Program Committee

Yorick Wilks University of Sheffield
Roberta Catizone University of Sheffield
Paola Velardi University of Roma "La Sapienza"
Maria Teresa Pazienza University of Roma "Tor Vergata"
Roberto Basili University of Roma "Tor Vergata"
Bran Boguraev Brandeis University
Sergei Nirenburg New Mexico State University
James Pustejowsky Brandeis University
Ralph Grishman New York University
Christiane Fellbaum Princeton University

Paper Submission

FORMATTING GUIDELINES:
Papers should not exceed 4000 words or 10 pages.

HARD COPIES:

Three hard copies should be sent to:

 Paola Velardi
 Dipartimento di Scienza dell'Informazione
 via Salaria 113
 00198 Roma
 Italy


ELECTRONIC SUBMISSION:
Electronic submission will be allowed in Poscript or Word per Mac or RTF.
An ftp site will be available on demand.
Authors should send an info email to Paola Velardi
(velardidsi.uniroma1.it) even
if they submit in paper form. An electronic submission should be
accompanied by a plain ascii text.

# NAME : Name of first author
# TITLE: Title of the paper
# PAGES: Number of pages
# FILES: Name of file (if also submitted electronically)
# NOTE : Anything you'd like to add
# KEYS : Keywords
# EMAIL: Email of the first author
# ABSTR: Abstract of the paper



IMPORTANT DATES

 Paper Submission Deadline (Hard Copy/Electronic) February 20
 Paper Notification March 20
 Camera-Ready Papers Due April 15
 L&CT workshop May 26

===========================================================================
Prof. Paola Velardi
Dipartimento di Scienza dell'Informazione
via Salaria 113
Universita' "La Sapienza"
00198 Roma
ph. +39-(0)6-49918356
fax +39-(0)6-8541842 8841964