* * * * * * * * * * * * * * * * * * * * * * * *
LINGUIST List logo Eastern Michigan University Wayne State University *
* People & Organizations * Jobs * Calls & Conferences * Publications * Language Resources * Text & Computer Tools * Teaching & Learning * Mailing Lists * Search *
* *
LINGUIST List 17.3077

Thu Oct 19 2006

Calls: Computational Linguistics/Discipline of Linguistics

Editor for this issue: Hannah Morales <hannahlinguistlist.org>

As a matter of policy, LINGUIST discourages the use of abbreviations or acronyms in conference announcements unless they are explained in the text. To post to LINGUIST, use our convenient web form at http://linguistlist.org/LL/posttolinguist.html.
        1.    Patrick Paroubek, Traitement Automatique des Langues

Message 1: Traitement Automatique des Langues
Date: 19-Oct-2006
From: Patrick Paroubek <paplimsi.fr>
Subject: Traitement Automatique des Langues

Full Title: Traitement Automatique des Langues

Linguistic Field(s): Computational Linguistics; General Linguistics

Call Deadline: 20-Nov-2006

Principles of Evaluation in Natural Language Processing.

Special Issue of the Journal 'Traitement Automatique des Langues' (TAL)

Deadline for submission: 20th November 2006

[Version française plus bas]

Redacteurs Invites:

Guest Editors:
Patrick Paroubek (LIMSI-CNRS)
Stéphane Chaudiron (MdRNT/U. Lille 3)
Lynette Hirschman (MITRE)

First Call

For this special issue of TAL, we invite papers about the fundamental
principles that underlie the use of evaluation methods in Natural Language
Processing (NLP). We wish to adopt a higher point of view which goes beyond
the horizon of a single evaluation campaign and have a more global approach
about the problems raised by the deployment of evaluation in NLP. Without any
prejudice, we do not wish to offer with this special issue yet another
tribune to articles relating the participation of a system in a given
evaluation campaign, or articles comparing the pros and cons of two metrics
for assessing performance in a particular task. Our intent is to address
more fundamental issues about the use of evaluation in NLP.


Specific topics include (but are not limited to):

1) Corpora in the evaluation process, their use, the development life
cycle, the synergy around the pair corpus - evaluation campaign.

2) Evaluation as a source of creation of linguistic resources. What help
does it bring in maintaining existing resources?

3) The question of re-using 'found corpora' for evaluation, i.e. corpora
that exist, with some level of annotation that can be adapted to provide
rich 'real world' corpora with somewhat noisy or incomplete annotations.

4) To evaluate implies to have a reference against which to gauge a
performance, but how is defined the reference in NLP? How should we deal
with the problem that often the reference is not unique (e.g. in machine

5) Which formalisms for evaluation in NLP?

6) Which characteristics should evaluation have in NLP? Comparative,
quantitative, qualitative... evaluation?

7) Technology evaluation and user/application oriented evaluation. How are
these two different kinds of evaluation perceived by the NLP community?

8) Evaluation and scientific progress, e.g. large scale evaluation programs
in NLP.

9) Which role does evaluation play in the NLP scientific process?

10) Some domains of NLP are reputed easier for evaluation than others
(parsing, semantics, translation). Myth or reality?

The Journal
(see http://www.atala.org/)

Articles are written in French or in English. Submissions in English are
only accepted for non-native speakers of French.

Important Dates

Submission Deadline: 20/11/2006
Acceptance Notification: 22/01/2007
Revised Final Version: 16/04/2007


Articles (25 pages maximum, PDF format) will be sent to:
Patrick Paroubek limsi.fr>

Style sheets are available on line at: http://tal.e-revues.com/appel.jsp

Special Issue Editorial Board

Mohand Boughanem (IRIT)
Frédéric Béchet (LIA, U. Avignon)
Hervé Blanchon (IMAG)
Jean-Francois Bonastre (LIA, U. Avignon)
John Carroll (U. Sussex)
Gael de Chalendar (CEA, Fontenay aux roses)
Robert Gaizauskas (U. Sheffield)
Guillaume Gravier (IRISA)
Tony Hartley (U. of Leeds)
Lori Lamel (LIMSI-CNRS)
Dominique Laurent (Synapse)
Joseph Mariani (LIMSI-CNRS)
Jean-Luc Minel (Paris X)
Adeline Nazarenko (Paris XIII)
Andrei Popescu-Belis (ISSCO, Genève)
Gerard Sabah (LIMSI-CNRS)
Anne Vilnat (LIMSI-CNRS)
Claire Waast (EDF)
Bony Webber (U. of Edimbhurg)

Respond to list|Read more issues|LINGUIST home page|Top of issue

Please report any bad links or misclassified data

LINGUIST Homepage | Read LINGUIST | Contact us

NSF Logo

While the LINGUIST List makes every effort to ensure the linguistic relevance of sites listed
on its pages, it cannot vouch for their contents.