LINGUIST List 17.3436|
Wed Nov 22 2006
Calls: Computational Ling/Traitement Automatique des Langues (Jrnl)
Editor for this issue: Hannah Morales
As a matter of policy, LINGUIST discourages the use of abbreviations
or acronyms in conference announcements unless they are explained in
the text. To post to LINGUIST, use our convenient web form at
Traitement Automatique des Langues
Message 1: Traitement Automatique des Langues
From: Patrick Paroubek <paplimsi.fr>
Subject: Traitement Automatique des Langues
Full Title: Traitement Automatique des Langues
Linguistic Field(s): Computational Linguistics; General Linguistics
Call Deadline: 15-Dec-2006
Principles of Evaluation in Natural Language Processing.
Special Issue of the Journal 'Traitement Automatique des Langues' (TAL)
Deadline for submission: 15th December 2006
Patrick Paroubek (LIMSI-CNRS)
Stéphane Chaudiron (MdRNT /U. Lille 3)
Lynette Hirschman (MITRE)
For this special issue of TAL, we invite papers about the fundamental
principles that underlie the use of evaluation methods in NLP. We wish
to adopt a higher point of view which goes beyond the horizon of a
single evaluation campaign and have a more global approach about the
problems raised by the deployment of evaluation in NLP. Without any
prejudice, we do not wish to offer with this special issue yet another
tribune to articles relating the participation of a system in a given
evaluation campaign, or articles comparing the pros and cons of two
metrics for assessing performance in a particular task. Our intent is
to address more fundamental issues about the use of evaluation in NLP.
Specific topics include (but are not limited to):
1) Corpora in the evaluation process, their use, the development life
cycle, the synergy around the pair corpus - evaluation campaign.
2) Evaluation as a source of creation of linguistic resources. What help
does it bring in maintaining existing resources?
3) The question of re-using 'found corpora' for evaluation, i.e. corpora
that exist, with some level of annotation that can be adapted to provide
rich 'real world' corpora with somewhat noisy or incomplete annotations.
4) To evaluate implies to have a reference against which to gauge a
performance, but how is defined the reference in NLP? How should we deal
with the problem that often the reference is not unique (e.g. in machine
5) Which formalisms for evaluation in NLP?
6) Which characteristics should evaluation have in NLP? Comparative,
quantitative, qualitative... evaluation?
7) Technology Evaluation and User/Application Oriented Evaluation, how are
these two different kinds of evaluation perceived by the NLP community?
8) Evaluation and scientific progress, e.g. large scale evaluation programs
9) Which role does evaluation play in the NLP scientific process?
10) Some domains of NLP are reputed easier for evaluation that others
(parsing, semantics, translation)--myth or reality?
The Journal (see http://www.atala.org/)
Articles are written in French or in English. Submissions in English are
only accepted for non-native speakers of French.
Extended Submission Deadline: 15/12/2006
Acceptance Notification: 22/01/2007
Revised Final Version: 16/04/2007
Articles (25 pages maximum, PDF format) will be sent to:
Patrick Paroubek limsi.fr>
Style sheets are available on line at: http://tal.e-revues.com/appel.jsp
Special Issue Editorial Board
Mohand Boughanem (IRIT)
Frédéric Béchet (LIA, U. Avignon)
Hervé Blanchon (IMAG)
Jean-Francois Bonastre (LIA, U. Avignon)
John Carroll (U. Sussex)
Gael de Chalendar (CEA, Fontenay aux roses)
Robert Gaizauskas (U. Sheffield)
Guillaume Gravier (IRISA)
Tony Hartley (U. of Leeds)
Lori Lamel (LIMSI-CNRS)
Dominique Laurent (Synapse)
Joseph Mariani (LIMSI-CNRS)
Jean-Luc Minel (Paris X)
Adeline Nazarenko (Paris XIII)
Andrei Popescu-Belis (ISSCO, Genève)
Gerard Sabah (LIMSI-CNRS)
Anne Vilnat (LIMSI-CNRS)
Claire Waast (EDF)
Bonnie Webber (U. of Edinburgh)
Respond to list|Read more issues|LINGUIST home page|Top of issue
Please report any bad links or misclassified data
LINGUIST Homepage | Read
LINGUIST | Contact us
While the LINGUIST List makes every effort to ensure the linguistic relevance of sites listed
on its pages, it cannot vouch for their contents.