LINGUIST List 17.2289

Thu Aug 10 2006

Calls: Computational Linguistics

Editor for this issue: Hannah Morales <hannahlinguistlist.org>

Directory         1.    Patrick Paroubek, Traitement Automatique des Langues
Message 1: Traitement Automatique des Langues
Date: 09-Aug-2006
From: Patrick Paroubek <paplimsi.fr>
Subject: Traitement Automatique des Langues



Full Title: Traitement Automatique des Langues

Linguistic Field(s): Computational Linguistics; General Linguistics

Call Deadline: 20-Nov-2006

Principles of Evaluation in Natural Language Processing.

Special Issue of the Journal 'Traitement Automatique des Langues' (TAL)

Deadline for submission: 20th November 2006

Redacteurs Invites:

Guest Editors: Patrick Paroubek (LIMSI-CNRS)(to be completed)

Preliminary Announcement

Just by looking at the number of publications about evaluation, it is clearthat the usage of evaluation methods has become from being only acontroversial topic to an undeniable fact. In Natural Language Processing(NLP), such methods are used now throughout System Development Life Cycle,for comparing different approaches about a given problem or even duringcorpus development and maintenance. The use of evaluation methods is notanymore restricted to a small community as it was for some time. Suchpractices are now encountered widely in processing of text, speech and evenmultimedia data. In addition to a special international conference aboutevaluation in NLP: LREC, there exists in France a national program'Technolangue' which transcends scientific barriers, since it hasprolongations in artificial vision with the 'Technovision' program, wherewe find evaluation activities about handwriting recognition and ancientdocument processing.

For this special issue of TAL, we invite papers about the fundamentalprinciples that underlie the use of evaluation methods in NLP. We wish toadopt a higher point of view which goes beyond the horizon of a singleevaluation campaign and have a more global approach about the problemsraised by the deployment of evaluation in NLP. Without any prejudice, we donot wish to offer with this special issue yet another tribune to articlesrelating the participation of a system in a given evaluation campaign, orarticles comparing the pros and cons of two metrics for assessingperformance in a particular task. Our intent is to address more fundamentalissues about the use of evaluation in NLP.

Topics

Specific topics include (but are not limited to):

1) Corpora in the evaluation process, their use, the development lifecycle, the synergy around the pair corpus - evaluation campaign.

2) Which formalisms for evaluation in NLP?

3) Comparative or Quantitative evaluation for NLP?

4) Technology Evaluation versus User/Application Oriented Evaluation--which is more appropriate for NLP?

5) To evaluate implies to have a reference against which to gauge aperformance, but how is defined the reference in NLP? How should we dealwith the problem that often it is not unique (e.g. in machine translation)?

6) Evaluation as a source of creation of linguistic resources. What helpdoes it bring in maintaining existing resources?

7) Evaluation and scientific progress, e.g. large scale evaluation programsin NLP.

8) Which role does evaluation play in the NLP scientific process.

9) Some domains of NLP are reputed easier for evaluation than others(parsing, semantics, translation) myth or reality?

The Journal

(see http://www.atala.org/)The journal TAL (Traitement Automatique des Langues) is an internationaljournal published since 1960 by ATALA (Association pour le TraitementAutomatique des Langues) with the help of CNRS. It is now becomingavailable in electronic form, with print on demand. The reading andselection process remain unchanged.

Language

Articles are written in French or in English. Submissions in English areonly accepted for non-native speakers of French.

Important Dates

Submission Deadline: 20/11/2006Acceptance Notification: 22/01/2007Revised Final Version: 16/04/2007

Format

Articles (25 pages maximum, PDF format) will be sent to: Patrick Paroubekpaplimsi.frStyle sheets are available on line at: http://tal.e-revues.com/appel.jsp