LINGUIST List 19.63
|
Tue Jan 08 2008
Calls: Computational Ling/Morocco; General Ling,Lang Acquisition/Uruguay
Editor for this issue: Ania Kubisz
<ania linguistlist.org>
|
As a matter of policy, LINGUIST discourages the use of abbreviations
or acronyms in conference announcements unless they are explained in
the text. To post to LINGUIST, use our convenient web form at
http://linguistlist.org/LL/posttolinguist.html.
|
Directory
1. Hélène
Mazo,
ELRA Workshop on Evaluation
2. Rocio
Raña Risso,
Proyecto de la Diáspora Latinoamericana
Message 1: ELRA Workshop on Evaluation
|
Date: 08-Jan-2008
From: Hélène Mazo <mazoelda.org>
Subject: ELRA Workshop on Evaluation
 E-mail this message to a friend
Full Title: ELRA Workshop on Evaluation
Date: 27-May-2008 - 27-May-2008
Location: Marrakech, Morocco
Contact Person: Victoria Arranz
Meeting Email: arranzelda.org
Web Site: http://www.lrec-conf.org/lrec2008/Workshops.html
Linguistic Field(s): Computational Linguistics
Call Deadline: 28-Jan-2008
Meeting Description
ELRA Workshop on Evaluation
Looking into the Future of Evaluation: when automatic metrics meet task-based
and performance-based approaches
To be held in conjunction with the 6th International Language Resources and
Evaluation Conference (LREC 2008)
27 May 2008, Palais des Congrès Mansour Eddahbi, Marrakech
Background
Automatic methods to evaluate system performance play an important role in the
development of a language technology system. They speed up research and
development by allowing fast feedback, and the idea is also to make results
comparable while aiming to match human evaluation in terms of output evaluation.
However, after several years of study and exploitation of such metrics we still
face problems like the following ones:
- they only evaluate part of what should be evaluated
- they produce measurements that are hard to understand/explain, and/or hard to
relate to the concept of quality
- they fail to match human evaluation
- they require resources that are expensive to create
etc.
Therefore, an effort to integrate knowledge from a multitude of evaluation
activities and methodologies should help us solve some of these immediate
problems and avoid creating new metrics that reproduce such problems.
Looking at MT as a sample case, problems to be immediately pointed out are
twofold: reference translations and distance measurement. The former are
difficult and expensive to produce, they do not cover the usually wide spectrum
of translation possibilities and what is even more discouraging, worse results
are obtained when reference translations are of higher quality (more spontaneous
and natural, and thus, sometimes more lexically and syntactically distant from
the source text). Regarding the latter, the measurement of the distance between
the source text and the output text is carried out by means of automatic metrics
that do not match human intuition as well as claimed. Furthermore, different
metrics perform differently, which has already led researchers to study
metric/approach combinations which integrate automatic methods into a deeper
linguistically oriented evaluation. Hopefully, this should help soften the
unfair treatment received by some rule-based systems, clearly punished by
certain system-approach sensitive metrics.
On the other hand, there is the key issue of ''what needs to be measured'', so
as to draw the conclusion that ''something is of good quality'', or probably
rather ''something is useful for a particular purpose''. In this regard, works
like those done within the FEMTI framework have shown that aspects such as
usability, reliability, efficiency, portability, etc. should also be considered.
However, the measuring of such quality characteristics cannot always be
automated, and there may be many other aspects that could be usefully measured.
This workshop follows the evolution of a series of workshops where
methodological problems, not only for MT but for evaluation in general, have
been approached. Along the lines of these discussions and aiming to go one step
further, the current workshop, while taking into account the advantages of
automatic methods and the shortcomings of current methods, should focus on
task-based and performance-based approaches for evaluation of natural language
applications, with key questions such as:
- How can it be determined how useful a given system is for a given task?
- How can focusing on such issues and combining these approaches with our
already acquired experience on automatic evaluation help us develop new metrics
and methodologies which do not feature the shortcomings of current automatic
metrics?
- Should we work on hybrid methodologies of automatic and human evaluation for
certain technologies and not for others?
- Can we already envisage the integration of these approaches?
- Can we already plan for some immediate collaborations/experiments?
- What would it mean for the FEMTI framework to be extended to other HLT
applications, such as summarization, IE, or QA? Which new aspects would it need
to cover?
We solicit papers that address these questions and other related issues relevant
to the workshop.
Workshop Programme and Audience Addressed
This full-day workshop is intended for researchers and developers on different
evaluation technologies, with experience on the various issues concerned in the
call, and interested in defining a methodology to move forward.
The workshop feature invited talks, submitted papers, and will conclude with a
discussion on future developments and collaboration.
Workshop Chairing Team
Gregor Thurmair (Linguatec Sprachtechnologien GmbH, Germany) - chair
Khalid Choukri (ELDA - Evaluations and Language resources Distribution Agency,
France) - co-chair
Bente Maegaard (CST, University of Copenhagen, Denmark) - co-chair
Organising Committee
Victoria Arranz (ELDA - Evaluations and Language resources Distribution Agency,
France)
Khalid Choukri (ELDA - Evaluations and Language resources Distribution Agency,
France)
Christopher Cieri (LDC - Linguistic Data Consortium, USA)
Eduard Hovy (Information Sciences Institute of the University of Southern
California, USA)
Bente Maegaard (CST, University of Copenhagen, Denmark)
Keith J. Miller (The MITRE Corporation, USA)
Satoshi Nakamura (National Institute of Information and Communications
Technology, Japan)
Andrei Popescu-Belis (IDIAP Research Institute, Switzerland)
Gregor Thurmair (Linguatec Sprachtechnologien GmbH, Germany)
Important Dates
Deadline for abstracts: Monday 28 January 2008
Notification to Authors: Monday 3 March 2008
Submission of Final Version: Tuesday 25 March 2008
Workshop: Tuesday 27 May 2008
Submission Format
Abstracts should be no longer than 1500 words and should be submitted in PDF
format to Gregor Thurmair at g.thurmairlinguatec.de.
Message 2: Proyecto de la Diáspora Latinoamericana
|
Date: 08-Jan-2008
From: Rocio Raña Risso <diasporalatinanyc.rr.com>
Subject: Proyecto de la Diáspora Latinoamericana
E-mail this message to a friend
Full Title: Proyecto de la Diáspora Latinoamericana
Short Title: ALFAL
Date: 18-Aug-2008 - 21-Aug-2008
Location: Montevideo, Montevideo, Uruguay
Contact Person: Rocio Raña Risso
Meeting Email: diasporalatinanyc.rr.com
Web Site: http://www.mundoalfal.org
Linguistic Field(s): General Linguistics; Language Acquisition;
Psycholinguistics; Sociolinguistics
Subject Language(s): Portuguese (por)
Spanish (spa)
Call Deadline: 31-Jan-2008
Meeting Description
The Research Group on Linguistic Issues Involving the Latin American Dispora is
a 'Proyecto de Investigación' meeting with the XV International Conference of
the Latin American Association of Linguistics and Philology (ALFAL) in
Montevideo, Uruguay from August 18 to 21.
Second Call for Papers
Abstracts of between 200 and 250 words are invited for 20 minute presentations
on linguistic research in sociolinguistics, psycholinguistics, or applied
linguistics (or combination) involving the Latin American Diaspora, defined as
peoples of Latin American heritage living outside their country of origin (e.g.,
in North America, Europe, or other Latin American countries). Such communities
are of interest because of the situations of dialect and language contact they
entail and the social, educational, and linguistic issues involved in their
adaptation to new matrix societies.
The abstract must be written in Spanish or Portuguese, the official languages of
the conference. Presentations at the conference must also be delivered in
Spanish or Portuguese. There is also the possibility of publishing a paper in
either of these languages in the conference proceedings.
To submit your abstract: send an e-mail to diasporalatinanyc.rr.com with your
name, affiliation, and the abstract in the body of the e-mail and as an attachment.
Please see the conference website for more information: http://www.mundoalfal.org
New Extended Deadline: 31-Jan-2008
Notice of acceptance: 15/03/08
Coordinators
Michael Newman
Associate Professor of Linguistics
michael.newmanqc.cuny.edu, 718-997-2871
Department of Linguistics and Communication Disorders
Queens College, CUNY; Flushing, NY 11367
Eva M. Fernández
Assistant Professor of Linguistics
eva.fernandezqc.cuny.edu, 718-997-2867
Department of Linguistics and Communication Disorders
Queens College, CUNY; Flushing, NY 11367
Fax: 718-997-2873
Rocío Raña Risso
RRana_Rissogc.cuny.edu
Department of Linguistics, The Graduate Center, CUNY
Read more issues|LINGUIST home page|Top of issue
|
|
Please report any bad links or misclassified data
LINGUIST Homepage | Read
LINGUIST | Contact us
While the LINGUIST List makes every effort to ensure the linguistic relevance of sites listed
on its pages, it cannot vouch for their contents.
|
|
