LINGUIST List 14.1340

Fri May 9 2003

Calls: Machine Translation; Germanic Syntax/UK

Editor for this issue: Marie Klopfenstein <marielinguistlist.org>

Directory

1. Leslie Barrett, MT Summit IX : MT Eval Worshop submission extension
2. s.j.hannahs, 18th Comparative Germanic Syntax Workshop

Workshop on MT Evaluation

Towards Systematizing MT Evaluation
(deadline extended!)

Theme:

Estimating the quality of any machine-translation system accurately is
only possible if the evaluation methodology is robust and systematic.
The NSF and EU-funded ISLE project has created a taxonomy that relates
situations and measures for a variety of MT applications. The
"Framework for MT Evaluation in ISLE" (FEMTI) is now available online
at http://www.issco.unige.ch/projects/isle/taxonomy3/. The effort of
matching these measures correctly with their appropriate evaluation
tasks, however, is an area that needs further attention. For example,
what effect do "user needs" have on the "functionality
characteristics" specified in the FEMTI guidelines? To what extent are
there unseen relationships in the branches of the taxonomy? How can we
judge when a given evaluation measure is appropriate? Issues that come
to bear on this question are the automation of MT evaluation, the
extension to MT applications such as automated speech-translation, and
the evaluation of the very training corpora that an MT system relies
on to improve output quality.

This workshop welcomes papers for 30-minute presentations on the
comparison between MT evaluation measures, studies of the behavior of
individual measures (i.e., meta-evaluation), new uses for measures,
analysis of MT evaluation tasks with respect to measures, and related
topics on this theme. We solicit submissions to the workshop that
address some of the following issues, however any other topic related
to MT Testing and Evaluation is also acceptable:

Machine Translation Evaluation Measures:

 a.. Use of existing measures in the ISLE hierarchy (FEMTI
guidelines)
 b.. New measures and their uses
 c.. Matching evaluation requirements (e.g., translation tasks, user
profiles) with measures
 d.. Effects of combining measures 

Evaluation Measures and Languages:

 a.. Is a metric's effectiveness language independent?
 b.. Counting grammatical features for evaluation

Evaluation and Domains

 a.. Measures for spoken Language translation
 b.. Domain-specific evaluation techniques
 c.. Using measures to evaluate the quality of a training corpus for
a given task

Automation vs. Human Testing

 a.. Which measures are suitable for automation?
 b.. Human/machine scoring comparisons
 c.. Human tester agreement: which measures fare best?

Submission Format

Papers (full papers up to 8 pages in length) must be submitted
electronically to: barrettsemanticdatasystems.com, or
andrei.popescu-belisissco.unige.ch

Papers are preferred in .pdf, .ps, .rtf, or .txt, format

Important Dates: NEW

Paper submission deadline: May 26

Notification: June 30

Camera-ready due: July 31

Note: the workshop will be held on September 23, 2003

Also see workshop website for updates:

http://www.issco.unige.ch/projects/isle/MTE-at-MTS9.html

Contacts

Leslie Barrett (Transclick, Inc., New York, NY)

Andrei Popescu-Belis (ISSCO/TIM/ETI, University of Geneva)

Organizers

Leslie Barrett (Transclick, Inc., New York, NY)

Maghi King (ISSCO/TIM/ETI, University of Geneva)

Keith Miller (MITRE Corp)

Andrei Popescu-Belis (ISSCO/TIM/ETI, University of Geneva)

Program Committee:

Bonnie Dorr (University of Maryland)

Eduard Hovy (Information Sciences Institute, University of Southern
California)

Maghi King (ISSCO/TIM/ETI, University of Geneva)

Bente Maegaard (Center for Sprogteknologi, Copenhagen, Denmark)

Keith Miller (MITRE Corp.)

Martha Palmer (University of Pennsylvania)

Ted Petersen (Univesity of Minnesota)

Andrei Popescu-Belis (ISSCO/TIM/ETI, University of Geneva)

Florence Reeder (MITRE Corp)

Nancy Underwood (ISSCO/TIM/ETI, University of Geneva)

Michelle Vanni (National Computer Security Center)

18th Comparative Germanic Syntax Workshop 
Short Title: CGSW 18

Date: 18-SEP-03 - 20-SEP-03
Location: Durham, United Kingdom
Contact: Anders Holmberg
Contact Email: anders.holmbergdurham.ac.uk 

Linguistic Sub-field: Syntax 
Call Deadline: 31-May-2003

Meeting Description:

Workshop on comparative Germanic syntax, including diachronic syntax
and language acquisition. 

		**Second Call for Papers**

 Comparative Germanic Syntax Workshop 18
 University of Durham, 18-20 September 2003

Papers are invited for the CGSW 18, to be held in Durham, England,
18-20 September 2003. Invited speakers:

Ian Roberts (Cambridge)
Manuela Sch�nenberger (Konstanz)
Halldor A. Sigurdsson (Lund)
Peter Svenonius (Tromsoe)
Susan Wurmbrand (McGill)

The workshop will take place in St. John's College, University of
Durham. There is no conference fee, but participants pay their own
costs. Accommodation at a reasonable rate is provided at the college:
40 GBP/night for full board. The workshop will start in the evening of
the 18th and end at 4pm the 20 September.

Papers (45 minutes inclusive) may concern any aspect of comparative
Germanic syntax, including diachronic syntax and language
acquisition. Abstracts (maximum 2 pages single spaced, 12 point, 2.5
cm. margins) to be sent to

Anders Holmberg
School of Linguistics and Language
University of Durham
Elvet Riverside 
Durham DH1 3JT
UK

Abstracts may be sent electronically (Word or pdf) to
anders.holmbergdurham.ac.uk

Deadline for abstracts: 31 May. 

Results of the selection of papers will be announced three weeks
later.