LINGUIST List 27.918

Mon Feb 22 2016

FYI: New NIST Evaluation: Low Resource HLT Evaluation

Editor for this issue: Ashley Parker <ashleylinguistlist.org>


Date: 19-Feb-2016
From: Shudong Huang <shudonghgmail.com>
Subject: New NIST Evaluation: Low Resource HLT Evaluation
E-mail this message to a friend

NIST is pleased to announce a new evaluation series aimed to advance human language technologies that provide rapid and effective response to emerging incidents where the language resources are very limited. This evaluation series is called Low Resource HLT Evaluation (LoReHLT). LoReHLT16 plans to offer three evaluation tasks: machine translation (MT), named entity recognition (NER), and topic labeling (TL).

Highlights:

- Two confirmed evaluation tasks: MT and NER (Topic Labeling is under development)
- Two training conditions: constrained (required) and unconstrained
- Surprise language evaluation
- Three evaluation checkpoints to gauge performance based on training resources given

2016 Schedule:

Feb 19 - May 2: Registration period
Jun 2 - 8: Dry run period
Jun 29: Encrypted evaluation data released
Jul 6: Surprise language announced and evaluation begins
Jul 13: Checkpoint 1 due
Jul 20: Checkpoint 2 due
Aug 3: Checkpoint 3 due
Aug 28 – 29: Post evaluation workshop in Washington state

For more information about the LoReHLT16 evaluation, please see the evaluation plan at http://www.nist.gov/itl/iad/mig/lorehlt16.cfm. If you have any questions, contact us at lorehlt_pocnist.gov.

Feel free to forward this message to your colleagues who may find the evaluation of interest.

Best Regards,
NIST LoReHLT team


Linguistic Field(s): Computational Linguistics

Page Updated: 22-Feb-2016