The purpose of this e-mail is to encourage participation in the task 'Parser Evaluation using Textual Entailments' in the 5th International Workshop on Semantic Evaluations, SemEval-2010 (http://semeval2.fbk.eu/semeval2.php) collocated with ACL-2010, July 15-16, Uppsala.
This shared task should be of interest to researchers working on * parsing * semantic relation extraction * recognizing textual entailments
Description:
Parser Evaluation using Textual Entailments is a shared task in the SemEval-2010 Evaluation Exercises on Semantic Evaluation. The task involves recognizing textual entailments (RTE) based solely on syntactic information:
* The man with the hat was tired. * The man was tired. (yes) * The hat was tired. (no)
Our goals in introducing this task are:
* To focus parser evaluation on semantically relevant phenomena. * To introduce a parser evaluation scheme that is formalism independent. * To introduce a targeted textual entailment task focused on a single linguistic competence. * To be able to collect high quality evaluation data from untrained annotators.
The following criteria were used when constructing the entailments:
* They should be decidable using only syntactic inference. * They should be easy to decide by untrained annotators. * They should be challenging for state of the art parsers.
You can find more details about the entailment generation process at the task website (http://pete.yuret.com). The trial dataset can be downloaded from the task website or the SemEval website (http://semeval2.fbk.eu). There will be no training data. The evaluation will be similar to the past RTE tasks. There is a Google group (http://groups.google.com/group/semeval-pete) for task related messages.
* February 19 - the development data available. * March 26 - the test data available. * April 2 - end of submission period for the task. * April 17 - Submission of description papers. * May 6 - Notification of acceptance. * July 15-16 - Workshop at ACL 2010, Uppsala.
linguistlist.org>
E-mail this message to a friend
