LINGUIST List 21.525|
Tue Feb 02 2010
Qs: Call for Participants: SemEval Shared Task 10
Editor for this issue: Danielle St. Jean
We'd like to remind readers that the responses to queries are usually best posted to the individual asking the question. That individual is then strongly encouraged to post a summary to the list. This policy was
instituted to help control the huge volume of mail on LINGUIST; so we would appreciate your cooperating with it whenever it seems appropriate.
In addition to posting a summary, we'd like to remind people that it is usually a good idea to personally thank those individuals who have taken the trouble to respond to the query.
To post to LINGUIST, use our convenient web form at http://linguistlist.org/LL/posttolinguist.html.
Call for Participants: SemEval Shared Task 10
Message 1: Call for Participants: SemEval Shared Task 10
From: Josef Ruppenhofer <josefrcoli.uni-sb.de>
Subject: Call for Participants: SemEval Shared Task 10
E-mail this message to a friend
Call for Participation
SemEval-2010 Shared Task #10:
Linking Events and Their Participants in Discourse
Training Data now available
This shared task should appeal to researchers working on:
- Semantic role labelling (FrameNet or PropBank)
- Co-reference resolution, or
- Information extraction.
Semantic role labelling (SRL) has traditionally been viewed as
sentence-internal problem. However, it is clear that there is an interplay
between local semantic argument structure and the surrounding
discourse. In this shared task, we would like to take SRL beyond the
domain of isolated sentences by linking local semantic argument
structures to the wider discourse context. In particular, we aim to find
fillers for roles which are left unfilled in the local context (null
instantiations, NIs). An example is given below, where the 'charges'
role ('arg2' in PropBank) of "cleared" is left empty but can be linked to
"murder" in the previous sentence:
"In a lengthy court case the defendant was tried for murder."
"In the end, he was cleared."
Short Task Description:
Participants can tackle either (or both) of the following two tasks, which
will be evaluated independently:
The test set will be annotated with GOLD Standard frames. For each
lexical unit annotated with a frame, participants have to:
- find the semantic arguments (role recognition)
- label them with the correct role (role labelling)
- find links between null instantiated roles and their fillers in the
wider discourse context if such links exist (NI linking)
The test set will be annotated with GOLD Standard local semantic
argument structure; only the fillers of null instantiations have to be
Note: We provide the annotated data in both FrameNet and PropBank
style. Participants can choose with which framework they want to work.
The two frameworks are evaluated separately.
For more details see the task web site
(http://www.coli.uni-saarland.de/projects/semeval2010_FG/) and the
Josef Ruppenhofer, Caroline Sporleder, Roser Morante, Collin Baker
and Martha Palmer. 'SemEval-2010 Task 10: Linking Events and Their
Participants in Discourse'. The NAACL-HLT 2009 Workshop on
Semantic Evaluations: Recent Achievements and Future Directions
(SEW-09), Boulder, Colorado, USA, June 4, 2009.
We provide training and (later) test data of running text from the fiction
domain. The data sets are freely available and downloadable from the
SemEval website (http://semeval2.fbk.eu/semeval2.php). The training
set for both tasks is annotated with:
- GOLD Standard semantic argument structure (FrameNet and
- linking information for null instantiations
- co-reference chains (for evaluation purposes)
Participants are also free to make use of external resources, such as
the FrameNet (FN) or PropBank/NomBank releases. (We distribute the
FN-style training set with an intermediate FrameNet release (1.4
- Josef Ruppenhofer, Saarland University, Germany
- Caroline Sporleder, Saarland University, Germany
- Roser Morante, University of Antwerp, Belgium
- Collin Baker, ICSI, Berkeley, USA
- Martha Palmer, University of Colorado, Boulder, USA
If you're interested in participating in the shared task, please email
Josef Ruppenhofer to inquire about joining our Google group:
Task #10 website:
SemEval 2010 website:
Read more issues|LINGUIST home page|Top of issue
Please report any bad links or misclassified data
LINGUIST Homepage | Read
LINGUIST | Contact us
While the LINGUIST List makes every effort to ensure the linguistic relevance of sites listed
on its pages, it cannot vouch for their contents.