LINGUIST List 20.3641
|
Wed Oct 28 2009
FYI: Call for Participation: GREC 2010 Shared Tasks
Editor for this issue: Danielle St. Jean
<danielle linguistlist.org>
|
To post to LINGUIST, use our convenient web form at
http://linguistlist.org/LL/posttolinguist.html.
|
Directory
1. Eric
Kow,
Call for Participation: GREC 2010 Shared Tasks
Message 1: Call for Participation: GREC 2010 Shared Tasks
|
Date: 26-Oct-2009
From: Eric Kow <E.Y.Kowbrighton.ac.uk>
Subject: Call for Participation: GREC 2010 Shared Tasks
 E-mail this message to a friend
First Call for Participation
Generation of References in Context (GREC) Tasks 2010
Part of Generation Challenges 2010, in conjunction with INLG 2010.
Generation Challenges is a shared-task evaluation initiative for research
that involves the generation of language, in the widest sense. Generation
Challenges 2010 is the fourth event in this initiative, and aims to
continue to provide a common forum for a number of different Shared Tasks (see
http://www.nltg.brighton.ac.uk/research/genchal10/).
As part of Generation Challenges 2010, we are organising three GREC Shared
Task Competitions. The first is the GREC-NEG (Named Entity Generation) task
which uses the GREC-People Corpus of 1,000 Wikipedia introduction sections
about people in which single and plural references to all people mentioned
in the text have been annotated; the task in GREC-NEG is to select
appropriate referential expressions for all mentions (singular and plural)
of people from a given list.
The second task, GREC-NER, is a straightforward Named Entity Recognition
task, using the new GREC-People-II extension to the GREC-People Corpus. The
third task is GREC-FULL which requires participating systems to combine NER
and NEG in order to improve references to people in texts, and this also
uses the GREC-People-II Corpus.
Submissions to all three tasks will be evaluated using a range of intrinsic
and extrinsic measures, some assessed automatically, some manually.
Submitted systems and evaluation results will be presented in a special
session at INLG'10 in Dublin in July 2010, and published in the INLG'10
proceedings.
1. Background
There has been increasing interest recently among text summarisation
researchers in postprocessing techniques to improve the referential clarity
and coherence of extractive summaries, and among language generation
researchers in generating referential expressions in context. The GREC
tasks are aimed at researchers in both of these groups, and the objective
is the development of methods for generating chains of referential
expressions for discourse entities in the context of a written discourse,
as is useful e.g. for postprocessing extractive summaries and repeatedly
edited texts (such as Wikipedia articles).
2. Data
The GREC data resources consist of introduction sections collected from
Wikipedia articles in which several broad syntactic categories of reference
to named entities have been annotated. The annotations include features
encoding syntactic and semantic information.
The GREC-People and GREC-People-II corpora consist of 1,000 texts each
in one domain, Wikipedia articles about people. Here, all references,
singular and plural, to all people mentioned in a text have been annotated.
Each text in these corpora therefore includes explicit coreference
annotation for at least one coreference chain.
For GREC-People and GREC-People-II we have test sets of 100 texts each,
where referential expressions have been selected by participants in an
elicitation experiment. In these test sets, there are three versions for
each corpus text, in each of which the referential expressions have been
manually selected by a single participant in the experiment.
Full details of the evaluation methods for GREC'10 will be given in the
Participants' Pack that will be distributed to registered participants.
3. Participation
Registration is now open at the GREC'10 homepage
(http://www.nltg.brighton.ac.uk/research/genchal10/grec). Once registered,
participants in the GREC-NEG Task will receive the complete training and
development set, evaluation software and detailed documentation
(collectively known as the Participants' Pack) for this task. Participants
in the other two tasks will first receive a sample of the training and
development data, to enable them to start building systems; they will
receive the complete Participants' Pack for GREC-NER/FULL by the end of 2009.
4. Proceedings and Presentations
The Generation Challenges 2010 meeting will be held as a special session at
INLG 2010 in July 2010 in Dublin. The session will include overviews of all
the shared tasks, including the GREC'10 Tasks. The participating systems
will additionally be presented as papers in the INLG'10 proceedings, and as
posters during the INLG'10 poster session.
GREC'10 papers will not undergo a selection procedure with multiple
reviews, but the organisers reserve the right to reject material which is
not appropriate given the participation guidelines.
5. Important Dates
Oct 2009 - First Call for Participation in GREC'10 Tasks;
- GREC-NEG Participants' Pack available
- GREC-NER/FULL Data Sample available
Dec 31, 2009 - GREC-NER/FULL Participants Pack available
Apr 5-May 2, 2010 - Test data submission period; 4-step submission process:
1. Fill in submission form (available on website from Apr 5).
2. Upload 2-page paper describing approach and reporting development set
results.
3. Download test data (inputs only).
4. Submit test data outputs at the latest 48 hours after download, but in
any case no later than May 2.
May 2, 2010 - Final deadline for submission of test data outputs.
May 2-23, 2010 - GREC'10 Evaluation period
Jul 7-9, 2010 - Generation Challenges meeting at INLG'10
6. Organisation
Anja Belz, NLTG, University of Brighton, UK
Eric Kow, NLTG, University of Brighton, UK
Jette Viethen, Macquarie University, Australia
GREC'10 homepage: http://www.nltg.brighton.ac.uk/research/genchal10/grec
Generation Challenges homepage:
http://www.nltg.brighton.ac.uk/research/genchal10
Generation Challenges email: nlg-stecitri.brighton.ac.uk
Linguistic Field(s): Computational Linguistics
Read more issues|LINGUIST home page|Top of issue
|
|
Please report any bad links or misclassified data
LINGUIST Homepage | Read
LINGUIST | Contact us
While the LINGUIST List makes every effort to ensure the linguistic relevance of sites listed
on its pages, it cannot vouch for their contents.
|
|
