From: Albert Gatt <albert.gattum.edu.mt>
Subject: Using Corpora in NLG (Evaluation Special Theme)
E-mail this message to a friend
Full Title: Using Corpora in NLG (Evaluation Special Theme)
Short Title: UCNLG+Eval
Date: 31-Jul-2011 - 31-Jul-2011
Location: Edinburgh, United Kingdom
Contact Person: Albert Gatt
Meeting Email: < click here to access email >
Web Site: http://www.nltg.brighton.ac.uk/ucnlg/ucnlg11/
Linguistic Field(s): Cognitive Science; Computational Linguistics
Call Deadline: 22-Apr-2011
EMNLP-11 Workshop on Language Generation and Evaluation
Language Generation and Evaluation (UCNLG+Eval) is a post-conference workshop at EMNLP-2011, Edinburgh, on 31 July 2011.
There are many branches of NLP research that involve the generation of language (summarisation, MT, human-computer dialogue, application front-ends, data-to-text generation, document authoring, etc.). However, it is not always easy to identify common ground among the generation components of these application areas, which has sometimes made it difficult for generic research in 'Natural Language Generation' (NLG) to engage with them effectively. Increasingly common corpus-based approaches across these areas, and in particular in NLG itself, offer a new perspective on this situation and the opportunity to explore synergies and differences from the common grounding of corpus data.
This workshop is the fourth in an occasional series seeking to provide a forum for discussing NLG and its links with these closely related fields from a corpus-oriented perspective. The workshops have the general aims
-To provide a forum for reporting and discussing corpus-oriented methods for generating language;
-To foster cross-fertilisation between NLG and other fields where language is automatically generated; and
-To promote the sharing of data and methods for the purpose of system building and comparative evaluation in all language generation research.
Each of these workshops has a special theme: at the first workshop (at Corpus Linguistics in 2005) it was use of corpora in NLG; at the second (UCNLG+MT at MT Summit XI in 2007) it was Language Generation and Machine Translation; at the third it was Language Generation and Summarisation (UCNLG+Sum at ACL-IJCNLP'09). The special theme of the 2011 workshop is Language Generation and Evaluation, and the event will showcase recent developments in methods for evaluating computationally generated language across NLP, continue the discussion of future directions and host an invited talk on shared-task evaluation campaigns.
Evaluation Special Theme:
The past five years have seen big changes in NLG evaluation. The field has moved from a situation where there were no comparative evaluation results for independently developed alternative approaches to the present increasingly rich diversity of data sets, methods and results for comparative evaluation (intrinsic and extrinsic, human-assessed and automatically computed). A distinctive and critical feature of these developments has been the community-led approach to the establishment of tasks, datasets and evaluation methods. The aim of the special evaluation theme at UCNLG+Eval is to provide a forum for reporting cutting-edge research on evaluation, taking stock of recent developments, discussing and comparing alternative approaches to evaluation and exploring possible directions for future development.
Call for Papers:
The UCNLG+Eval Workshop organisers invite submissions addressing the special theme of evaluating computationally generated text as well as submissions on all aspects of using corpora in the generation of language. Specific topics include, but are not limited to:
-Statistical and machine learning approaches to language generation
-Development and annotation of corpora for language generation research
-Reuse of corpus resources developed for NLU (e.g. treebanks) in language generation research
-Domain-specific vs. general-purpose corpora for language generation research
-Evaluation of automatically generated language
-Meta-evaluation of evaluation methods for language generation
-Uses of corpora in the evaluation of automatically generated language
-Proposals for new shared tasks in language generation
Note that by 'language generation research' we mean any field in which language is automatically generated including research commonly coming under the headings of NLG, MT, document summarisation and human-computer dialogue.
Papers should describe original and unpublished work, emphasizing actual rather than intended work, and should indicate clearly the state of completion of the reported work. Wherever appropriate, concrete evaluation results should be included. Papers that are being submitted to other conferences or workshops should indicate this.
Submissions should follow the two-column format of the ACL 2011 proceedings. Each paper may consist of up to nine (9) pages of content, and any number of additional pages with references only. Only submissions in pdf format will be accepted. Submissions should describe original, unpublished work. Please use the official ACL 2011 style files (at http://www.acl2011.org/call.shtml). We reserve the right to reject submissions that do not conform to these styles, including letter size and font size restrictions. Submission is electronic, using the START system, via a link on the workshop website.
The deadline for submission is 22 April 2011.
Each paper will be reviewed by at least three programme committee members. Final decisions on the technical programme will be made by the workshop organisers.
As reviewing will be blind, papers should not include the authors' names and affiliations. Self-references that reveal the authors' identity, e.g., 'We previously showed (Smith, 1991) ...', should be avoided. Instead, use citations such as ''Smith previously showed (Smith, 1991) ...''. Acknowledgments sections should be removed before submission.
Papers will be reviewed according to the following criteria:
- Meaningful comparison
- Sharing of resources
A link to the evaluation form used by reviewers is available from the workshop website.
The proceedings of the workshop will be edited by the workshop organisers and published by the EMNLP 2011 conference organisers.
22 April 2011: Deadline for papers submissions
20 May 2011: Notification of acceptance to authors of workshop papers
03 June 2011: Camera-ready copies due
31 July 2011: UCNLG+EVAL workshop in Edinburgh
Anja Belz, NLTG, University of Brighton, UK
Roger Evans, NLTG, University of Brighton, UK
Albert Gatt, University of Malta, Malta
Kristina Striegnitz, Union College, USA
Aoife Cahill, Stuttgart University, Germany
Charlie Greenbacker, University of Delaware, USA
Emiel Krahmer, Tilburg University, NL
Mirella Lapata, University of Edinburgh, UK
Oliver Lemon, Heriot-Watt University, Edinburgh, UK
Daniel Marcu, ISI, University of Southern California, USA
Kathy McKeown, Columbia, USA
Karolina Owczarzak, NIST, USA
Ehud Reiter, Aberdeen, UK
This Year the LINGUIST List hopes to raise $67,000. This money will go to help
keep the List running by supporting all of our Student Editors for the coming year.
See below for donation instructions, and don't forget to check out Fund
Drive 2011 site!
There are many ways to donate to LINGUIST!
You can donate right now using our secure credit card form at
Alternatively you can also pledge right now and pay later. To do so, go to:
For all information on donating and pledging, including information on how to
donate by check, money order, or wire transfer, please visit:
The LINGUIST List is under the umbrella of Eastern Michigan University and as
such can receive donations through the EMU Foundation, which is a registered
501(c) Non Profit organization. Our Federal Tax number is 38-6005986. These
donations can be offset against your federal and sometimes your state tax return
(U.S. tax payers only). For more information visit the IRS Web-Site, or contact
your financial advisor.
Many companies also offer a gift matching program, such that they will match
any gift you make to a non-profit organization. Normally this entails your
contacting your human resources department and sending us a form that the
EMU Foundation fills in and returns to your employer. This is generally a simple
administrative procedure that doubles the value of your gift to LINGUIST, without
costing you an extra penny. Please take a moment to check if your company
operates such a program.
Thank you very much for your support of LINGUIST!
Read more issues|LINGUIST home page|Top of issue