LINGUIST List 25.1817

Tue Apr 22 2014

FYI: Call For Participation: TREC 2014 Temporal Summarization Track

Editor for this issue: Sara Couture <>

Date: 21-Apr-2014
From: Fernando Diaz <>
Subject: Call For Participation: TREC 2014 Temporal Summarization Track
E-mail this message to a friend

Call For Participation

TREC 2014 Temporal Summarization Track

Unexpected news events such as natural disasters or mass protests represent a unique information access problem where the performance of traditional approaches deteriorates. For example, immediately after an event, the corpus may be sparsely populated with relevant content. Even when, after a few hours, relevant content is available, it is often inaccurate or highly redundant. At the same time, crisis events demonstrate a scenario where users urgently need information, especially if they are directly affected by the event. The goal of the Temporal Summarization track is to develop systems which allow users to efficiently monitor in real time the information associated with an event over time. Specifically, we are interested in developing systems which can detect useful, new, and timely sentence-length updates about a developing event.

The track organizers provide a standardized experimentation platform, including a corpus (developed by and shared with the Knowledge Base Acceleration track), test events, relevance judgments, and evaluation metrics. In order to encourage participation, we will be releasing a smaller, filtered version of the KBA corpus. All participants will have access to last year's corpus, training events (and judgments), as well as evaluation scripts.

We anticipate the following schedule:
April 15, 2014: preliminary material released (corpus, training events with judgments, evaluation scripts available at
Late May 2014: evaluation topics released; smaller, filtered corpus available
August 2014: participants submit runs
October 2014: results released
November 18-21, 2014: TREC 2014 Conference

Application Details
Organizations wishing to participate in TREC 2014 Temporal Summarization Track should respond to this call for participation by submitting an application. Participants in previous TRECs who wish to participate in TREC 2014 must submit a new application. To apply, submit the online application at

The application system will send an acknowledgement to the email address supplied in the form once it has processed the form.

Any questions about conference participation should be sent to the general TREC email address, trec (at)

About TREC
The Text Retrieval Conference (TREC) workshop series encourages research in information retrieval and related applications by providing a large test collection, uniform scoring procedures, and a forum for organizations interested in comparing their results. Now in its 23rd year, the conference has become the major experimental effort in the field. Participants in the previous TREC conferences have examined a wide variety of retrieval techniques and retrieval environments, including cross-language retrieval, retrieval of web documents, multimedia retrieval, and question answering. Details about TREC can be found at the TREC web site,

Linguistic Field(s): Computational Linguistics; Text/Corpus Linguistics

Page Updated: 22-Apr-2014