LINGUIST List 25.3898

Fri Oct 03 2014

Confs: Computational Linguistics/USA

Editor for this issue: Anna White <>

Date: 02-Oct-2014
From: Ekaterina Shutova <>
Subject: SemEval 2015 Task 11: Sentiment Analysis of Figurative Language in Twitter
E-mail this message to a friend

SemEval 2015 Task 11: Sentiment Analysis of Figurative Language in Twitter

Date: 01-Jun-2015 - 01-Aug-2015
Location: ---, USA
Contact: Ekaterina Shutova
Contact Email: < click here to access email >
Meeting URL:

Linguistic Field(s): Computational Linguistics

Meeting Description:

TASK11SEMEVAL-2015: Sentiment Analysis of Figurative Language in Twitter

One of the most difficult problems when assigning either positive or negative polarity in sentiment analysis tasks is to accurately determine what is the truth value of a certain statement. In the case of literal language, when there is not a secondary meaning, existing techniques already achieve good results. However, in case of figurative language such as irony and affective metaphor, when secondary or extended meanings are intentionally profiled, the affective polarity of the literal meaning may contrast sharply with the affect created by the figurative meaning. Nowhere is this effect more pronounced than in ironic or sarcastic language, which delights in using affirmative language to convey negative meanings. Metaphor, irony and figurative language more generally demonstrate the limits of conventional techniques for the sentiment analysis of literal texts.

So figurative language creates a significant challenge for a sentiment analysis system, as direct approaches based on words and their lexical semantics are often shown to be inadequate in the face of indirect meanings. It would be convenient then if such language were rare and confined to specific genres of text, such as poetry and literature. Yet the reality is that figurative language is pervasive in almost any genre of text, and is especially commonplace on the texts of the Web and in social media communications. Figurative language often draws attention to itself as a creative artifact, but is just as likely to be viewed as part of the general fabric of human communication. In any case, Web users employ figures of speech (both old and new) to project their personality through a text, especially when limited to the 140 characters of a tweet.

This significant challenge is the basis for SemEval Task 11, for which trial and training data are now available!

The Task:

Given a set of tweets that are rich in irony and metaphor, the goal is to determine whether the user has expressed a positive, negative or neutral sentiment. A fine-grained sentiment scale (scores in the range -5...+5) is be used to capture the effect of irony and metaphor on the perceived sentiment of a tweet. Participating systems will have to assign sentiment scores from the same fine-grained scale to a set of given tweets.

Trial (1000 figurative tweets) and training data (8000 figurative tweets): annotated and available at

Important Dates:

December 5, 2014: Evaluation starts
December 20, 2014: Evaluation ends
January 30, 2015: Paper submission due
February 28, 2015: Paper reviews due
March 30, 2015: Camera-ready papers due
Summer 2015: SemEval workshop



Tony Veale, University College Dublin
John Barnden, University of Birmingham
Antonio Reyes, ISIT
Paolo Rosso, Univiversitat Politècnica de València
Ekaterina Shutova, UC Berkeley

Page Updated: 03-Oct-2014