LINGUIST List 29.2939

Tue Jul 17 2018

Calls: Italian; Computational Linguistics/Italy

Editor for this issue: Everett Green <>

Date: 17-Jul-2018
From: Tommaso Caselli <>
Subject: EVALITA 2018 6th Evaluation Campaign of NLP and Speech Technology for Italian
E-mail this message to a friend

Full Title: EVALITA 2018 6th Evaluation Campaign of NLP and Speech Technology for Italian

Date: 12-Dec-2018 - 13-Dec-2018
Location: Turin, Italy
Contact Person: Tommaso Caselli
Meeting Email:
Web Site:

Linguistic Field(s): Computational Linguistics

Subject Language(s): Italian (ita)

Call Deadline: 03-Sep-2018

Meeting Description:

The 6th Evaluation of NLP and Speech Tools for Italian

The 6th evaluation campaign EVALITA 2018 will be held in Turin, as a satellite
event of the 5th Italian Conference on Computational Linguistics - CLiC-it
2018 (December 10-12, 2018).
EVALITA 2018 is an initiative of AILC (Associazione Italiana di Linguistica

As in the previous editions (, EVALITA 2018 will be
organized along few selected tasks, which provide participants with
opportunities to discuss and explore both emerging and traditional areas of
Natural Language Processing and Speech for Italian. The participation is
encouraged for teams working both in academic institutions and industrial

Final Call for Papers:

EVALITA 2018 - Final Call for Registration + Best System Award Announcement

The general objective of EVALITA ( is to promote the
development of language technologies for the Italian language, providing a
shared framework where different systems and approaches can be evaluated in a
consistent manner. EVALITA is an initiative of the Italian Association for
Computational Linguistics (AILC) and it is endorsed by the Italian Association
for Artificial Intelligence (AI*IA) and the Italian Association for Speech
Sciences (AISV).

For the 2018 edition tasks are organized along the following tracks:

Affect, Creativity and Style
- ABSITA - Aspect-based Sentiment Analysis
- ITAMoji - Italian Emoji Prediction
- IronITA - Irony Detection in Twitter
- GxG - Cross-Genre Gender Prediction

Dialogue Systems
- iLISTEN - itaLIan Speech acT labEliNg
- IDIAL - Italian DIALogue systems evaluation

Hate Speech
- AMI - Automatic Misogyny Identification (aligned with IberEval2018)
- HaSpeeDe - Hate Speech Detection

- NLP4FUN - Solving language games
- SUGAR - Spoken Utterances Guiding chef’s Assistant Robots

The complete list of tasks is available here:

The training data are available already for all tasks and registration are
open until September 3, 2018. Please visit the EVALITA web site where
information about all tasks is currently available, and indicate your
intention to participate and the specific task(s) you are interested in. It’s
easy! You can do so by filling out the registration form at:

As usual, the results of the evaluation will be disseminated at the final
workshop, which will be organised in Turin as a co-located event of the 5th
Italian Conference on Computational Linguistics - CLiC-it 2018 (December
10-12, 2018).

We encourage participation from both academic institutions and industrial

Best system Evalita 2018

In line with CliC-it, for this edition of the EVALITA Campaign, we will award
the work of the participants, especially that of young researchers.
Specifically, we will be awarding the “best system” across tasks.

In order to do so, we will solicit from the task organizers a shortlist of
two or three papers describing systems that identified as eligible candidates
for the award, based on the criteria for evaluation listed below. The
shortlist should include only papers for which at least a PhD candidate, a
master or a bachelor student is among the authors. Based on the task
organizers recommendations, we will evaluate the systems in the shortlists and
select the best system for Evalita 2018 emerging from all the tasks.

Criteria for evaluation are:

- Novelty, to be decline as novelty of the approach with respect to the state
of the art (e.g. a new model or algorithm), or novelty of features (for
discrete classifiers);
- Originality, to be declined as identification of new linguistic resources
employed to solve the task (for instance, using WordNet should not be
considered as a new resource), or identification of linguistically motivated
features; or implementation of theoretical framework grounded in linguistics;
- Critical insight, to be declined as a deep error analysis that highlights
limits of the current system and pave direction to future challenges
- Technical soundness and methodological rigor.

EVALITA-2018 Chairs:

Tommaso Caselli (Center for Language and Cognition Groningen,
Rijksuniversiteit Groningen, The Netherlands)
Nicole Novielli (Dipartimento di Informatica, Università degli Studi di Bari
“A. Moro”, Italy)
Viviana Patti (Dipartimento di Informatica, Università degli Studi di Torino,
Paolo Rosso (PRHLT Research Center, Universitat Politècnica de València,


Page Updated: 17-Jul-2018