Publishing Partner: Cambridge University Press CUP Extra Wiley-Blackwell Publisher Login

FYI: Call for Participation: CLEF-ER Challenge

Author: Johannes Hellrich

Linguistic Field(s): Computational Linguistics

Subject Language(s): Dutch

FYI Body: Call for Participation

CLEF-ER 2013: Named Entity Recognition (NER) in parallel multilingual biomedical corpora (aiming for terminology translation)

Objectives: Multilingual identification of mentions of named entities (attribution of CUIs) in corpora, where each corpus is either in English, French, German, Spanish, and Dutch. Currently patent texts, titles of Medline abstracts and EMEA documents serve as parallel corpora. The mention annotations will be evaluated as part of the challenge and, in addition, the mention annotations will be analysed to contribute to an augmented biomedical terminological resource.

Task: Participants produce mention annotations in a multilingual document where the annotations make explicit reference to entities, which are specified through a Concept Unique Identifier (CUI, from the UMLS). Challenge participants should at least contribute one non-English annotated corpus and have to register at the submission site (

A number of resources are available too:
(1) a terminological resource (TR) containing English and non-English concepts in combination with their CUIs,
(2) a selection of corpora in English, where the entity mentions have been annotated automatically with their CUIs, and
(3) a selection of corpora in different languages other than English (i.e., de, fr, sp, nl) that have to be annotated with entity mentions and their CUIs.

Note: Each non-English corpus has a parallel English corpus. Not all languages (de, fr, sp, nl) are available for each corpus. Parallel corpora offer the opportunity to use machine-translation or bi-lingual term identification methods.

The results will be presented on the CLEF-ER workshop, which is part of the CLEF 2013 Conference, 23-26 September in Valencia, Spain. The challenge is sponsored by the EU project “Mantra”. For more details on the task please visit or send an email to


15.12.2012: CLEF and CLEF-ER registration opens

5.02.2013: English and non-English corpora available, including consensus corpora in English

30.04.2013: submission system for runs closes

31.05.2013: preliminary evaluation results released

15.06.2013: deadline for submission of working notes papers

23.09.2013: CLEF 2013 conference starts (26.09.2013: CLEF 2013 conference ends)


Dr. Dietrich Rebholz-Schuhmann, Dr. Fabio Rinaldi, Dr. Simon Clematide, Institute of Computational Linguistics, Universität Zürich (Ch)

Prof. Dr. Udo Hahn, Johannes Hellrich, Julie Laboratory, Universität Jena (D)

Dr. Jan Kors, Dr. Erik van Mulligen, Erasmus University Medical Center, Rotterdam (Nl)

Dr. David Milward, Dr. Ian Lewin, Linguamatics Ltd, Cambridge (UK)
Dr. Philip Daumke, Michael Poprat, Averbis GmbH, Freiburg (D)

Back   FYI main page