Academic Paper |
|
|
|
|
| Title: | Wisdom of crowds versus wisdom of linguists – measuring the semantic relatedness of words |
| Author: | Torsten Zesch |
| Institution: | Technische Universität Darmstadt |
| Author: | Iryna Gurevych |
| Institution: | Technische Universität Darmstadt |
| Linguistic Field: | Computational Linguistics; Semantics; Text/Corpus Linguistics |
| Subject Language: |
English
German |
| Abstract: |
'In this article, we present a comprehensive study aimed at computing semantic relatedness of word pairs. We analyze the performance of a large number of semantic relatedness measures proposed in the literature with respect to different experimental conditions, such as (i) the datasets employed, (ii) the language (English or German), (iii) the underlying knowledge source, and (iv) the evaluation task (computing scores of semantic relatedness, ranking word pairs, solving word choice problems). To our knowledge, this study is the first to systematically analyze semantic relatedness on a large number of datasets with different properties, while emphasizing the role of the knowledge source compiled either by the ‘wisdom of linguists’ (i.e., classical wordnets) or by the ‘wisdom of crowds’ (i.e., collaboratively constructed knowledge sources like Wikipedia). The article discusses benefits and drawbacks of different approaches to evaluating semantic relatedness. We show that results should be interpreted carefully to evaluate particular aspects of semantic relatedness. For the first time, we employ a vector based measure of semantic relatedness, relying on a concept space built from documents, to the first paragraph of Wikipedia articles, to English WordNet glosses, and to GermaNet based pseudo glosses. Contrary to previous research (Strube and Ponzetto 2006; Gabrilovich and Markovitch 2007; Zesch et al. 2007), we find that ‘wisdom of crowds’ based resources are not superior to ‘wisdom of linguists’ based resources. We also find that using the first paragraph of a Wikipedia article as opposed to the whole article leads to better precision, but decreases recall. Finally, we present two systems that were developed to aid the experiments presented herein and are freely available for research purposes: (i) DEXTRACT, a software to semi-automatically construct corpus-driven semantic relatedness datasets, and (ii) JWPL, a Java-based high-performance Wikipedia Application Programming Interface (API) for building natural language processing (NLP) applications. |
|
|
|
|
This article appears in Natural Language Engineering Vol. 16, Issue 1, which you can read on Cambridge's site or on LINGUIST . |
|
|
|
|
Back
Add a new paper Return to Academic Papers main page Return to Directory of Linguists main page |
|
|
About LINGUIST
|
Contact Us
|
Report Broken Link
While the LINGUIST List makes every effort to ensure the linguistic relevance of sites listed on its pages, it cannot vouch for their contents. |
|
