Academic Paper |
|
|
|
|
| Title: | A unified alignment algorithm for bilingual data |
| Author: | Christoph Tillmann |
| Institution: | IBM T.J. Watson Research Center |
| Author: | Sanjika Hewavitharana |
| Institution: | Carnegie Mellon University |
| Linguistic Field: | Computational Linguistics |
| Abstract: | The paper presents a novel unified algorithm for aligning sentences with their translations in bilingual data. With the help of ideas from a stack-based dynamic programming decoder for speech recognition (Ney 1984), the search is parametrized in a novel way such that the unified algorithm can be used on various types of data that have been previously handled by separate implementations: the extracted text chunk pairs can be either sub-sentential pairs, one-to-one, or many-to-many sentence-level pairs. The one-stage search algorithm is carried out in a single run over the data. Its memory requirements are independent of the length of the source document, and it is applicable to sentence-level parallel as well as comparable data. With the help of a unified beam-search candidate pruning, the algorithm is very efficient: it avoids any document-level pre-filtering and uses less restrictive sentence-level filtering. Results are presented on a Russian–English, a Spanish–English, and an Arabic–English extraction task. Based on simple word-based scoring features, text chunk pairs are extracted out of several trillion candidates, where the search is carried out on 300 processors in parallel. |
|
|
|
|
This article appears in Natural Language Engineering Vol. 19, Issue 1, which you can read on Cambridge's site or on LINGUIST . |
|
|
|
|
Back
Add a new paper Return to Academic Papers main page Return to Directory of Linguists main page |
|
|
About LINGUIST
|
Contact Us
|
Report Broken Link
While the LINGUIST List makes every effort to ensure the linguistic relevance of sites listed on its pages, it cannot vouch for their contents. |
|
