Publishing Partner: Cambridge University Press CUP Extra Publisher Login

New from Cambridge University Press!


Revitalizing Endangered Languages

Edited by Justyna Olko & Julia Sallabank

Revitalizing Endangered Languages "This guidebook provides ideas and strategies, as well as some background, to help with the effective revitalization of endangered languages. It covers a broad scope of themes including effective planning, benefits, wellbeing, economic aspects, attitudes and ideologies."

New from Wiley!


We Have a New Site!

With the help of your donations we have been making good progress on designing and launching our new website! Check it out at!
***We are still in our beta stages for the new site--if you have any feedback, be sure to let us know at***

Review of  Translation Quality Assessment

Reviewer: Zhi Huang
Book Title: Translation Quality Assessment
Book Author: Joss Moorkens Sheila Castilho Federico Gaspari Stephen Doherty
Publisher: Springer Nature
Linguistic Field(s): Computational Linguistics
Issue Number: 30.720

Discuss this Review
Help on Posting

“Translation Quality Assessment: From Principles to Practice”, edited by Joss Moorkens, Sheila Castilho, Federico Gaspari and Stephen Doherty, appears in Volume 1 of book series “Machine Translation: Technologies and Applications”, edited by Andy Way. This volume has three parts, each focusing on one aspect of translation quality assessment. Part I discusses scenarios for translation quality assessment and contains four papers. Part II explores the development of applications of translation quality assessment and includes four papers. Part III examines translation quality assessment in practice and contains three papers. A combination of the articles in this book tries to link principles with practice in translation quality assessment, including descriptions of approaches to translation quality assessment, metrics and error analysis for translation quality assessment, as well as applications of machine translation and future perspectives on its quality. The book is intended for those who are interested in machine translation evaluation, translation quality assessment technologies, and applications and practice of translation quality assessment.

Part I: Scenarios for Translation Quality Assessment

Paper 1: Approaches to Human and Machine Translation Quality Assessment (Sheila Castilho, Stephen Doherty, Federico Gaspari, and Joss Moorkens)

This paper introduces human and machine translation quality assessment and approaches to the measurement of translation quality in different settings. A number of automatic machine translation evaluation metrics are discussed in terms of standards, consistency, social quality, risks, and implications for education and training. Examples include Word Error Rate used by Nieben et al. (2000), Translation Error Rate by Snover et al. (2006) and Bilingual Evaluation Understudy by Papineni et al. (2002). It provides an insight into the current knowledge and practice of translation quality assessment in research, industry and education.

Paper 2: Translation Quality, Quality Management and Agency: Principles and Practice in the European Union Institutions (Joanna Drugan, Ingemar Strandvik, and Erkka Vuorinen)

This paper explores translation quality management and agency in the setting of the European Union institutions, particularly European Commission’s Directorate-General for Translation (DGT). DGT’s quality management model and practical quality management tools are discussed, followed by challenges in consistency of approaches and quality. The impact and implications of translation quality management are also considered, with regard to power, agency, professionalism, and values.

Paper 3: Crowdsourcing Translation Quality: Novel Approaches in the Language Industry and Translation Studies (Miguel A. Jimenez-Crespo)

In this paper, the practice of crowdsourcing is introduced to explore its impact on translation quality. The papaer reviews the consolidation of process-based approaches to guarantee quality, the expansion of the fitness for purpose model, and the distribution of responsibility to different agents. The paper also explores novel practices and workflows to guarantee quality focusing on contrasting professionals or crowdsourcing participants to assess the quality of MT output.

Paper 4: On Education and Training in Translation Quality Assessment (Stephen Doherty, Joss Moorkens, Federico Gaspari, and Sheila Castilho)

This paper acknowledges the lack of education and training in translation quality assessment and introduces a range of viewpoints and resources for the provision of education and training in academic settings. It proposes Translation Quality Assessment models for education and training and provides a guide to help educators and translators choose the various types of TQA compatible with their own scenarios. At the end, the paper mentions the current status of TQA education and training and gives predicts for its development trend, confirming that the industry will have its own metrics and models that will drive the industry to change for the better.

Part II: Development of Applications of Translation Quality Assessment

Paper 5: Metrics for Translation Quality Assessment: A Case for Standardising Error Typologies (Arle Lommel)

This paper starts with an introduction of the Multidimensional Quality Metrics (MQM) and Dynamic Quality Framework (DQF) projects for translation quality assessment. An overview of MQM is detailed focusing on its existing approaches and the overall structure of MQM including hierarchy, dimensions, specifications, severities and weights, scoring, and holistic vs analytic evaluation. The DQF error typology is then discussed and integrated with MQM with concluding remarks on the status and plans for the future, offering a way to escape the inconsistency and subjectivity that have so far characterised TQA.

Paper 6: Error Classification and Analysis for Machine Translation Quality Assessment (Maja Popovic)

This paper explores the different approaches and tasks to analyse errors in machine translation output. It starts with an introduction to manual error classification and error typologies, providing a table of error categories, followed by an overview of error typologies and tasks with detailed analysis. The challenges for automatic error classification are also discussed and analysed with an evaluation as well. Other methods for error analysis are also explored, such as analysis of (un)matched sequences, and checking and evaluating specific linguistic features.

Paper 7: Quality Expectations of Machine Translation (Andy Way)

This paper addresses whether machine translation can be useful for human translators, especially as a productivity enhancer. It starts with the background information on machine translation use and evaluation at present, and then considers the inherent problems with automatic evaluation metrics and problems with automatic evaluation use. It also answers the question whether automatic evaluation corroborates human evaluation. Finally, the paper discusses how MT is currently applied and what expectations and perceptions of MT quality are. The author concludes this paper with a prediction of its continued use as a production tool besides translation memory.

Paper 8: Assessing Quality in Human- and Machine-Generated Subtitles and Captions (Stephen Doherty and Jan-Louis Kruger)

This paper explores the current and future issues in quality assessment in human and machine-generated subtitling and captioning. The guiding principles for quality in audiovisual translation (AVT), such as accuracy, presentation, timing and error-based metrics, are discussed first, followed by the implications and insights from AVT research with a focus on the cognitive load and immersion in subtitled and captioned media. Finally, the paper raises challenges and possible solutions for stakeholders to consider in order to encourage dialogue between disciplines so that the quality in AVT can be enhanced and further developed in such an evolving technological environment.

Part III: Translation Quality Assessment in Practice

Paper 9: Machine Translation Quality Estimation: Applications and Future Perspectives (Lucia Specia and Kashif Shah)

This paper addresses the importance of predicting the quality of machine translation output with a review of various practical applications. Experiments are carried out to show positive results in quality estimation at sentence level, that is, filtering low quality cases from post-editing, selecting the best machine translation system under multiple choices, improving machine translation performance by selecting parallel data and sampling for human quality assurance. In the conclusion, the author asserts that the approaches to quality estimation have the potential to make machine translation more useful to end-users.

Paper 10: Machine Translation and Self-post-editing for Academic Writing Support: Quality Explorations (Sharon O’Brien, Michel Simard, and Marie-Josee Goulet)

This paper discusses the possibility of using machine translation and self-post-editing as a second-language academic writing aid through a number of quality assessment measures. It also compares participant perceptions, temporal effort and revisions required. The results show that more participants prefer to use machine translation again in the future to support their writing process and participants believe that the quality is maintained. This suggests a high potential of using machine translation and self-post-editing as a useful tool when accessing international academic publishing via the current Lingua Franca of English so that the cognitive burden of the authors can be reduced.

Paper 11: What Level of Quality Can Neural Machine Translation Attain on Literary Text? (Antonio Toral and Andy Way)

This paper introduces a new neural approach to machine translation and assesses the quality attainable for novels by the two most common paradigms to machine translation, NMT and PBSMT. The authors built the first in-domain PBSMT and NMT systems for literary text by training them on large volumes of parallel novels. With a comparison of the output from machine translation systems and that from published human translations, surprisingly promising results are shown, especially considering this special text type. It is of great value for future research in terms of the assessment of the feasibility of using machine translation to assist with the translation of literary text.


This book is nicely presented with each chapter contributing an important aspect of machine translation quality assessment. The focus of this book is on the product of translation, describing scenarios for translation quality assessment and developing applications of translation quality assessment. The three parts of this book cover the practical side of translation quality assessment in an emerging technological society. At this critical moment for the translation industry, when fast-paced economy and social development require more efficient and effective translation products, I believe this book provides essential knowledge and insights into translation technology and quality assurance. Its target readers can be all kinds of industry practitioners as well as enthusiasts, including but not limited to translators, teachers, students, researchers or business managers. The book embraces dominant methods from different translation scenarios, and provides a comprehensive collection of contributions by international experts in translation quality assessment and human and machine translation evaluation. The whole book is well structured to form a picture of human and machine translation quality assessment from principles to practice.

Although it is understandable that this book focuses on machine translation quality assessment because of the fast development of technology and the trend of using such a technology in translation, my suggestion would be some more focus on human translation quality assessment and the comparison of human and machine translation quality assessment, to enrich the range of topics covered under this title “Translation Quality Assessment”. Without a balance of research on both human and machine translation quality assessment, I think the book may be more suitably called “Machine Translation Quality Assessment”. I would also be interested to read more articles about the applications of machine translation quality assessment in practical scenarios including translation teacher development, translation teaching techniques, use of machine translation in more settings, and the possibility of using machine translation on more text types.


Nieben, Sonja, Franz Jisef Och, George Leusch, and Hermann Ney. 2000. An evaluation tool for machine translation: fast evaluation for MT research. In: Proceedings of the second international conference on language resources and evaluation, Athens, 31 May-2 June 2000. 39-45.

Snover, Matthew, Bonnie Dorr, Richard Schwartz, Linnea Micciulla, and John Makhoul. 2006. A study of translation edit rate with targeted human annotation. In: Proceedings of the 7th conference of the Association for Machine Translation in the Americas: “Visions for the future of Machine Translation”, Cambridge, 8-12 August 2006. 223-231.

Papineni, Kishore, Salim Roukos, Todd Ward, and Wei-Jing Zhu. 2002. BLEU: a method for automatic evaluation of machine translation. In: Proceedings of the 40th annual meeting on Association for Computational Linguistics, Philadelphia. 311-318.
Zhi Huang is a PhD candidate at Macquarie University focusing on translation teacher effectiveness and effective teaching qualities. He is an Australian NAATI certified translator between Chinese and English languages. He has completed Master of Advanced Translation at Macquarie University and Master of Education in TESOL at the University of Sydney. His research interests involve English language teaching, teacher quality, translation theory and pedagogy. He has published articles at English Language Teaching, T&I Review, and Journal of Language Teaching and Learning.