LINGUIST List 32.1088
Thu Mar 25 2021
Review: Computational Linguistics; Pragmatics; Semantics: Bender, Lascarides (2019)
Editor for this issue: Jeremy Coburn <jecoburnlinguistlist.org>
Maria-Jose Arrufat-Marques <arrufatm
Linguistic Fundamentals for Natural Language Processing II E-mail this message to a friend Discuss this message
Book announced at https://linguistlist.org/issues/30/30-4511.html
AUTHOR: Emily M. Bender
AUTHOR: Alex Lascarides
TITLE: Linguistic Fundamentals for Natural Language Processing II
SUBTITLE: 100 Essentials from Semantics and Pragmatics
SERIES TITLE: Synthesis Lectures on Human Language Technologies edited by Graeme Hirst
PUBLISHER: Morgan & Claypool Publishers
REVIEWER: Maria-Jose Arrufat-Marques, Universitat Jaume I
This riveting volume is a compilation of 100 topics distributed in 14 chapters that highlight the importance of linguistic knowledge, interactional context, and the semantic and pragmatic mechanisms behind language understanding and use, and their application for diverse natural language processing (NLP) tasks involving written and spoken data. Each topic from 1 to 100 is addressed as “#(number)”. For example, “#0” and “#1” are the first ones and found in the first paragraph below.
In Chapter 1, “Introduction”, #0 presents the main reasons why considering semantics and pragmatics is important for NLP systems building. As the authors state, the ultimate goal of NLP is to create systems able to transfer as much information across different “domains, tasks, and speakers” (p. 1), which can only be achieved once there is full comprehension of a language, including semantics and pragmatics. #1 delves into this idea. Considering common sense reasoning is essential in NLP, being knowledgeable about semantics can be advantageous in devising systems to understand and generate language.
Chapter 2, “What is meaning?”, explores fundamental ideas related to the meaning of natural language from a philosophical and formal semantics perspective and the positive effects it may bring to include them to construct NLP systems. #2 adopts a philosophical stance to describe the logic side of language. #3 asserts that common reasoning, coherence, and world knowledge are essential to comprehend natural languages. #4 tackles the different meaning levels (i.e.: semantics, content, and implicature knowledge) required to understand whole word meaning. #5 explains the differences and importance of locutionary, illocutionary, and perlocutionary acts; and distinguishes them from #4. #6 presents the importance of indirect speech acts and common ground to correctly interpret anaphora. #7 highlights that speech acts are not only properties of but also display relations among utterances. #8 and #9 show that linguistic meaning not only refers to the actual meaning of the word, but also the inclusion of emotions and social meaning. #10 discusses the ambiguous nature of natural languages. #11 examines language processing differences between NLP systems and humans. #12 and #13 illustrate that meaning in face-to-face interaction emerges from both verbal and non-verbal behaviors and the complex intricacy between them. #14 reports that meaning inference differs, being more abstract if using a compositional semantics approach, and more concrete if using coherence or cognitive cooperative techniques in interaction.
Chapter 3, “Lexical Semantics: Overview”, tackles some main points in lexical semantics. #15 discusses that understanding words from a formal grammar perspective implies missing out much of their meaning. Words carry predictable but also idiosyncratic features in their meaning and only from a more comprehensive understanding of words will they be modelled optimally. #16 tackles three aspects inherent in individual words: word sense, semantic roles, and connotations. #17 explores the applications of these three issues to groups of words: multiword expressions (MWE).
Chapter 4, “Lexical Semantic: Senses”, frames in depth the aforementioned idiosyncratic, predicable, or productive nature of senses, as well as the disadvantages and benefits it may bring to NLP systems. #18 explores the multiple meanings words can have, which may differ depending on the usage context and might transfer among languages. #19 and #20 tackle the various types and relations of regular polysemy and constructional polysemy respectively. #21 introduces homonymy, how it differs from polysemy, and the benefits it brings to NLP. #22 and #23 relate to #19 as the former two discuss a process to create new words and the latter linguistic and human issues that can impede and resolve meaning problems in such new words. #24 presents various ways that support the gradual meaning variation of words. #25 introduces vector space representations and #26 discusses it further in relation to word meaning identification from a formal semantics viewpoint. #27 to #29 deal with metaphors and the matters that play a role in their creation, meanings, and interpretations. #30 and #31 discuss the influence of transfer and defeasible information in word senses.
Chapter 5, “Semantic Roles”, discusses how parts of a sentence (arguments) relate to the linguistic object they refer to and to various linguistic resources used to display such arguments. #32 explains that some arguments exemplify the existence and variety of relationships among words or other arguments. #33 explores the idea that syntax and word meaning will enhance the identification of the different arguments in a sentence in relation to their predicate. #34 identifies the granularity by which semantic roles can be examined and thus classified. Some examples are given: FrameNet on one extreme, PropBank and English Resource Grammar on the other, and VerbNet in the middle. #35 provides some sentences where the foci are the verbs, by which the authors explain that depending on the verb used, we expect one type of specific complement (arguments) over others. #36 states that argument omission does not imply meaning ambiguity. For example, subject omission presents no problem in Japanese or Spanish, but it may be more problematic in English. Regardless, the meaning comes across since other linguistic resources are used. This issue is highlighted as essential to consider for NLP task generation.
Chapter 6 discusses “Collocations and Other Multiword Expressions” from a semantics perspective, highlighting the necessity of using grammars and parsers to consider their specific syntax and semantics features. #37 introduces MWEs, defined as co-existing groups of words having specific meaning and form and allowing for some variation regarding their syntax, semantic, pragmatic, or frequency of use. MWEs are considered single lexical units because they can have multiple different meanings, which can vary over time, and dynamic because new ones can be created. However, they pose some difficulties for NLP specialists due to their particular forms and being language specific. #38 focuses on collocations, a type of MWEs that depend both on their form and meaning for a successful use and understanding. They are difficult to learn -by humans and machines- due to their dialectal variation, less frequent use, and the fact that substituting a word within a collocation for a synonym will make an incorrect use and weird sense of the whole unit. #39 provides examples such as “make a cake” or “heavy sleeper” to explain that the sense of a collocation is clearer than the meaning of the words conforming it on their own. #40 discusses that collocation frequency and strength do not go hand in hand and that accessing and using large corpora and metrics will make the task of building up NLP systems more accurate and effective. #41 comments on the fact that MWEs are ambiguous and that sense extension can stem from a polysemic, homonymic, and metaphorical relationship between the senses. #42 tackles the different ways to generate MWEs. However, the resulting MWEs may be just one word; and this is a difficulty for building NLP systems, as they need to be trained to recognize all lexical particles that fulfill the same semantic sense. #43 discusses the phenomena of grammaticalization and idiomatization as two possible ways for a language to acquire new MWEs. #44 discusses the syntactic flexibility of MWEs, meaning that some do not allow any change -thus being fixed expressions- and others accept some word order change, i.e., idiomatically combining expressions. The authors express the need to develop parsers that identify both types of expressions in a precise and accurate manner. #45 tackles the phenomenon of compositionality in MWEs, a phenomenon by which certain types of MWEs will allow some variation(s) in their form that will not affect their meaning, and which happen in many languages. #46 depicts the difficulty that may be brought by trying to embody the meaning of the idiomatic expression with the meaning of the parts conforming it. Thus, this needs to be included in grammars, where the authors state the need to address this issue by for example flagging the specific meanings of certain words when used in idiomatic expressions, so that the grammar can identify each use accurately.
Chapter 7, “Compositional Semantics”, focuses on meaning representation composition from a formal semantic approach, being truth and reference key aspects related to compositional semantics, to ultimately make predictions about logical relationships. #47 introduces what compositional semantics is. It studies the representation of the predicate-argument structure by means of syntactic analyses. Attention is given to other important related issues, including wedge elimination, entailment, and validity. #48 presents different options to represent compositional semantics, such as semantic typing, grounded language acquisition, and broad coverage parsing. #49 discusses that expression of comparative relations is language dependent considering the following three essential elements may vary among languages: “the gradable property, the entity bearing the property, and a standard of comparison” (p. 77). #50 explains how subtle meaning differences may arise from interpreting coordinated sentences and states that systems used for NLU may need to be trained and prepared to discern such different semantic interpretations. #51 to #53 tackle quantifiers and how they semantically influence and relate to other components in a sentence or text from a set theory perspective, which is the base of formal semantics. In contrast to #53, #54 discusses that the meaning of other scopal operators (e.g., negation) can indeed be sorted out by closely looking at syntax. #55 states that the NLP application selected will decide how to best represent scopal operators, this not being critical for applications such as information retrieval, but it is essential for classification tasks. #56 reports on the interplay between word senses and compositional semantics as seen in syntactic analyses since the former are structured and semantically complex, including nouns and quantifiers. #57 discusses different ways to infer semantic equivalence, including distributional models of words, vector averaging, composition, or grammar-driven approaches.
The scope of Chapter 8, “Compositional Semantics Beyond Predicate-Argument Structure”, is summarized in #58: compositional semantics not only explores morphology and syntax, but also other language elements including tense, aspect, evidentials, presuppositions, and politeness. #59 focuses on the different ways that arguments can be identified, e.g.: word order, case marking, or agreement markers, as dictated by language. #60 discusses tense based on Reichenbach’s (1947) interpretation, in contrast to Prior’s (1957). Reichenbach proposes three key elements determining the use of a (past) tense: the speech time, event time, and reference time whose element it refers to anaphorically is specified by the situational context. #61 comments on aspect, as it portrays the internal temporal properties on a given event--situational aspect--and how it is perceived -viewpoint aspect. These two systems interact, the former being lexical and compositional in nature; and the second referring to the grammatical resources a language offers to express a temporal perspective of a given context. #62 tackles evidentials, which encompass the origin of the information a person refers to and how certain said person is about said information. Depending on the language, evidentiality may be grammaticalized or not. #63 and #64 discuss politeness, which can be expressed via grammaticalized forms, and other words, such as ‘please’ or the use of mitigators (‘could’ instead of ‘can’) to show social distance and save speaker’s face. #65 focuses on politeness markers to solve reference problems, explaining it with the Japanese honorific system.
Chapter 9, “Beyond Sentences”, looks into contextual elements essential to resolve ambiguity difficulties. #66 and #67, from a coherence-based model perspective, assert that to comprehend discursive meaning an update function is needed to understand the information conveyed in a coherent and structured manner and the context where this occurs. #68 illustrates that word meaning understanding will lead to understanding discourse and vice versa: word processing enhances discourse comprehension and a structured discourse facilitates word ambiguity solving. #69 explains discourse understanding from a dynamic semantics perspective. #70 discusses the pros and cons of using game theory to understand discourse and generate dialogues.
Chapter 10, “Reference Solution”, underscores the essential role of linguistic context to identify referents in discourse understanding and its NLP applications. #71 illustrates the importance of reference identification and application in NLP tasks to process discourse. #72 discusses the different types of referents. #73 describes various grammatical features affecting referents and their antecedents. #74 reports on the unambiguous, logical nature of the discourse segment where the antecedent can be found. #75 explains the effect of modals, negations, and conditionals. #76 discusses the importance of discourse structure and discourse coherence particularly.
Chapter 11, “Presupposition”, concentrates on the effect of different phenomena when processing presuppositions. #77 differentiates entailments from presuppositions, both being sentence properties. #78 exemplifies the different types of presupposition triggers, i.e., the words or expressions prefacing presuppositions, which vary among languages. #79 illustrates the complexity of presupposition depending on the linguistic features used to express it. #80 discusses that and exemplifies how presuppositions will be accommodated only when there is no discourse antecedent. #81, similar to #76, shows discourse coherence is also very relevant in understanding presupposition.
Chapter 12, “Information Status and Information Structure”, explains these linguistic phenomena and describes different factors influencing them. #82 introduces information status via the components of an implication hierarchy to demonstrate the linguistic connections between words and their referents to speakers’ shared knowledge. #83 exemplifies the different morphological and syntactic options to demonstrate information status, which vary across languages. #84 introduces information structure, differing from information status, as it shows that utterances can be new or given depending on the context, marked by prosody, and having meaning and marking components. #85 discusses the four complex concepts by which the meaning of information structure can be expressed: topic, focus, background, and contrast. #86 illustrates different morphological (prosody, lexis) and syntactical means to express information structure marking across languages. #87 describes that information structure marking is achieved most reliably by considering both prosodic stress and tune together. #88 argues that both information structure and truth conditions interplay to resolve semantic ambiguity.
Chapter 13, “Implicature and Dialogue”, observes contextual devises being fundamental to resolve ambiguity issues from both semantics and pragmatics perspectives. #89 defines and exemplifies implicatures, as implied meaning conveyed by the speaker that surpasses the linguistic meaning of the utterance. #90 discusses Grice’s (1975) implicatures (conversational and conventional) and the four conversational maxims from which the former implicatures come in order to understand implicatures and language users’ inferring mechanisms in communication. #91 asserts that cognitive reasoning is not always needed to understand implicatures, and introduces the rationale behind coherence-based theories to support that statement. #92 describes that comprehending and creating semantic meaning should not be done following the same logic pathways, meaning less time should be needed to do the latter and more to achieve the former. #93 tackles the “safety” of using implicatures in conversation; considering the aim of the speaker and the interpretation of the hearer, the consequences can be (un)safe. #94 exemplifies how utterances and implicatures can be rejected or agreed upon explicitly or implicitly. #95 exemplifies how silence can also be meaningfully used as an implicature tool, mainly when speakers do not think alike. #96 emphasizes the importance of prosody to influence on speaker’s stance, and describes why this should be considered in NLP systems building.
Chapter 14, “Resources”, encompasses a series of resources to analyze semantic representations from smaller to bigger units. #97 exemplifies a wide range of tools to identify and categorize lexical semantic meaning useful to apply in different NLP tasks. #98 focuses on semantic meaning at the sentence level, providing examples of sembanks, a compilation of texts that include semantic annotations. #99 illustrates a series of sembank-trained and grammar-based parsers that incorporate annotations to represent meaning. #100 describes a series of different corpora that also include annotations of discourse meaning. A summary of the main goals the authors aim to obtain with this volume is included at the end of this chapter.
This book is an excellent, well-documented compilation of integral linguistic, semantic, and contextual matters that affect language use and understanding and their application to NLP. These are the main objectives the authors aim to achieve and that have been discussed thoroughly throughout this volume. A positive trait of this volume is the title of each of the 100 topics, which is a summary of the content explored in each topic. This was a very helpful first step to understanding the topics before diving into each of them. Additionally, the organization of all 100 topics in 14 chapters is coherent, and such organization makes the comprehension of all these issues clear and more schematized. Some of the subjects discussed may be difficult to understand if the reader does not have any previous knowledge of computational linguistics, formal logic, NLP or machine learning. This is the one drawback I would point out from this volume. Nevertheless, the examples and explanations compensate for that shortcoming. Another issue I found is that it refers to semantics and pragmatics but I have felt that more prominence is given to semantics and formal semantics to explain and describe even pragmatic aspects of the language. Coming from an applied pragmatics training, I was expecting a more social or applied pragmatics approach to some of the issues discussed. Nevertheless, exploring familiar linguistic and contextual aspects from another perspective has been informative and enlightening. A very positive aspect I would like to highlight is all the lines for further research that the authors identify all throughout their volume. This makes this piece of work of highly research value.
This book may be useful for a varied audience, ranging from PhD students to more experienced scholars in different subfields of linguistics to engineers working on NLP tasks and NLU research who might find enriching the consideration of the linguistic nature of such processes. In sum, this volume is an important example of interdisciplinary research and work that puts forward the essential role of different linguistic forms, their meaning, and the importance of context for language usage and understanding and the wide range of applications in NLP.
Prior, Arthur. 1957. Time and modality. Oxford University Press.
Reichenbach, Hans. 1947. Elements of symbolic logic. Macmillan.
ABOUT THE REVIEWER
María-José Arrufat-Marqués holds a PhD in Applied Linguistics from Universitat Jaume I (Spain). As a Fulbright scholar, she earned a MA in Applied Second Language Acquisition from Carnegie Mellon University (USA). María-José’s research interests include: second language acquisition, interlanguage pragmatics, pragmatic development, formulaic language, technology-enhanced language learning and teaching, and language attitudes.
Page Updated: 25-Mar-2021