LINGUIST List 23.3587

Mon Aug 27 2012

Review: Computational Linguistics; Semantics: Mani & Pustejovsky (2012)

Editor for this issue: Rajiv Rao <>

Date: 27-Aug-2012
From: Dorothea Hoffmann <>
Subject: Interpreting Motion
E-mail this message to a friend

Discuss this message

Announced at
AUTHORS: Mani, Inderjeet; Pustejovsky, JamesTITLE: Interpreting MotionSUBTITLE: Grounded Representations for Spatial LanguagePUBLISHER: Oxford University PressYEAR: 2012

Dorothea Hoffmann, The University Chicago, USA


The monograph ‘Interpreting Motion’ sets out to describe a new approach incomputational linguistics to understand and map natural language descriptions ofmotion. The authors aim to “offer an integrated perspective on how languagestructures concepts of motion, and how the world shapes the way in which motionis linguistically expressed” (5). Therefore, the novelty of the approach lies inthe attempt to describe an interdisciplinary method of combining research on thesemantics of motion verbs and locative constructions from a cross-linguisticperspective with qualitative spatial reasoning. The aim is to develop a modelfor computational linguistics based on mapping spatial and temporal relations torepresent motion in natural language. Within this approach, topological,orientation and distance relations are analyzed as being expressed in verb,adverbial and prepositional phrases. Ultimately, the approach is meant to leadto mapping text to data representations with practical consequences.

The first chapter introduces the topic, outlines the technical approach andsituates the publication within other works on spatial prepositions and motionverbs. In particular, some challenges of a computational approach to naturallanguage analysis are introduced, such as considering typologicalcharacteristics of languages, making use of diverse corpora, and some issues ofhuman and machine annotation. Additionally, consequences of combining linguisticdescription and theory with computational specifications are addressed. Forexample, Talmy’s (1985, 2000a, 2000b, 2007, 2009) typological distinction ofverb- and satellite-framed languages with the addition of Slobin’s (1996, 2004,2006) equipollently-framed ones results in a formal distinction of location-(path) and action-based (manner) predicates, as well as a combination of thetwo, for the latter type in qualitative reasoning.

Specifically, the authors proclaim essential desiderata for their approach inthis chapter. These include that semantic representations need to be expressiveenough for natural language. The aim is to develop a denotational semantictheory, and a compositional analysis. Additionally, all representations need tosupport qualitative reasoning and all systems need to be accurate and efficientenough to support practical applications.

Concerning the theoretical background of linguistic descriptions of motion, theauthors provide a comprehensive summary of previous studies on spatialprepositions and motion verbs. These, for example, classify spatial prepositionsas locative or directional (Miller and Johnson-Laird, 1976). Within cognitivelinguistics, they are termed by such concepts as ‘contact’ and ‘inclusion’(Evans et al., 2007), while Jackendoff (1983) defines them within his theory ofLexical Conceptual Structure (LCS). Finally, spatial prepositions have beenanalyzed as vector-representations mapping sets of points in accordance withsemantic content (Zwarts and Winter, 2000, Zwarts, 2003). Mani and Pustejovskycome to the conclusion that this research is of no use for their computationalapproach since it lacks corpus-based evidence. Regarding motion verbs, variousapproaches are discussed, including Langacker’s (1987) topological view ofmotion verbs, Jackendoff’s LCS, Word Net's (Fellbaum, 1998) ranking senses ofwords based on corpora, Verb Net's (Kipper et al., 2006) ability to providesyntactic and semantic information about verbs, Frame Net's (Bake et al., 2003)reliance on the theory of Frame Semantics (Fillmore, 1976), various approachesto verb classification based on qualitative reasoning, and compositionalsemantics. Overall, the authors conclude that none of these approaches isconsistent with the desiderata set out to govern their methodology.

Finally, some limitations of the study are briefly addressed. Pragmatic andpsycholinguistic considerations are not part of the book’s content since it isprimarily concerned with semantic theory. Finally, the authors do not claim toprovide a thorough survey of the field, rather choosing to only introduce selectrelevant literature.

The second chapter discusses how motion is expressed in natural language bydeveloping a framework for analyzing different parameters of spatial meaning.First, static spatial relations are briefly discussed, including topological(e.g. “on”, “in”), orientational (e.g. “over”, “under”), topometric (e.g.“near”, “far”), and topo-orientational relations (e.g. “on the wall”, “hang overthe desk”). Particular emphasis is laid on “the domain of points, lines,regions, and the relations between them” (31). Following this, a discussion ofmotion includes argument structure and role selection, event structure, path andmanner of motion verbs, paths and orientation, and measuring distance. Based onthe ‘Region Connection Calculus 8’ (RCC 8) (Randell et al., 1982) the model isenriched to deal with orientation and distance as well as motion. The RCC 8 is acalculus of relations of eight jointly exhaustive and pairwise disjointrelations used to analyze static spatial descriptions involving prepositions innatural language.

Path verbs are discussed in some detail and an additional path element isassumed in lexical argument structure. Motion is then represented in terms oftransitions in spatial configurations, along with particular temporalconstraints. In contrast, manner verbs are analyzed as indicating motion alone,without a path element within the verb. In line with Talmy’s typology, mannerverbs require a path adjunct and path verbs may add a manner adjunct. Thefoundations for a semantics of motion laid out in the chapter then form thebasis for a logic of motion, ‘Dynamic Interval Temporal Logic’ (DITL), allowingthe authors to model events and states as programs.

In Chapter 3, spatial and temporal representations and inference methods areexamined with regards to qualitative reasoning and are applied to spatialphenomena in languages involving topological and orientation relations.Particular emphasis is laid on qualitative representations of ‘Topology’ and‘Frames of Reference’ (FoR) (Levinson, 2003, Levinson and Wilkins, 2006a,Pederson et al., 1998). Topological relations are analyzed with the alreadymentioned RCC-8 relations mapped onto interval calculus (Allan, 1984) relationsto combine spatial and temporal relations with one another. Furthermore, forintrinsic FoR, the ‘Oriented Point Relational Algebra’ (OPRA) (Moratz et al.,2005) is used to describe the relation between oriented points when size andshape of the ground is of no importance. Absolute FoR is analyzed using the‘Cardinal Direction Calculus’ (CDC) (Goyal and Egenhofer, 2000, Skiadopoulousand Koubarakis, 2005) for relations between objects when each is positioned interms of a coordinate system. Finally, relative FoR makes use of the ‘DoubleCross Calculus’ (DCC) (Freksa, 1992), describing the position of the Figurerelative to the Ground as seen by an observer. In conclusion, the authors statethat for a sufficient qualitative analysis of topological and FoR relations, theindividual calculi mentioned are not enough. A combination of these remains achallenge for future work.

Chapter 4 applies the methods discussed in Chapters 2 and 3 to the concept ofmotion. Generally, motion expressions are analyzed within a cognitively inspiredspatiotemporal model of change. In this approach, Talmy’s distinction can bemodeled based on ‘Dynamic Interval Temporal Logic’ (DITL) (Pustejovsky andMoszkowicz, 2011), in which prepositional, verb and noun meanings are integratedtogether compositionally. This “combines mechanisms from temporal logic with theability to update state information from dynamic logic” (90). Generally, theauthors aim to fulfill two criteria, namely, discussing how each component of abasic motion frame is semantically grounded, as well as how theserepresentations map to a compositional interpretation of the motion expressionin a language. Some path and manner of motion verbs are discussed in detail andprovided with a DITL definition.

Chapter 5 introduces a practical methodology for humans in annotating linguistictext corpora with information on motion to allow for automatic text to sketchmapping by a computer. To achieve reliable human annotation, best practices forspatial annotation (including toponyms) are examined alongside motionannotation. While some of these issues are well-discussed problems, theextraction for topological and orientation relations has not been as developed.Following ‘ISO Space’ (Pustejovsky et al., 2010) which has rich representationsof paths and distinguishes between manner and path verbs, in addition tosubclasses of motion events, the authors conclude that in general, capabilitiesof assembling automatic motion tracking from natural language narratives arewell underway.

Finally, Chapter 6 summarizes the authors’ approach of using representationsbased on qualitative reasoning to describe the meaning of motion verbs andspatial expressions. Additionally, the chapter discusses potential advantages aswell as practical application options. These can be found in route navigation,mapping travel narratives and multimedia tagging of static images, audio, video,question-answering, communicating with artificial agents, and rendering scenesfrom text. Additionally, some open issues are mentioned. These include, forexample, the problem of ‘Fictive Motion’, adequately capturing cross-linguisticvariation, integrating 3D representations of spatial entities, the challenges ofdeveloping methods to quickly generate training data, and data preparation methods.


The main aim of the authors is to provide an interdisciplinary approach tocombining linguistic analysis of motion event descriptions with qualitativespatial reasoning to develop a computational model for the mapping of motionevents in natural language. The monograph succeeds in carefully laying outessential issues in both fields of interest and in providing an excellentoverview of current developments and trends. In an easy to follow, step-by-stepdiscussion, the authors sensibly add components to examine this highly complexissue. While open questions and problems remain at the end of the book,‘Interpreting Motion’ provides an excellent discussion of the problems ofmapping motion from natural language aimed primarily at a computational linguistaudience. However, because of the well-structured summaries and explanations ofthe many calculi and intervals used in the model, the book might also be usefulto a non-expert audience of linguists interested in text-to-data mapping ofstatic spatial and dynamic language.

Chapter 2 is particularly compelling in presenting an excellent overview ofissues in spatial and motion linguistics and pointing out direct links toexisting computational models of relevance to the aims of the monograph. Thiskind of discussion gives the reader the opportunity to directly reflect on andevaluate the current state of the art with regards to both disciplines – thesemantics of spatial language, and computational models for their description.Furthermore, the authors succeed in guiding the reader through the discussionwithout overly simplifying the complex features of either field. Especiallyuseful in this chapter are numerous exemplary analyses of path as well as mannerof motion verbs, which provide the theoretical underpinnings with helpfulillustrations.

Additionally, Chapter 5 is also particularly convincing in postulating highlyvaluable practical guidelines for developing and expanding procedures insemantic annotation. This discussion is able to provide useful parameters forhuman as well as automated linguistic annotation that go beyond the topic of themonograph. As a result, the authors accomplish improving best practices ofannotation as well as expanding the outreach of their study.

The main criticism of the monograph comes from a certain inconsistency inincluding issues of cross-linguistic variation in the semantics of spatial andmotion event descriptions without adequately discussing these in relation to acomputational approach. While the authors show some detailed knowledge of thetypological literature on the topic, this is not further exploited in the laterstages of the book. Additionally, some terminology, such as ‘orientation’ for‘FoR’ is not chosen well, especially when considering that a number of recenttypological publications on languages distinguish between ‘orientation’ (e.g.“he is facing the tree”) and ‘FoR’ (e.g. “he is in front of the tree”)(Bohnemeyer and O'Meara, 2012, Terrill and Burenhult, 2008).

Additionally, Mani and Pustejovsky appear to leave out some importantdistinctions of motion event literature. For example, ‘change of location’ isequalized with ‘motion’, however, Levinson and Wilkins (2006b) describe cleardistinctions in the verbal semantics between translocational movement (e.g. “hewent from the garden to the house”), change of location (e.g. “he left thegarden and arrived at the house”) and change of locative relation (e.g. “heended up in the house”). Furthermore, while issues concerning the lexicalizationof ‘source’ and ‘goal’ are discussed, ‘passed grounds’ (i.e. ‘via’) are left outof the examination.

All in all, the monograph is a good source for computational linguists andothers interested in text-to-sketch mappings of spatial and motion events andprovides some well-grounded discussions of relevant and detailed problems andsolutions.


Allan, J. 1984. Towards a general theory of action and time. ArtificialIntelligence 23:123-154.

Bake, C.F., Fillmore, Charles, and Cronin, B. 2003. The structure of theFrameNet database. International Journal of Lexicography 16:281-296.

Bohnemeyer, Juergen, and O'Meara, Carolyn. in press. Vectors and frames ofreference: Evidence from Seri and Yucatec. In Space and Time across Languagesand Cultures, eds. Luna Filipovic and Kasia M. Jaszczolt: John Benjamins Ltd. .

Evans, V., Bergen, B.K., and Zinken, J. 2007. The cognitive linguisticsenterprise: An overview. In The Cognitive Linguistics Reader, eds. V. Evans,B.K. Bergen and J. Zinken, 3-26. London: Equinox Publishers.

Fellbaum, C. ed. 1998. WordNet: An Electronic Lexical Database. Cambridge, MA:MIT Press.

Fillmore, Charles. 1976. Frame Semantics and the nature of language. Paperpresented at Conference on the Origin and Development of Language and Speech,New York

Freksa, C. 1992. Using orientation information for qualitative spatialreasoning. In Theories and methods of spatiotemporal reasoning in geographicspace, eds. A.U. Frank, I. Campari and U. Formentini, 162-178. Berlin: Springer.

Goyal, R., and Egenhofer, M.J. 2000. Consistent queries over cardinal directionsacross different levels of detail. Paper presented at 11th InternationalWorkshop on Database and Expert Systems Applications, Greenwich, UK.

Jackendoff, R. 1983. Semantics and Cognition: Current studies in Linguistics.Cambridge, MA: MIT Press.

Kipper, K., A., Korhonen, Ryant, N., and Palmer, M. 2006. Extending VerbNet withNovel Verb Classes. Paper presented at Fifth International Conference onLanguage Resources and Evaluation, Genoa.

Langacker, Ronald W. 1987. Foundations of Cognitive Grammar. Stanford: StanfordUniversity Press.Levinson, Stephen C. 2003. Space in Language and Cognition. Explorations inCognitive Diversity. Cambridge: Cambridge University Press.

Levinson, Stephen C., and Wilkins, David. 2006a. Grammars of space: explorationsin cognitive diversity: Language, culture, and cognition ; 6. Cambridge, UK ;New York: Cambridge University Press.

Levinson, Stephen C., and Wilkins, David. 2006b. Patterns in the Data. Towards asemantic typology of spatial description. In Grammars of Space. Explorations incognitive diversity, eds. Stephen C. Levinson and David Wilkins, 512-552.Cambridge: Cambridge University Press.

Miller, G.A., and Johnson-Laird, M. 1976. Language and Perception. Cambridge,MA: Belknap Press of Harvard University Press.

Moratz, R., Dylla, F., and Frommberger, L. 2005. A relative orientation algebrawith adjustable granularity. Paper presented at Workshop on Agenst in Real-Timeand Dynamic Environments (IJCAI), Edinburgh, Scotland.

Pederson, Eric, Danziger, Eve, Wilkins, David, Levinson, Stephen C., Sotaro,Kita, and Senft, Gunter. 1998. Semantic Typology and Spatial Conceptualisation.Language 74:557-589.

Pustejovsky, J., Moszkowicz, J.L., and Verhagen, M. 2010. ISO-SpaceSpecification: Version 1.3.

Pustejovsky, J., and Moszkowicz, J.L. 2011. The qualitative spatial dynamics ofmotion in language. Journal of Spatial Cognition and Computation 11:15-44.

Randell, D.A., Cui, Z., and Cohn, A.G. 1982. A spatial logic based on regionsand connection. Paper presented at 3rd International Conference on KnowledgeRepresentation and Reasoning, San Mateo, CA.

Skiadopoulous, S., and Koubarakis, M. 2005. On the consistency of cardinaldirection constraints. Artificial Intelligence 163:91-135.

Slobin, Dan I. 1996. Two ways to travel: Verbs of motion in English and Spanish.. In Grammatical Constructions. Their Form and Meaning eds. M Shibatani and S.A. Thompson, 195-219 Oxford: Clarendon Press.

Slobin, Dan I. 2004. The many ways to search for a frog: Linguistic typology andthe expression of motion events. In Relating Events in Narrative: Vol. 2Typological and contextual perspectives, eds. S. Stroemqvist and L. Verhoeven,219-257. Mahwah, NJ: Lawrence Erlbaum Associates.

Slobin, Dan I. 2006. What makes manner of motion salient? Explorations inlinguistic typology, discourse and cognition. In Space in Languages. LinguisticSystems and Cognitive Categories, eds. Maya Hickmann and Stephane Robert, 59-81.Amsterdam, Philadelphia: John Benjamins.

Talmy, Leonard. 1985. Lexicalization patterns: semantic structure in lexicalforms. In Language Typology and Syntactic Description: Grammatical Categoriesand the Lexicon, ed. Shopen, 57-149. Cambridge: Cambridge University Press.

Talmy, Leonard. 2000a. Toward a Cognitive Semantics: Concept Structuring Systemsvol. 1. Cambridge, MA: MIT Press.

Talmy, Leonard. 2000b. Toward a cognitive semantics. Typology and Process inConcept Structuring vol. 2. Cambridge, MA: MIT Press.

Talmy, Leonard. 2007. Lexical Typologies In Language Typology and SyntacticDescription, ed. Timothy Shopen, 66-168. New York: Cambridge University Press.

Talmy, Leonard. 2009. Main Verb Properties and Equipollent Framing. InCrosslinguistic Approaches to the Psychology of Language. Research in theTradition of Dan Isaac Slobin, eds. Jiansheng Guo et al., 389-402. New York:Psychology Press.

Terrill, Angela, and Burenhult, Niclas. 2008. Orientation as a strategy ofspatial reference. Studies in Language 32:93-136.

Zwarts, J., and Winter, Y. 2000. Vector Space Semantics: A model-theoreticanalysis of locative prepositions. Journal of Logic, Language and Information9:171-213.

Zwarts, J. 2003. Vectors across spatial domains: from place to size,orientation, shape and parts. In Representing Direction in Language and Space,eds. Emile Van der Zee and Jon Slack, 39-68. Oxford: Oxford University Press.


Dorothea Hoffmann received her PhD from the University of Manchester, Great Britain in 2011. She is now a postdoctoral fellow at the University of Chicago on a language documentation project of MalakMalak, an endangered language of the Daly River Area in Australia, funded by the Endangered Language Documentation Programme. Her research interests include typology, lexical semantics, language contact, narrative structure, cognitive linguistics, Australian indigenous languages and culture, as well as discourse-based studies of space and motion.

Page Updated: 27-Aug-2012