LINGUIST List 22.4 Tue Jan 04 2011 Diss: Comp Ling/Psycholing: McCrae: 'A Computational Model for the ...' Editor for this issue: Mfon Udoinyang <mfonlinguistlist.org> To post to LINGUIST, use our convenient web form at http://linguistlist.org/LL/posttolinguist.cfm. Directory         1.     Patrick McCrae , A Computational Model for the Influence of Cross-Modal Context upon Syntactic Parsing Message 1: A Computational Model for the Influence of Cross-Modal Context upon Syntactic Parsing Date: 29-Dec-2010 From: Patrick McCrae <patrick.mccraeinformatik.uni-hamburg.de> Subject: A Computational Model for the Influence of Cross-Modal Context upon Syntactic Parsing E-mail this message to a friend Institution: Hamburg University Program: Department of Informatics Dissertation Status: Completed Degree Date: 2010 Author: Patrick McCrae Dissertation Title: A Computational Model for the Influence of Cross-Modal Context upon Syntactic Parsing Dissertation URL: http://www.sub.uni-hamburg.de/opus/volltexte/2010/4800/index.html Linguistic Field(s): Computational Linguistics Psycholinguistics Dissertation Director(s): Wolfgang Menzel Maosong Sun Christopher Habel Dissertation Abstract: The focus of this thesis is to motivate, specify and validate a computational model for the cross-modal influence of visual scene context upon natural language understanding and the process of syntactic parsing, in particular. We argue for a computational model that establishes cross-modal referential links between words in the linguistic input and entities in a visual scene context. Cross-modal referential links are assigned on the basis of conceptual compatibility between the concepts activated in the linguistic modality and the concepts instantiated in visual context. The proposed model utilises the thematic relations in the visual scene context to modulate attachments in the linguistic analysis. We base our model architecture on the cognitive framework of Conceptual Semantics by Ray Jackendoff. In our model, we adopt the central tenet of Conceptual Semantics that all cross-modal interactions of non-linguistic modalities with language are mediated by Conceptual Structure, a single, uniform representation of linguistic and non-linguistic semantics. Conceptual Structure propagates the influence of the non-linguistic modalities into syntactic representation via a syntax-semantics interface. The purpose of this interface is to map between the syntactic and the semantic representation by means of representational correspondence rules. Our model implements central aspects of the cognitive architecture in Conceptual Semantics. We encode the semantic information for all entities, be they linguistic or non-linguistic in nature, on a single level of semantic representation. In particular, the semantic part of linguistic analysis and visual scene information are included in this representation. The semantic preferences arising from visual context constrain the semantic part of linguistic analysis. The semantic part of linguistic analysis, in turn, constrains syntactic analysis via the syntax-semantics interface. In this way, our model achieves a semantically mediated propagation of non-linguistic visual scene information into syntactic representation. In Part III of the thesis we validate our model's context integration behaviour under a range of experimental conditions. The integration of visual scene context as a hard constraint on linguistic analysis enforces an absolute dominance of visual context information over linguistic analysis. As a result, hard integration may afford a contextualised linguistic analysis that violates linguistic well-formedness preferences in order to be semantically compatible with the modelled visual context. Integrating visual context information as a soft constraint on linguistic analysis permits a balance between conflicting linguistic and contextual references based on the strength of the individual preferences. Under soft integration, our model also diagnoses which aspects of linguistic analysis are in conflict with visual context information. Diagnosis constitutes an important cognitive capability of natural systems in the situated cognition. We further demonstrate our model's robustness to conceptual underspecification in the contextual representation. Our experiments show that the integration of conceptually underspecified context representations provides sufficient information to support the process of syntactic disambiguation. The capability of processing conceptually underspecified semantic information is a relevant capability of natural systems when dealing with perceptual uncertainty and perceptual ambiguity. Read more issues|LINGUIST home page|Top of issue Page Updated: 04-Jan-2011       About LINGUIST    |   Contact Us       While the LINGUIST List makes every effort to ensure the linguistic relevance of sites listed on its pages, it cannot vouch for their contents.