Publishing Partner: Cambridge University Press CUP Extra Publisher Login

New from Cambridge University Press!

ad

Revitalizing Endangered Languages

Edited by Justyna Olko & Julia Sallabank

Revitalizing Endangered Languages "This guidebook provides ideas and strategies, as well as some background, to help with the effective revitalization of endangered languages. It covers a broad scope of themes including effective planning, benefits, wellbeing, economic aspects, attitudes and ideologies."


New from Wiley!

ad

We Have a New Site!

With the help of your donations we have been making good progress on designing and launching our new website! Check it out at https://linguistlist.org/!
***We are still in our beta stages for the new site--if you have any feedback, be sure to let us know at webdevlinguistlist.org***

Review of  Natural Language Processing and Cognitive Science


Reviewer: Laurel A. Schenkoske
Book Title: Natural Language Processing and Cognitive Science
Book Author: Bernadette Sharp Rodolfo Delmonte
Publisher: De Gruyter Mouton
Linguistic Field(s): Computational Linguistics
Psycholinguistics
Cognitive Science
Issue Number: 27.1284

Discuss this Review
Help on Posting
Review:
Reviews Editor: Sara Couture

SUMMARY

'Natural Processing and Cognitive Science' is a conference proceedings of the 2014, 11th annual NLPCS Workshop, the aim of which ''was to foster interactions among researchers and practitioners in ... (NLP) by taking a Cognitive Science perspective'' (preface). The edited volume contains 25 independent peer-reviewed articles, many written by graduate students, and very broadly connected by the conference theme. The papers cover a very broad array within the scope of Natural Language Processing and Cognitive Science. Most of the articles are early work and pilot studies, as is typical for proceedings, and present exciting new venues for research in their respective fields. Research methodologies include experimental and computational studies, with people and with machines. The keynote speech was delivered by Jared Bernstein of Stanford University, and is the basis of the first article of the volume. In this review, each article is discussed separately, in the order it appears within the volume.

''Benchmarking Automated Text Correction Services'' served as the keynote address (Bernestein, et. al.). They compared two automated error detection services over 24 short middle school essays. Both engines had reported accuracy rates above 90%, yet Bernstein, et. al., found both to have high error rates: one had better precision (found only real errors, but also missed several), while the other had better recall (caught more real errors, but counted many correct items as errors). Human teachers were found to have high variability, both within each individual teacher and across various teachers. The authors of the study suggest that a system consisting of a machine with high recall plus a human checker would make the best fit. Nothing was said about meaning, style, coherence, and flow, except that these were intentionally ignored in the study.

''Enhancing Word Sense Disambiguation Using a Hybrid Knowledge-Based Technique'' discusses the issue of lexical semantic ambiguities – often problematic for machine-based computations. The authors combine an adapted version of Lesk's Algorithm (1986) and Jiang & Coranth's Similarity measure (1997), and determine that their hybrid measure outperforms both methods when tested alone in the disambiguation of word senses.

''Disambiguating Distributional Neighbors Using a Lexical Substitution Dataset'' proposes a new method of clustering polysemous targets. They use a lexical substitution dataset to attribute one correct sense to each target word, based on the number of similar substitutes. Unlike more traditional approaches (e.g. wordnets), this method is able to assign value to neighbors related to both senses, as well as to neither sense.

''An Evolutionary Game Theoretic Approach to Word Sense Disambiguation'' uses a semi-supervised model and Game Theory to represent words (nodes) as players who choose a strategy – a semantic class membership. The approach uses data (information on semantic class) from labeled nodes (words) and extends that information to unlabeled nodes / words. This works because the players are always neighbors, and often share a class. Their approach is different from others in that they do not need to train the system on large corpora, but use instead small sets of labeled data.

''The Main Challenge of Semi-Automatic Term Extraction Methods'' demonstrates that the main challenge is the difficulty of determining rank and threshold of terms. (The threshold may be either a percentage or a number of candidate words.) After comparing 21 extraction methods of statistical, linguistic, and hybrid knowledge, using three different corpora, the authors conclude that best precision, best recall, and best F-measure are dependent on the type of knowledge.

''News Text Segmentation in Human Perception'' compares the results of human psycholinguistic tasks against computational experiments, in determining text predictability in the homogenous genre of news texts. Using a corpus of Russian news sources, they look at various language levels, from words to syntagmas to propositions to discourse. They found that the keywords extracted by humans and those by the program were very different. Unlike the computational system, human informants identified words and syntagmas relevant to the context. The authors also found that, overall, more keywords in a segment correlated with higher syntagmatic weight (i.e., frequent in the document, but rare in the corpus).

''Preliminary Study of TV Caption Presentation Method for Aphasia Sufferers'' describes how a small set of aphasia sufferers preferred a summarized version of closed captioning over the original version. The authors have developed a unified interface in which captions from the video provider are 1) automatically summarized, and 2) passed on to volunteers for manual summarization.

''Extraction of Concrete Entities and Part-Whole Relations'' proposes a method for meronymy (part-whole) extraction of concrete objects from a corpus. Precision rates were somewhat low, but the authors suggest this could be improved with better filtering of non-relevant words.

In ''Human Association Network and Text Collection,'' the authors base their model on the fact that semantic information may be present in human communication, even when lexical information is not. For example, “terrier” and “animal” are semantically related in memory. They develop an algorithm for text-driven extraction of direct and indirect associations.

''Using Function Words for Authorship Attribution'' displays a comparison between two methods of attributing authorship of a text: a style marker, which relies on sequential rules, versus function-word frequency, which relies on ''bag-of-words assumptions.'' It seems they set out to find better results in the rule-based method, but find instead that the bag-of-words assumptions have much higher accuracy, as previous research had also found.

In ''Recognition of Discursive Verbal Politeness,'' the authors pilot a program to automatically annotate text to identify ''Politeness Verbal Phrases,'' including both positive and negative politeness. Results are high, close to matching the manual annotation.

''Politeness versus Perceived Engagement'' looks at perceived engagement between speaker and hearer to determine how differently weighted Face-Threatening Acts are implemented. Such acts, according to Politeness Theory, include expressions of disapproval and criticism, as well as orders and requests (Brown & Levinson, 2004). Despite politeness being subjective, with individual differences, they found that when a speaker wants to continue an interaction, he or she needs to use more politeness strategies for a less engaged hearer. The authors plan to apply these findings to research in human-machine interaction.

''Sentiment, Polarity and Function Analysis in Bibliometrics'' offers a review of works on both citation analysis (bibliometrics) and sentiment analysis (SA), and propose a marriage of the two, claiming that simple, quantitative citation counts do not offer enough insight into well-cited authors or works, and sometimes even offering false insight. Instead, the authors claim, a qualitative measure of sentiment for each cited work should be included.

In ''The Detection and Analysis of Bi-Polar Phrases and Polarity Conflicts,'' the authors conduct a sentiment analysis of Noun Phrases (NPs) (e.g., “just punishment”) and Verb Phrases (VPs) (e.g., “He admires his sick friend”) for polarity expectations and violations. They insist more fine-grained distinctions are needed than simply ''positive'' or ''negative,'' and therefore use a modified Appraisal Theory (Martin & White, 2005). They first create a multilingual lexicon of adjectives and nouns that are either positive or negative, and further classified as either ''Appreciation,'' ''Affect,'' or Judgment.'' They extend the study to bi-polar VPs (where verbs carry an unexpected polarity for their arguments), and finally combine the two analyses.

''Automatically Evaluating Atypical Language in Narratives by Children with Autistic Spectrum Disorder'' looks a set of narratives from children with ASD. The large dataset allowed the authors to distinguish between general language deficiencies and those specific to ASD. They were also able to look at much more distinctive features than just text length. There were some specific findings in pronoun use and topic coherence, and a more profound finding in sentiment and references to mental state, with the ASD participants using fewer than their matched controls.

In ''How to Make Right Decisions Based on Corrupt Information and Poor Counselors,'' it is unclear how the title relates to the article. The authors review open-source Question Answering systems, and their inherent problems, such as time dependency and ambiguity. They provide suggestions for updating the OpenEphyra (Schlaefer, N., et. al., 2011) open-source QA system, specifically in its interface with various internet search engines.

''Meta-Learning for Fast Dialog System Habituation to New Learners'' introduces an example of a human assistant learning the behaviors regarding phone calling and contact information of her boss, and extends that example to the ''learning,'' or habituation, of a software app. This pilot study uses a statistical dialog manager to learn its users intent for phone calling behavior. Crucially, parameter settings are first learned from a sample population, and are then further refined for an individual user.

''Evaluation of Freely Available Speech Synthesis Voices for Halef'' may be a difficult article to follow for someone unfamiliar with speech synthesis research, as some key acronyms are not explained. The authors analyzed several freely available voices from two competing text-to-speech (TTS) systems - Festival and Mary - and found the best voices to belong to the Mary system. They conclude this is the best system for Halef (Help Assistant Language-Enabled and Free).

In ''You Shall Find the Target Via its Companion Words,'' the authors developed a Human-Computer-Interaction system for overcoming Tip-of-the-Tongue (TOT) states (a search problem in speech production). Since speakers experiencing a TOT access problem typically have some concept of the word – meaning especially – the authors were able to design a system using those known associations to help the speaker access the ''misplaced'' word. They performed a comparative analysis of WordNet, Extended WordNet, the Edinburg Association Thesaurus, unstructured language corpus, and Rogert's Thesaurus, in which each yielded both strengths and weaknesses; a combination of systems is proposed. The system itself consists of a two-step process. First, the user inputs a semantically related word, and the entire lexicon (some 60,000 words) is dramatically reduced. Second, the user is presented with a categorical tree, made up of nodes, or word sets, with possible target words. From here, the user may either chose his or her target word, or search a particular word set further.

''Topic Modeling for Entity Linking Using Keyphrase'' is an update and improvement of an Entity Linking system that the authors completed in 2012 and 2013. Within a Knowledge Base (here, Wikipedia), the system links entries to references via a multi-step process between Query and Answer. The last step, candidate ranking and clustering of entries not in the knowledge base, is their main challenge and focus.

In ''Extraction of Polish Multiword Expressions For the Automatic Extraction of Polish Multiword Expressions'' (MWE), the authors experiment with Dictionary Pattern Extraction (DM) and Syntactic Pattern Extraction (SM), and several combined methods. In their most successful pattern – ''DM for SM results'' – the output is a list of words with relevant declension information, with both good precision and good recall.

''Beyond Classical Set'' offers an extension of classical Set Theory for Natural Language Semantics. The authors put forth a theory of typed sets, which they call a “natural extension to classical sets” (p. 270). The theory considers “urelements” (simple and unanalyzable elements of sets) as binary, each with a core and a type. Their Typed Sets accommodates the concepts of predication and aggregation, and allows for finer grained analysis of meaning and natural language processing.

''Exploring the Effects of Root Expansion, Sentence Splitting, and Ontology in Arabic Answer Selection'' addresses the issue of imperfection in Arabic Question Answering Systems (as well as all Question Answering Systems). The authors compare their system (ALQASIM 2.0) to two other Arabic Question Answering for Machine Reading and Evaluation systems (QA4MRE). These include their own previous ALQAISM 1.0 (Ezzeldin, et. al., 2013) and IDRAAQ (Abouenour, et. al., 2012). The 2.0 system proposed here consists of three main modules: Document Analysis, Question Analysis, and Answer Selection. Notably different from the other systems is that it focuses on answer selection and validation. Through their approach to sentence splitting and root expansion, they are able to achieve much higher accuracy, as well as give partial credit for unanswered questions that the system could not answer.

In ''Computer Assisted Translation in Ancient Texts,'' the authors developed a Computer-Assisted Translation (CAT) system specifically for the use with the ancient Babylonian Talmud, to be translated into Italian. The authors describe the rich linguistic variety in the text, but also explain that it contains many quotations and lexically repetitive, formulaic language, leading them to base the translation system on Translation Memory (TM). Aside from improved translation speed, their main objective is to form a collaborative environment for translators and scholars familiar with the Babylonian Talmud.

''A Rational Statistical Parser'' demonstrates the development of a computational syntactic parser to work as rationally as a human one – finding the ''optimal path to the solution'' (p. 303). The three objectives for their model to be rational is that it choose the most probable analyses according to experience, that it be quick, and that it minimize cognitive costs. Their model assigns probability and entropy scores to syntactic trees in order to determine their weight and to find the best path – a successful method, according to their experimental results.

EVALUATION

The volume is an excellent addition to the fields of Natural Language Processing and Cognitive Science. NLPCS appears to be a student-friendly conference and proceedings, helping to open pathways for new and exciting research into the fields. The articles span a great diversity, and serve as springboards to potentially very significant work in their respective sub-fields. The genres also cover a literature review, psycholinguistic and computational studies, and human-to-human and human-to-computer interactions.

While many articles are only loosely connected, some cover intimately related themes (e.g., Word Sense Disambiguation, Sentiment Analysis). The 25 articles are not organized into sections or arranged by topic. For the most part, they are arranged in a logical manner, e.g., those articles discussing text linking programs are grouped together, as are some on lexical semantics. But the article on Tip-of-the-Tongue states, and another handling formal semantics, do not appear until much later. Furthermore, there are two separate articles that discuss Question Answering Systems, which appear at very different points in the collection. While there can be no clear linear path linking one article to the next, a better attempt could have been made to keep like articles together.

Aside from the organization among the papers, improvements upon some minor weaknesses would enhance the articles and the book itself. The volume would have benefitted from some major copy editing. Most of the authors are non-native English speakers, and spelling and word order errors – even in the chapter headings – is a point of confusion and slower processing on the part of the reader. Another possible point of concern is that some tables are colored, while most are black and white. Did all the authors know they had the option to submit colored tables and charts? Finally the 25 articles represent vastly different formatting techniques for in-text citations, further reducing the coherency of the volume.

The articles themselves are well formulated. Each begins with a general introduction, including a plan, goal, or rationale of the research, and then the organization for the rest of the article. They are well thought out and include solid research methods, described in the text and clearly represented in charts and tables.

Because of the broad spectrum of topics, it is doubtful that many people will read this volume cover-to-cover. But many of the articles contained within will certainly be of interest to students and researchers of NLP, cognitive sciences, linguistics, psychology, and computer programming. Most of the papers propose cutting-edge research in their fields, and propose work to carry the research even further.

REFERENCES

Abouenour, L., Bouzoubaa, K., & Rosso, P. (2012). IDRAAQ: New Arabic Question Answering System based on query expansion and passage retrieval. In CLEF 2012 Workshop on Question Answering for Machine Reading Evaluation (QA4MRE). Rome, Italy.

Bernstein, J. (2014, October). Benchmarking Automated Text Correction Services. Keynote Address presented at the 11th Annual NLPCS Workshop,Venice, Italy.

Brown, P. & Levinson, S.C. (1978). Politeness: Some universals in language usage. Cambridge UP: Cambridge, UK.

Ezzeldin, A. M., Kholief, M. H., & El-Sonbaty, Y. (2013). ALQASIM: Arabic language question answer selection in machines. In Information Access Evaluation. Mutilinguality, Multimodality, and Visualization (pp. 100-103). Springer: Berlin, Germany.

Jaing, J. J. & Coranth, D. W. (1997). Semantic similarity based on corpus statistics and lexical taxonomy. In C. K. Huang & R. Sproat (Eds.), Proceedings of the 10th International Conference on Research in Computational Linguistics (pp. 19-33). Taipei, Taiwan.

Lesk, M. (1986). Automatic sense disambiguation using machine readable dictionaries: how to tell a pine cone from an ice cream cone. In V. DeBuys (Ed.), SIGDOC ‘86: Proceedings of the 5th annual international conference on systems documentation (pp. 24-26). New York, NY: ACM.

Martin, J. R. & White, P. R. (2005). The language of evaluation: Appraisal in English. London: Palgrave MacMillan.

Schlaefer, N., Nyberg, E., J.C.J.C. & Chu-Carroll, J. (2011). Statistical source expansion for question answering. (Doctoral dissertation). Technologies Institute, School of Computer Science, Carnegie Mellon University.
 
ABOUT THE REVIEWER:
Laurel Schenkoske received her MA in German Studies and Linguistics from the University of Wisconsin- Milwaukee, where she continued further study in the field of formal Linguistics. She is currently a student in the SLAT (Second Language Acquisition and Teaching) PhD program at the University of Arizona, specializing in Processes, with a particular interest in second-language sentence processing. She has spent many years in the classroom, teaching a variety of Linguistics, German, and Composition courses.