Book announced at https://linguistlist.org/issues/33.3317

AUTHOR: Heli Tissari
TITLE: Frequency in Language
SUBTITLE: Memory, Attention and Learning
PUBLISHER: Cambridge University Press
YEAR: 2022

REVIEWER: Heli Tissari

INTRODUCTION

To understand the book “Frequency in Language: Memory, Attention and Learning”, one could begin by reading its last chapter. The aim of the book is not simply to collect and report information about frequency in language. Rather, the book is a manifesto. It is a call to shift the focus of cognitive linguistics. Divjak writes (pp. 260-261):

“The general cognitive abilities that are typically mentioned in introductions to cognitive linguistics and that have given rise to the bulk of work done within this framework relate to categorization (abstraction, analogy) and imagination (imagery, metaphor, metonymy, blends). But there is also memory, attention and learning, and these have far less often been the focus of (cognitive) linguists’ attention. This book aimed to close that gap[.]”

SUMMARY

The topic of Divjak’s book has elsewhere been called ‘probabilistic processing’, explained as follows (Littlemore 2009: 36):

“[P]robabilistic processing refers to the remarkable sensitivity that [language] learners have regarding the relative frequency with which certain forms are used in particular contexts in the input they receive, and their ability to match their output according to what they think might be appropriate. In other words, probabilistic processing can be seen as a kind of ‘intuitive statistics’.”

Divjak sets a bold aim already in the Introduction, suggesting that “[t]he time has come to shift gears to theoretical questions that go beyond demonstrating frequency effects” (p. 3), that is, beyond showing that probabilistic processing occurs. She explains further that “it is not well known in linguistic circles how frequency achieves these effects” (p. 3). The aim of the book is thus to start explaining what happens in the mind/brain when we learn and use a language.

The book is divided into four parts. The first part deals with frequency; the second part deals with memory; the third with attention; and the fourth with predicting and learning.

The first chapter deals with the history of studies of word frequencies, starting from as early as sixteenth-century word lists. It then tells how psychologists started to discover word frequency effects. For example, it mentions the lexical decision paradigm where, “subjects decide whether a string of letters forms a word or not” (p. 24) and tend to recognize high-frequency words faster than low-frequency words. Divjak laments the fact that much of the research into word frequencies has focused on single words and that people have only recently started to also consider the context in which the words occur. She then continues to discuss “word frequency distributions and the beginning of quantitative linguistics” (pp. 29-38). She mentions many names but pays particular attention to Zipf’s work, explaining Zipf’s first and second law and his principle of economy and least effort. We thus learn, for example, that “[i]n any text, the most frequent word will occur approximately twice as often as the second most frequent word, three times as often as the third most frequent word, and so on” (p. 31, Zipf’s first law).

In the second chapter, Divjak shifts the attention to grammar. In her own words, she “focus[es] on how grammar could be the mind’s response to repetition and look[s] at linguistic theories that rely on frequency of occurrence as a proxy for experience” (pp. 38-39). In other words, in this chapter, she takes seriously the cognitive linguistic idea that language is usage-based, meaning, among other things, that children learn grammar in the process of hearing it being used. She has boldly titled the chapter “Measuring exposure: Frequency as a linguistic game changer” (pp. 40-71). In considering how frequency could be a linguistic game changer, she discusses, among other things, work related to Goldberg’s construction grammar (e.g., Goldberg 2009), and Stefanowitsch and Gries’s work on collostructions (e.g., 2003), thus tying her work to the cognitive linguistic paradigm. She ponders, for example, the role of positive and negative evidence in language learning. At the end of the chapter, she nevertheless says that we do not yet know much about language cognition.

In Chapter Three, Divjak continues to “explore how frequency of occurrence could support the emergence of structure” (p. 72). She starts with explaining how humans detect statistical patterns and how common patterns are conventionalized. Then, she turns to probabilistic grammar and, lastly, to information theory. She begins the section on probabilistic grammar by referring to Chomsky’s argument that the “problem” of language learning “is unsolvable without strong prior constraints on the language, given that the linguistic stimulus is poor, i.e. partial and full of errors” (p. 80; she refers to Chomsky 1965). According to her:

“[t]aking a probabilistic standpoint makes learning look more tractable: learning no longer requires (re)constructing one particular grammar with certainty; it merely requires approximating it (speakers might, for example, learn slightly different idiolects) with sufficiently high probability” (p. 81).

Information theory, in its turn, is a “mathematical model of information transmission and a valuable tool for understanding the difficulties associated with such transmission” (p. 89). It has to do with the probability that something is going to happen, and can be applied, for example, to the predictability of what someone is going to say.

Towards the end of the chapter Divjak returns to drawing a line between Chomskyan and cognitive linguistics, stating that “[k]nowledge of language does not need to be understood as a minimal set of categorical rules or constraints, but as a (possibly redundant) set of gradient rules, which may be characterized by a statistical distribution” (pp. 95-96).

Chapter Four begins the second part of the book, dedicated to memory. In this chapter, Divjak asks such questions as “How does frequency help commit experiences to memory?”, “How do experiences change the brain?”, and even “What is memory?”. She defines ‘memory’ as follows: “Memory is the cover term for the ability to make what is learned persist through time” (p. 99). She considers memory, among other things, from the point of view of duration, dividing it into sensory memory (<1 sec), working memory (<30 sec) and long-term memory (p. 105). She also discusses the “physiology or neurobiology of memory” (pp. 108-114), as well as “memory systems, memory processes and neural mechanisms of memory storage” (pp. 115-119). She does all this to finally arrive at “computational models of memory for language” (pp. 118-119) and “memory effects that have been shown to play a crucial role in language learning” (p. 119 and onwards). The first of these effects is very familiar by now, namely frequency.

In her summary of the fourth chapter, Divjak issues a sub-manifesto: “For linguists, the time has come to delve deeper into research on memory.” (P. 128) She justifies her manifesto as follows: “Because memory for linguistic information is like memory for other information, frequency effects in language are memory effects and we need to understand how frequency achieves these effects.” (P. 128)

Divjak builds the following bridge between Chapters Four and Five (p. 129): “An important insight to take with us into the next chapter is that frequency alone does not ensure a memory trace is left.” The key word of Chapter Five is ‘entrenchment’, which Divjak defines as the “process by which linguistic experiences are mentally encoded and committed to memory” (p. 131). She begins the chapter by delimiting the borders of what she is going to discuss, which she does by comparing entrenchment in the mind with conventionalization in society. She then provides an analysis of entrenchment into three types: (1) “entrenchment as reduction in cognitive effort and increase in automatization” (pp. 137-139), (2) “entrenchment as unit formation for access and retrieval” (pp. 139-141), and (3) “entrenchment as chunking, possibly resulting in fusion” (pp. 141-143). She gives the “phonological reduction of English auxiliaries, such as I’ve, I’m and I’ll” as one example of fusion (p. 141).

Moving forward in Chapter Five, Divjak poses new questions: “How are repeated experiences recorded?” (pp. 143-148), “What is entrenched?” (pp. 148-150), “Is a threshold number of occurrences required for entrenchment?” (pp. 150-151), “Which frequency measure is ideal for predicting entrenchment?” (pp. 151-153), and “Is repetition frequency itself causal?” (pp. 153-155). The last question has to do with the fact that “[t]he frequency with which words occur is strongly correlated with other characteristics” (p. 153). Explaining this further, Divjak points out that “[h]ighly frequent words tend to be short in length, concrete rather than abstract, easily imaginable and they are acquired at an early age” (p. 153; Whaley 1978). After her thorough discussion of entrenchment, Divjak sounds a by now familiar note by bluntly stating that the “notion of entrenchment is frequently invoked in linguistic research, but the concept remains vague” (p. 155).

Then begins the third part of the book, which deals with attention. Divjak is again critical of the use of this term (p. 180): “The term ‘attention’ is typically used in a vague manner, including within psychology, where it originated.” She nevertheless dedicates an entire chapter to “The brain’s attention-orienting mechanisms” (pp. 161-181), and another chapter to “Salience: Capturing attention in and through language” (pp. 182-202). We can see here how she is all the time set out to characterize the general cognitive abilities which also apply to language before she applies them to language in particular (consider her manifesto).

Divjak suggests that “attention researchers typically define attention as the selection of information” (p. 163). She discusses psychological experiments dealing both with “attention and auditory perception” (pp. 164-166) and “attention and visual perception” (pp. 166-167). She talks about attention that is directed “bottom-up” and attention that is directed “top-down”. “Bottom-up” attention has to do with what captures a person’s attention in the immediate situation, while “top-down attention” has to do with “goals and tasks under cognitive, volitional control” (p. 168). She asks a very interesting question (p. 169): “Can linguistic processes take place without deliberate effort?” She suggests that syntactic processing is automatic while semantic processing is not. She discusses linguistic tasks related to object recognition and scene perception, and finally tackles the big question of the relationship between attention and memory.

Lastly in the chapter, Divjak explains very clearly why “attention is important for linguists” (p. 180). This is above all “because some have argued that attention, not repetition frequency, determines which information is encoded in memory” (p. 180). She then further explains that “[a]ttention plays a role for language in at least three respects” (p. 180). Firstly, it “may well be crucial for language development” (p. 180); and secondly, we are not capable of processing all the information about our surroundings but need attention to make selections. The third aspect has to do with explicit and implicit memories: the former “require conscious attention during encoding” (p. 180), while the latter do not.

In Chapter Seven, Divjak juxtaposes linguists’ as opposed to psycholinguists’/psychologists’ understandings of attention and language. She asks, among other things, what salience is in language and linguistics. She suggests that linguists operate with at least four definitions of salience: (1) “cognitive salience refers to a temporary mental activation state” and “ontological salience denotes an inherent property of entities in the real word”; (2) “salient is that which is foremost on one’s mind”; (3) salience is the “frequency with which a word is used to denote an experience”, or (4) “salient is that which is least expected” (p. 196). She summarizes the importance of salience as follows (pp. 197-198): “By its definition, salience refers to any (aspect of a) stimulus that makes it apparent to the perceiver. And what stands out has a better chance of being noticed and encoded in memory.”

Chapters Eight to Ten belong to the final section of the book. Chapter Eight deals with predicting, and Chapter Nine deals with learning. Divjak begins Chapter Eight by discussing “[p]redicting from stored memories” (pp. 206-212), but the bulk of the chapter is dedicated to “memoryless prediction”, and more specifically, “Bayesian predictive coding frameworks” (pp. 212-223). Divjak explains the attraction of Bayesian frameworks as follows (p. 231): “[B]iological implausibility has long plagued linguistic models and theories, and methodological insularity has impeded interdisciplinary approaches. A Bayesian approach promises to resolve both issues.” She explains that many disciplines have adapted Bayesian modelling of the brain and have been able to explain many things with the help of it. At the same time, she is nevertheless wary of Bayesian modelling because “[t]he ideal and rational Bayesian observer uses evidence in the best possible way to perform in a close to optimal fashion”, but the same does not necessarily apply to humans: “we appear to survive on good enough behaviour” (p. 131).

Since I am running out of space in this review, I will only mention one interesting detail from Chapter Nine. Divjak seems to be enthusiastic about Skinner’s (1957) monograph “Verbal Behavior”, which she calls “[t]he most direct application of insights from learning theory to language” (p. 243). In her view, Chomsky (1959), in a famous review, “distorted Skinner’s views” (p. 243). Chapter Ten I have already discussed in the introduction.

EVALUATION

Above all, I would like to say that Divjak takes Lakoff’s (1990) Cognitive Commitment very seriously. The famous commitment “is a commitment to make one’s account of human language accord with what is generally known about the mind and the brain” (Lakoff 1990: 40).

Lakoff (1990: 41) points out that “[c]ognitively real generalizations may not at all accord with generalizations arrived at by classical techniques of linguistic analysis”. This is precisely the conclusion that Divjak arrives at. She underlines that the linguistic categories linguists (even cognitive linguists) tend to operate with were not designed to “reflect the workings of the mind” (p. 271). Therefore, “[w]e need to think about what to count, we need to think about how to count and we need to think about the conclusions we can draw from all that counting” (p. 270).

Curiously though, while following Lakoff’s advice, Divjak has taken a rather big step from “the bulk of work done within [the cognitive linguistic] framework” (p. 261). This is due to her ability to imagine and put into words what could be done instead. Considering the popularity of the cognitive linguistic approach, and even its relatively recent quantitative turn (Janda 2013), this is no small feat.

I want to make a couple more points before I finish this review. Firstly, the feat that Divjak achieves with this book did not happen overnight, nor did she do everything alone. Already in 2012, Divjak and Gries edited a book titled “Frequency Effects in Language Learning and Processing”. A quick search in this book produces 58 hits for the word “learning”, 26 hits for the word “memory”, and 11 hits for the word “attention”. It looks like the seeds for the current book.

Secondly, the intriguing question is who is going to follow Divjak’s lead and how. If we compare the current book with such a cognitive linguistics classic as “Metaphors We Live By”, it seems clear that it is more challenging to become a cognitive linguist à la Divjak than to become a cognitive linguist à la Lakoff and Johnson. That there is already a paperback edition of Divjak’s book suggests that there is nevertheless a public that is interested in the kind of issues she promotes. I would assume that this public does not simply consist of “old school” cognitive linguists interested in new insights but also other linguists and, quite importantly, people from other disciplines such as psychology and computer science.

REFERENCES

Chomsky, Noam. 1959. Review of Verbal behavior by B. F. Skinner. Language 35(1). 26-58.
Chomsky, Noam. 1965. Aspects of the theory of syntax. Cambridge, MA: The MIT Press.
Goldberg, Adele. 2009. The nature of generalization in language. Cognitive Linguistics 20(1). 93-127.
Gries, Stefan Th. & Dagmar Divjak (eds.). 2012. Frequency effects in language learning and processing. Berlin: De Gruyter Mouton.
Janda, Laura. 2013. Cognitive linguistics – the quantitative turn: The essential reader. Berlin: De Gruyter Mouton.
Lakoff, George. 1990. Is abstract reason based on image schemas? Cognitive Linguistics 1(1). 39-74.
Lakoff, George & Mark Johnson. 1980. Metaphors we live by. Chicago: The University of Chicago Press.
Littlemore, Jeannette. 2009. Applying cognitive linguistics to second language learning and teaching. London: Palgrave Macmillan.
Skinner, B. F. 1957. Verbal behavior. New York: Appleton-Century-Crofts.
Stefanowitsch, Anatol & Stefan Th. Gries. 2003. Collostructions: Investigating the interaction of words and constructions. International Journal of Corpus Linguistics 8(2). 209-243.
Whaley, C. P. 1978. Word-nonword classification time. Journal of verbal learning and verbal behavior 17. 143-154.

ABOUT THE REVIEWER

Heli Tissari works as an associate professor of English at Umeå University in Sweden. Her research interests include cognitive and corpus linguistics, emotions, English historical linguistics, and semantics. Her latest published work used Natural Semantic Metalanguage to explain the concept of ‘chastity’ in 18th century English.