The idea is that if a word is predictable in its context, then an optimal processor will already have done the work of processing that word, and so very little work remains to be done when the predicted word is really encountered (Hale, 2001 Jurafsky, 2003). The first are expectation‐based theories, which hold that the observed difficulty in processing a word (or phoneme, or any other linguistic unit) is a function of how predictable that word is given the preceding context. Furthermore, our processing cost function is capable of providing intuitive explanations for complex phenomena at the intersection of probabilistic expectations and memory limitations, which no explicit model has been able to do previously.īroadly speaking, models of difficulty in incremental sentence processing can be divided into two categories. In particular, we aim to introduce a processing cost function that can derive the effects of both probabilistic expectations and memory limitations, which have previously been explained under disparate and hard‐to‐integrate theories. The goal of this paper is to give a high‐level information‐theoretic characterization of the integration function along with a linking hypothesis to processing difficulty which is capable of explaining diverse phenomena in sentence processing. By characterizing the processing difficulty that occurs when each symbol is integrated, we hope to be able to sketch an outline of the processes going on inside the integration function for certain inputs. This processing difficulty can be observed in the form of various dependent variables such as reading time, pupil dilation, event‐related potentials on the scalp, etc. Some utterances are harder to understand than others, and within utterances some parts seem to engender more processing difficulty than others. The goal of much research in psycholinguistics has been to characterize this integration function by studying patterns of differential difficulty in sentence comprehension. The function which combines w i and r i −1 to yield r i is called the integration function. Upon receiving the ith symbol w i, the listener combines it with the previous incremental representation r i −1 to form r i. An utterance is taken to be a stream of symbols denoted w, which could refer to either words or smaller units such as morphemes or phonemes. Fig.1 1.Ī schematic view of incremental language comprehension. The incremental view of language processing is summarized in Fig. When this function is applied successively to the symbols in an utterance, it results in the listener's final interpretation of the utterance. Under a strong assumption of incrementality, the process of language comprehension is fully characterized by an integration function which takes a representation r and an input symbol w and produces an output representation r'. This is an integration of two parts: The listener must combine a representation r, built based on what she has heard so far, with the current symbol w to form a new representation r'. Over the years, extensive evidence from experiments as well as theoretical considerations has led to the conclusion that this process is incremental: The information contained in each word is immediately integrated into the listener's representation of the speaker's intent. Furthermore, we demonstrate that dependency locality effects, a signature prediction of memory‐based theories, can be derived from lossy‐context surprisal as a special case of a novel, more general principle called information locality.įor a human to understand natural language, they must process a stream of input symbols and use them to build a representation of the speaker's intended message. We show that this model provides an intuitive explanation for an outstanding puzzle involving interactions of memory and expectations: language‐dependent structural forgetting, where the effects of memory on sentence processing appear to be moderated by language statistics. Our model, lossy‐context surprisal, holds that the processing difficulty at a word in context is proportional to the surprisal of the word given a lossy memory representation of the context-that is, a memory representation that does not contain complete information about previous words. In this work, we present a new model of incremental sentence processing difficulty that unifies and extends key features of both kinds of models. Models that explain and predict this difficulty can be broadly divided into two kinds, expectation‐based and memory‐based. A key component of research on human sentence processing is to characterize the processing difficulty associated with the comprehension of words in context.
0 Comments
Leave a Reply. |