Distributional Semantics, Holism, and the Instability of Meaning
Abstract Large language models are built on the so-called distributional semantic approach to linguistic meaning that has the distributional hypothesis at its core. The distributional hypothesis involves a holistic conception of word meaning: the meaning of a word depends upon its relations to other words in the model. A standard objection to holism is the charge of instability: any change in the meaning properties of a linguistic system would lead to many changes in the entire system. This chapter examines whether the instability objection poses a problem for distributional models of meaning. First, it distinguishes between distinct forms of instability that these models could exhibit and argues that only one such form is relevant for understanding the relation between instability and communication: differential instability. Differential instability is variation in the relative distances between points in a space rather than variation in the absolute position of those points. The chapter distinguishes differential and absolute instability by constructing two smaller language models. It demonstrates the two forms of instability by showing that these models change as the corpora they are constructed from increase in size. It argues that the instability these models display is constrained by the structure and scale of relationships between words, such that the resistance to change for a word is roughly proportional to its frequent and consistent use within the language system. The differential instability that language models exhibit allows for productive forms of meaning change while not leading to the problems raised by the instability objection.