Byte-pair Encoding Captures Aspects of Word Meaning that Distributional Semantics Overlooks
Large language models learn word meanings through distributional semantics (i.e., patterns in text), and distribution does not always capture information about taxonomy and function, which are essential to human representations of word meaning. However, sound and spelling often conveys such information, and language...