Skip to content
Conference Open access

Reading Time Modeling with Pixel-Based Language Models

Oct 2026 · Proceedings of the 2026 Conference on Empirical Methods in Natural Language Processing · pp. 27481-27500 · 0 citations · 47 references
Computer Science

TL;DR

This study first integrates PLMs into the modeling of naturalistic human reading and examines their use to estimate next-word predictability, which can also integrate orthographic and parafoveal information, demonstrating that next-word prediction difficulty has strong predictive power for gaze duration and word skipping behavior across various languages.

Abstract

Tokenization-free, pixel-based language models (PLMs) have recently been proposed as an alternative to conventional token-based language models (TLMs), typically motivated by practical considerations, e.g., the flexibility to support new languages or vocabularies. Nevertheless, PLMs also have the potential to serve as infrastructure for computational (psycho)linguistics, where tokenization has been a source of bias in estimating information-theoretic values, such as surprisal, and has pushed the bottom-up visual orthographic features outside the scope. In this study, we first integrate PLMs into the modeling of naturalistic human reading and, in particular, examine their use to estimate next-word predictability, which can also integrate orthographic and parafoveal information. Our results demonstrate that next-word prediction difficulty, computed in pixel space, has strong predictive power for gaze duration and word skipping behavior across various languages. In particular, their advantage is complementary to existing token-level surprisal and entropy estimates. Cross-lingual analyses with script variations also clarify their distinctive visual processing biases, facilitating research into how bottom-up visual information is incrementally integrated into expectation-based reading.

Read PDF

Similar papers

#natural language process... Conference Open access Sep 2026

Using LMs to Model the Effects of Context and Coreference during Sentence Comprehension

This work systematically vary the context window size of GPT-2 across four large-scale naturalistic English reading-time datasets and observes a U-shaped relationship, demonstrating that tracking long-range coreference relations is one important factor for the alignment between LM surprisal and human reading behavior.

Kohei Kajikawa, Lin Ai, Tatsuki Kuribayashi et al. · 0 citations
Open access Oct 2026

Semantic Update as a Predictor of Reading Time: Moving Beyond Word-Level Surprisal

This study explores the role of semantic prediction error in shaping reading behavior, as reflected in reading time during naturalistic reading. We operationalize semantic prediction error using semantic update (SU), a measure derived from the Sentence Gestalt (SG) model, which simulates incremental predictive comprehe...

Siddharth Gupta, A. Lopopolo, Milena Rabovsky · 0 citations
#artificial intelligence Open access Oct 2026

StructuralCost: A controlled reading time dataset for modeling human sentence processing difficulty

We introduce STRUCTURALCOST, a self-paced reading dataset of 475 participants and 40,800 observations isolating the processing cost of long-distance subject-verb dependency resolution. We replicate a low-powered psycholinguistic finding at NLP scale, namely that human reading times at the main verb increase with depend...

Nina Nusbaumer, Iria de-Dios-Flores, Corentin Bel et al. · 0 citations

We use cookies to run the site and, with your consent, for analytics and to show ads. See our Cookie Policy.