Skip to content
Preprint

Prompt-Model Interaction Reaches the Fixed Points: A deterministic, task-free structural readout -- and the factorizations of it that failed

Aug 2026 · 1 citation · ⚡ 1 influential · 17 references
Computer Science

TL;DR

Four phenomenological factors -- prose-versus-markup, a universal direction, bidirectionality, instruct-resistance, and four phenomenological factors -- were each withdrawn within one run of being proposed, dissolved by widening the sample.

Abstract

That a prompt's effect is not a property of the prompt is established: prompts optimised for one model degrade on another, and rankings reorder under neutral reformatting. That evidence is about task accuracy, which cannot say whether the interaction is a fact about task machinery or about the conditional distribution itself. We ask on a readout with no task in it: the fixed-point structure of the short-window argmax map x_{t+1} = argmax_x p(x | x_{t-1}, x_t), censused from 96 starts. It is deterministic, so nothing can be helped or hurt, and it exists only at short windows -- four of six models lose it entirely by window 16 -- so everything here concerns how a model reads a fragment. Two results. First, the interaction reaches this readout at full magnitude: nine tokens of conditioning move the fixed-point fraction across most of its range, change a four-way structural class, and reorder models, while instruction tuning worth 60.5 IFEval points moves the class by zero. Second, nothing we proposed carries it. Prefix length fails: the effect is not monotone. Four phenomenological factors -- prose-versus-markup, a universal direction, bidirectionality, instruct-resistance -- were each withdrawn within one run of being proposed, dissolved by widening the sample. And the nearest mechanistic account, attention-sink dominance of early tokens, predicts the sign of the shift on 2 of 5 models -- chance -- while a length-by-content cross shows it holds on real text and fails on our probe's uniformly random input, so we are outside its regime, not against it. One fixed nine-token prefix drives four models toward 0 and two toward 1; the bidirectionality survives in-distribution starts. On this readout the unit of explanation is the prompt-model pair. The recurring error it caught in us has a name: a criterion with a shape applied to a quantity with no room to vary.

View source

Similar papers

#artificial intelligence Preprint Sep 2026

Scoring the Wrong Question: Readout Failures in Constrained-Option Evaluation

Constrained-option scoring reads a model's probabilities for a fixed set of permitted answers, so it returns a score even when the model is about to write something else. We study a prompt that quotes a multiple-choice item, ending in the item's own answer instruction, and asks for a forecast of whether a reader model...

Jia-Xuan Guo, Ke-Jia Zhang, Shuo Xin et al. · 0 citations
#artificial intelligence Preprint Sep 2026

Reasoning on the Simplex: Geometric Fixed-Point Models

Geometric Fixed-Point Reasoning (GFPR), in which the iterated state is the prediction itself: a field of categorical beliefs on a product of simplices, whose argmax is the answer at every step.

T. Daulbaev, И. Д. Глазков, Maxim V. Rakhuba et al. · 0 citations
#artificial intelligence Preprint Sep 2026

Same Outcome, Different Readout: What Does a Steerable Valence Direction in LLMs Represent?

Claims about what an internal direction in an LLM represents need evidential constraints beyond an observer's prior beliefs about the system. Decodability and successful activation steering do not, by themselves, establish which construct the direction tracks. This gap is especially consequential for welfare-relevant i...

Wei-Han Li, Xin-Lei Chen, Yu-Han Song et al. · 0 citations
Preprint Aug 2026

Activation-Space Order-Swap Geometry: A Site-Asymmetry Audit

Order-dependent activation statistics are often interpreted as evidence of interaction, but that interpretation can be confounded by where interventions enter the network. We introduce a no-fit site-asymmetry audit. For a twice-differentiable readout, the open-path order-swap decomposes into a canonical additive respon...

Anqi Peter Li · 0 citations
#artificial intelligence Preprint Aug 2026

The Halt Vector: Internalizing a Causal Steering Intervention for Efficient Reasoning

The mechanism is a halt vector: a difference-of-means direction at layer 18 of this model whose steering strength controls how long it thinks, while a replicated value axis does nothing, and what works is reconstructing the whole steered activation with those dimensions pinned to their natural values.

Dylan Jayabahu, Tinuade Adeleke · 0 citations
#machine learning Review Sep 2026

Machine learning for the LHC physics program: a 2025-2026 stocktake

The first sentence of this abstract--and the introduction to these proceedings--was authored by a human, but the bulk of this document was generated by an agentic AI system. In this talk, I take stock of machine learning (ML) for the LHC physics program over the twelve months from May 2025 to May 2026. The corpus is th...

Jesse Thaler · 0 citations

We use cookies to run the site and, with your consent, for analytics and to show ads. See our Cookie Policy.