Searching for Emergent Geometric Coordinate Systems in LLM Hidden Representations: A Replication Attempt

Abstract

This is a mechanistic-interpretability replication attempt testing whether large language models (Qwen2.5-0.5B, Mistral-7B-v0.1) encode small closed relational sets, such as days of the week and months, as literal low-dimensional geometry in hidden-activation space, via a graph-Laplacian probe. We report six results, including two chained null results, last-token causal patching and a literal 2D rotation test, that isolate a genuine positive finding: a full-rank rotational causal code recovered via subject-token patching and orthogonal-Procrustes rotation. This code generalizes partially to other small closed sets (zodiac signs, planets, playing cards) in proportion to subject-name token length. Every claimed number is backed by real code and real intermediate data files at each pipeline stage. Statistics are done properly, with binomial and Fisher exact tests and Wilson confidence intervals, and the negative results (the last-token patching failure, the literal 2D rotation failure) are treated as first-class evidence alongside the positive finding rather than omitted.