A Region, Not a List: A Pre-Registered Bilingual Induction-Circuit Probe Across Three Scales

Abstract

Mechanistic interpretability has mapped the induction circuit –- the attention head that copies a token it has seen in context –- almost exclusively in English models, and a circuit localized in one small carrier is routinely read as if it described the family. This paper does both tests at once. We report a pre-registered head-level probe of induction inside a Chinese-dominant bilingual model (Qwen2.5-0.5B-Instruct; 24 layers x 14 heads), run on CPU only and with no external interpretability library, and then apply the same driver, the same $20$ parallel sentence pairs, the same score and the same frozen criteria to a larger carrier of the same family (Qwen2.5-1.5B-Instruct; 28 layers x 12 heads) and to a third carrier at a further scale (Qwen2.5-3B-Instruct; 36 layers x 16 heads). At the reference scale, a token-level random-repetition probe and a Chinese natural-language probe recover the identical top-8 head set (Jaccard $1.000$): the natural-language phenomenon is the same mechanism, head for head. Chinese and English share six of eight heads, with language-specific members (Chinese L16H02/H03; English L09H10 and the final-layer L23H02), and Chinese induction strength is $4.2 $ English ($0.1912$ vs.\ $0.0454$); forced character-level segmentation leaves the head set largely intact ($0.600$, granularity-invariant). The cross-scale test splits. The instrument replicates: the circuit is detectable with the same margin (signal maxima $0.971/0.993$ against null maxima $0.031/0.028$), the Chinese cross-level consistency holds ($0.455$), language sharing holds ($0.455$), and the granularity reading is preserved ($0.600$, granularity-invariant). The membership does not replicate uniformly: the English cross-level consistency is non-monotone across scale ($0.600$ at $0.5$B, fails at $1.5$B with $0.231 < 0.4$, restored at $3$B with $0.455$), and the strength asymmetry narrows with scale ($4.2 4.6 2.0 $) while remaining positive. A post-hoc check under three set definitions separates the English failure at $1.5$B (partly a top-$k$ truncation artifact: top-16 restores $0.455$) from the granularity reversal, which is substantive at $1.5$B (all three definitions agree) but definition-sensitive at $3$B (top-8 $0.600$, score $>0.5$ only $0.167$) –- the participation claim rests on the pre-registered top-8 definition, not on definition robustness. The durable cross-scale objects are therefore the circuit region and its strength asymmetry; the exact membership list is not. We report the negative branches as the result, not as a failure. Pre-registrations, both probe ledgers, the shared driver and analysis scripts are available from the authors upon reasonable request via email.