Author

Alexey Dontsov

2 papers indexed here

We haven’t gathered this author’s papers yet. Follow them and we’ll fetch their work.

Not the right person? Other researchers publish under this name.

#artificial intelligence Preprint Oct 2026

Sparse Transformer: a Controlled Testbed for Mechanistic Interpretability

Mechanistic interpretability aims to decompose neural network computations into human-understandable features, yet progress is limited by the absence of ground-truth units to evaluate against. We address this with a Sparse Transformer (ST): a 5-layer language model trained on TinyStories whose wide, TopK-sparse MLP for...

Anton Korznikov, Alexey Dontsov, Elena Tutubalina · 0 citations

#natural language process... Preprint Sep 2026

What Attention Recalls and Recurrence Controls in Hybrid Language Models

Two cache-level interventions are introduced: Split-prefill keeps only the KV cache or only the recurrent state from a prefilled context, then generates an answer, and state-swap pairs the KV cache from one context with the recurrent state from another in a single forward pass.

Kirill Afendulev, Alexey Dontsov, Elena Tutubalina et al. · 2 citations · ⚡1