Skip to content

Training and Inference Dynamics of PLDR-LLMs: Row-Map Collapse, Renormalization, and Predictive Reduction

Sep 2026 · 0 citations
Computer Science

TL;DR

This monograph develops a unified account of training and inference in Power Law Decoder Representation language models (PLDR-LLMs) with exact identities, conditional dynamical claims, and finite empirical findings, with proofs, selected formal checks, and compact numerical evidence.

Abstract

This monograph develops a unified account of training and inference in Power Law Decoder Representation language models (PLDR-LLMs). Exact finite work identities decompose changes in the absolute energy of the row-centered learned map into parameter contributions, signed interactions, and numerical observation defects. Positive affine blocking retains restarts at the row-constant face, while the augmented AdamW state supplies the complete dynamical description. Predictive renormalization acts on the complete conditional training law for a single pass over distinct corpus target blocks, retaining optimizer memory, remaining data, schedule, and numerical policy. Autonomous reductions require closure; approximate reductions carry successor and emission errors. Finite-population covariance, matched physical clocks, matrix fluxes, and signed temporal energy connect row dynamics to model-wide observations. Absolute row collapse, relative row concentration, operator stabilization, and predictive accuracy are distinguished. Experiments reveal observer and optimizer dependence, reject the tested autonomous row-state candidates, and support finite conditional prediction and state-specific operator reduction. Independent single-pass families exhibit moving finite fluctuation regions without establishing a thermodynamic critical class. Conditional symmetry, head limits, covariance flows, and readout error budgets specify assumptions needed to transfer scaling laws to inference. The theory separates exact identities, conditional dynamical claims, and finite empirical findings, with proofs, selected formal checks, and compact numerical evidence.

View source

Similar papers

#machine learning Preprint Sep 2026

The Dynamics of Continuous Mixture Collapse in Language Models

This work studies why pretrained language models often fail to preserve mixtures of many components and shows that exact preservation generally requires context-dependent correction, whose required dimensionality can grow with the number of components.

Ali Backour · 0 citations
#artificial intelligence Preprint Sep 2026

Principled Thoughts for Latent Recursive LLM Systems

REST (REpresentation-Supervised Thoughts), a training objective that turns four properties of a valid thought representation into differentiable losses added to CE, instantiate in latent single-agent and multi-agent systems, without architectural changes or added parameters at inference.

Fahd Seddik, F. Fard · 0 citations
#machine learning Preprint Aug 2026

Linear RNN Scaling Laws: When Longer Sequences Beat More Sequences

A tractable teacher--student model where a stable latent linear RNN generates trajectories and a sketched linear recurrent student is trained by safeguarded full-batch WSD gradient descent on next-token prediction is studied.

Zi-Yan Chen, Zhong-Zhu Zhou, Pei-Lin Liu et al. · 0 citations
#machine learning Preprint Oct 2026

When Dynamic Scaling Fails: A Scale-State Feedback Governs Recurrent-State Quantization in Linear-Attention and SSM LLMs

Linear-attention and state-space LLMs carry long context in a fixed-size recurrent state, and quantizing that state is the natural way to cut its cost during long decode. Unlike a weight tensor, the state is rewritten at every step, so a quantizer applied at each step sits inside the recurrence and its errors compound....

Yu-Hang Jiang · 0 citations

Related blog posts

MIT News · Artificial Intelligence Aug 27, 2026

Looking beyond natural sequences

A new machine-learning framework aims to improve the success rate of computational protein design while moving away from results that reproduce sequences found in nature.

We use cookies to run the site and, with your consent, for analytics and to show ads. See our Cookie Policy.