The Genealogy of Large Language Models: From Auxiliary Tools in ASR to Foundational Transformers and Back Again

Unifying the structures of language in a neural population code.

It is concluded that explaining how language can emerge from neural population codes, in both biological and artificial systems, will not be achieved through the incremental refinement of algebraic-symbolic theories but will demand new theoretical paradigms.

The Versatility of Large Language Models: A Comprehensive Review and Structured Survey of Architectures, Applications, Challenges, and Future Trajectories

This survey reviews the evolution of language models from early statistical approaches to modern Transformer-based architectures and summarizes key developments, including attention mechanisms, scaling laws, alignment techniques, and efficient inference methods.

The Pitfalls of Maximum Likelihood Training of TPMs for Controllable Language Generation

This work finds that likelihood-trained TPMs can result in failed generations due to overly large corrections to the LM’s logits, and trains TPMs with LM-aligned objectives that better align with the LM token-probability space.

A LSTM language model learns Hindi-Urdu case-agreement interactions, and has a linear encoding of case

Interpretability in the Era of Large Language Models: Mechanistic Methodology, Empirical Practices, and Applications

This tutorial provides a comprehensive, end-to-end view of LLM interpretability, transitioning from microscopic neural analysis to macroscopic application and deployment, and explores how these interpretability paradigms scale and inspire the design of frontier architectures, agentic systems, and thinking models.

Comparative Benchmarking of Probabilistic, Recurrent, and Self-Attention Models for Autoregressive Genomic Sequence Modeling

Transformer-based language models have achieved remarkable success in natural language processing, and their structural parallels with DNA sequences, both being linear strings over a finite alphabet, motivate their application to genomics. Although discriminative genomic language models such as DNABERT have been explored, autoregressive generative approaches remain comparatively underutilized. This study presents an empirical comparative evaluation of three classes of autoregressive sequence models: N -gram statistical models, long short-term memory (LSTM) recurrent networks, and transformer-based architectures, applied to human gene nucleotide sequences. Rather than processing full-length genomic sequences, which impose prohibitive computational costs, we restrict analysis to sequences of up to 1,000 nucleotides sourced from the National Center for Biotechnology Information Gene Database. Models are evaluated using perplexity on held-out sequences and, more practically, by their ability to distinguish genuine gene sequences from synthetically mutated variants across three mutation levels. Our results demonstrate that LSTM-based models consistently achieve the best mutation-detection accuracy across all conditions, while N -gram models with Laplace smoothing perform competitively relative to their simplicity and low computational cost. Transformer models, despite their theoretical capacity for long-range dependency modeling, show lower mutation-detection accuracy in this constrained, short-sequence setting. This work provides a resource-efficiency analysis and empirical benchmark for model selection in constrained genomic modeling tasks. It highlights that computationally expensive deep learning architectures do not unconditionally outperform lightweight statistical baselines on small, vocabulary-constrained genomic datasets, and identifies clear directions for future investigation.