small language model

Augustinian BabyLM: What Ostensive Definition Can and Cannot Teach a Small Language Model

A language model normally begins training with random word embeddings: whatever'banana'means must be learned from training corpora. I implement St. Augustine's picture of word learning, meaning by ostension, for a small masked language model (DeBERTa) trained on 10M words: before training, visually grounded tokens rece...

Automatic Lyric Transcription for Greek Songs: Scaling and Task Composition Effects in Whisper Adaptation

Results show that scaling consistently improves performance, while multitask learning acts as a beneficial regularizer primarily for smaller-capacity models.

Distribution-aware Language Neuron Identification in Multilingual Large Language Models

Distribution-aware Language Neuron selection is proposed, which leverages pairwise relationships between per-language activation distributions over the full activation range, including negative values, to quantify each neuron's language specificity by clustering languages using pairwise overlap coefficients between the...

Data-Efficient Language Modeling: From Frontier Advancement to Principle-Guided Model Improvement

Learning from limited text requires models to use context, generalize to new inputs, and retain useful capabilities. Qiushi Engine conducted a long-horizon, end-to-end autonomous research program on BabyLM 2026 Strict-Small, within 10 million corpus words and 100 million cumulative word presentations. Three stages conn...

Structural priors for data-efficient language learning

Non-language data can serve as a partial substitute for language data for the training objective of next-token prediction but does not reliably support broader linguistic generalization, and transfer from non-language data is less efficient than additional language data.

Your Model Already Knows Don't Teach It, Learn to Ask It: Soft Prompting for Few-Shot Adaptation of Vision-Language Models

Modern VLMs already encode much of what is needed for specialized domains; the challenge is learning how to ask, suggests modern VLMs already encode much of what is needed for specialized domains; the challenge is learning how to ask.

Why Does Post-Training Quantization Work?

Comparing full-precision and quantized forward passes, and identifying two mechanisms that characterize pretrained quantization robustness, these mechanisms explain why quantization error that passes through numerous layers can still produce only small output changes, and are verified across models and quantization set...

Flow Duality and Source Geometry for Categorical Generation

Continuous and discrete flow matching are usually treated as separate constructions. This paper identifies a duality between them: projecting continuous convex-interpolant paths with one-hot targets through a position-wise argmax yields discrete convex-interpolant paths. The result requires source laws with appropriate...

GEOSTEER: Geodesic Optimization for Activation Steering in Large Language Models

GeoSteer is proposed, an optimization-based method for norm-preserving activation steering that consistently improves over state-of-the-art activation steering baselines and replaces predefined one-step edits with adaptive, geometry-aware optimization.

Chain-of-Thought supervision enables explainable and efficient medical summarization with Small Language Models

Major Change in Human Life: The Decline of the Traditional PC and the Rise of the Supercomputer PC

The "death of the normal PC" should not be seen as the extinction of computers used for routine work, rather, it symbolizes the demise of the conventional computing paradigm, where the PC is essentially a passive instrument for executing programs.

Extracting AI agent-accessible data from biodiversity literature with corpus

[1.4.0] - 2026-09-09 Theme — v1.4 silent wrongs and the hazards behind them v1.3 made the pipeline show where evidence came from. This cycle went after what survives behind that: results that are wrong without saying so, and build hazards that cost hours per run without ever failing. The organizing complaint is that a...