Skip to content

Category

small language model

3,366 papers

#artificial intelligence Preprint Oct 2026

Learning to Read Out: Unembedding Dynamics in Language Model Pretraining

When does a language model acquire a capability? Developmental studies usually answer this by tracking when models start producing the correct token. But that prediction passes through the unembedding matrix, or readout, which is itself still being learned during pretraining. We show that a model's hidden states can en...

Matteo He, William F. Shen, Alexandru Iacob et al. · 0 citations
#artificial intelligence Preprint Oct 2026

Same Context, Different Agents: A Behavioral Study of Semantic Layers, Orchestration and Compaction in Data Agents

Answering questions over heterogeneous data means combining evidence from files, databases, documents and the web. Data agents automate this, but their success depends on context. Three context-engineering techniques (CETs) help by shaping it: semantic layers describe the data, orchestration splits work among agents, a...

Zhivar Sourati, Marianne Menglin Liu, Zheng Wang et al. · 0 citations
#artificial intelligence Preprint Oct 2026

Interpreting Language Model Parameters

The method, adVersarial Parameter Decomposition (VPD), optimizes for decompositions of neural network parameters into simple subcomponents that preserve the network's input-output behavior even when many subcomponents are ablated, including under ablations that are adversarially selected to destroy behavior.

Lucius Bushnaq, Dan Braun, Oliver Clive-Griffin et al. · 4 citations · ⚡1
#artificial intelligence Review Oct 2026

Improving Environment-Agnostic Coding Agents: The Role of Scaffolding

Stronger language models may increasingly construct useful procedures for themselves, reducing the advantage of developer-supplied scaffolding. We investigate why this advantage persists and whether an environment-agnostic agent can be improved without environment-specific engineering. We evaluate a Codex-based agent i...

Sergey Rodionov · 0 citations
#artificial intelligence Preprint Oct 2026

Disagreement-Guided Weak-to-Strong Routing

Small language models are increasingly deployed on edge devices and robotic systems to meet cost and latency constraints, yet their limited capabilities can make reliable deployment challenging. Stronger models can provide assistance when needed, but deciding when to route a weak-model response to a stronger model ofte...

Sima Noorani, Shayan Kiyani, George Pappas et al. · 0 citations
#artificial intelligence Preprint Oct 2026

PROBE: Learning to Audit Policy Compliance in Tool-Using LLM Agents

Modern agentic systems extend language models with external tools, enabling them to complete complex tasks through multi-turn interactions. As these agents are deployed in consequential settings, they are often constrained by policies governing pricing, privacy, confirmation, safety, and other requirements. However, ex...

Kshitij Mishra, Abhijith Sharma, Nils Lukas et al. · 0 citations
#artificial intelligence Preprint Oct 2026

Ontology Refinement with Embeddings and LLMs: A Case Study on CASTON

Ontologies are widely used to represent and organize complex knowledge in domains such as biomedicine and healthcare. However, constructing reliable ontologies remains challenging, as class hierarchies, whether manually or automatically constructed, may contain subtle subsumption errors that require careful refinement....

Hui Yang, Zi-Yu Li, Xuan Liu et al. · 0 citations
#artificial intelligence Preprint Oct 2026

Learning from Interaction Experience via Localized Context Distillation

Learning from interaction experience is essential for enabling large language models to continually improve during deployment. Context distillation offers a promising path: privileged information extracted from experience is provided to a teacher, whose behavior is then distilled into a student model that has no access...

Zhe-Zheng Hao, Yong-Kang Yang, Xia-Ling-Zhang-Yu-Xin Xia et al. · 0 citations
#artificial intelligence Preprint Oct 2026

DocPilot: Interactive Agent Orchestration and Training for Long Visual Document Understanding

Multimodal long-document question answering requires models to seek sparse evidence across pages and modalities, understand heterogeneous document content, and reason over the collected evidence. Even advanced vision-language models (VLMs) can overlook sparse evidence and struggle with cross-modal reasoning in a single...

Zhuo-Wen Liang, Xiao-Tian Lin, Yu-Yu Luo et al. · 0 citations
#artificial intelligence Preprint Oct 2026

A Self-Improving Agent that Grows its Own DSL: Oracle-Free Operator Induction for Exact Question Answering

A self-improving agent has to decide which new skills to keep, and in practice it keeps whatever earns a high reward, convinces a judge model, or compresses its library, even when the skill is wrong. That is fine until the answers have to be exactly right: over a table of transactions, a total is either right or wrong....

Yun-Shiuan Chuang, Yi-Rou Ge, Yi-Xi Li et al. · 0 citations

An enhanced framework for automated vehicle crash severity prediction under unobserved heterogeneity using natural language processing and data augmentation.

Objective Autonomous vehicle (AV) crashes continue to occur, indicating the need for a comprehensive understanding of their characteristics to support safety management in future mixed-traffic environments. Previous studies employing text mining or rule extraction have faced limitations in generalizing findings due to...

Song-Ha Lee, Nuri Park, Hojae Kim et al. · 0 citations
#small language model Open access Oct 2026

UAV-ESPrune: Evidence-Aware Visual Token Pruning for UAV Vision-Language Reasoning

Low-altitude unmanned aerial vehicles (UAVs) require compact multimodal inference for perception, environment understanding, and planning-oriented reasoning. Vision-language models (VLMs) provide a shared interface for these tasks, but long visual-token sequences increase language-model prefill computation, sequence-de...

Ziyang Xu, Peng Han · 0 citations

From tech blogs

See all →
MIT News · Artificial Intelligence Oct 23, 2026

Q&A: How AI came to model the “messy” human mind

Assistant Professor Benjamin Lindquist studies how engineers deliberately injected randomness into early AI, giving rise to deep-learning models that sometimes hallucinate.

Microsoft Research Blog Oct 22, 2026

Flint-interactive: A language for creating interactive data visualizations

Flint-interactive gives AI agents a language for creating interactive visualizations. Click a chart, filter data, highlight related content, and explore beyond the view the author originally chose. The post Flint-interactive: A language for creating interactive data visualizations appeared first on Microsoft Research.

We use cookies to run the site and, with your consent, for analytics and to show ads. See our Cookie Policy.