Skip to content

Category

small language model

3,317 papers

#natural language process... Preprint Oct 2026

CLUE: Uncovering Social Context for Pragmatic Understanding

Understanding a conversation requires connecting what is said to the speakers'relationships, expectations, and earlier exchanges. Prior work shows that supplying relevant context can improve models'social judgments, but much of this information may already be present in the conversation. This motivates training models...

Cheng-Fei Wu, Zhao-Qing Wu, Dan Goldwasser · 0 citations

Intern-Decision: Efficient Multimodal Decision Making with Autoregressive Masked Language Modeling

Interactive agents repeatedly choose among a finite set of actions, labels, or ordinal outcomes. Generating a free-form response for each choice adds latency and complicates the interface between a model and its environment. We present Intern-Decision, a family of 0.8B, 2B, and 4B multimodal decision models that return...

Tong Zhu, Zhe Xu, Ze-Feng He et al. · 0 citations
#natural language process... Preprint Oct 2026

Identifying Effective Strategies in Prompt Evolution for Stress and Emotion Prediction

Early stress detection and risk prediction are crucial for timely mental health assessment and support. While large language models have recently shown promise for social media stress prediction, relying on accuracy can obscure false positives. Also, prompt revisions yield inconsistent gains across models and tasks, wi...

Seung-Jun Yi, Seung-Jun Yi, Sriram Sattiraju et al. · 0 citations
#machine learning Preprint Oct 2026

The Right Byte First: How UTF-16 Helps Multilingual Boundary Prediction

This work trains small causal predictors on matched FineWeb-2 text and evaluates on FLORES-200 under UTF-8, UBE, and UBE's same-length big-endian variant, finding that UBE's mean error falls below UTF-8's final mean error after only 25% and 37.5% of training.

Hyunsik Kim · 0 citations
#machine learning Preprint Oct 2026

HyCo: Hybrid Drafting for Cost-Efficient Mobile-Cloud Speculative Decoding

Device-cloud speculative decoding reduces server computation, but growing a deep draft tree with a small language model (SLM) requires repeated passes on a mobile neural processing unit (NPU). We present HyCo, which assigns depth to the server model's native multi-token prediction (MTP) head, executed on the device, an...

Hanbin Bae, Heejin Choi, Joohyung Lee et al. · 0 citations
#machine learning Preprint Oct 2026

Augment the Cache, Replace the Activation: A 2x2 Study of Internal-State Sharing

Multi-agent systems built from frozen language models communicate through text, which discards the internal state the sender already computed and makes the receiver pay to rebuild it. Work on internal channels avoids that cost, framing the problem as transfer: agent A sends information to agent B. Every agent pair that...

Adam Deng, Shi-Cheng Liu, Ya-Da Zhu · 0 citations
#machine learning Preprint Oct 2026

Ranking and Partitioning of Attention Heads for Sparse and Continual Reinforcement Learning Fine-tuning

We study sparse and continual reinforcement learning fine-tuning of language models through attention-head ranking and partitioning. For sparse learning, we rank attention heads by gradient norm to identify a small subset of heads that can be trained effectively. For continual learning, we partition heads across tasks...

Rui-Zhe Shi, Zai-Wen Yang, Qiu-Bo Wang et al. · 0 citations
#machine learning Preprint Oct 2026

PocketTSLM: Deploying an On-Device Soft-Prompt Time-Series Language Model for Privacy-Preserving Smartphone Reasoning

Consumer phones and wearables increasingly collect physiological time-series data, yet most mobile systems treat these streams as summaries or text rather than as temporal evidence. This representation is poorly matched to personal health systems that need to retrieve, route, or summarize sensor evidence while preservi...

Max Rosenblattl, Nikhil Krishnaswamy, Leon Nissen et al. · 0 citations
#machine learning Preprint Oct 2026

Can Capability Enhancement be Localized?

Transformer-based language models have shown strong capabilities across many tasks and domains. However, prior work shows that, for a given task, not all attention heads matter equally. Specifically, they suggest that many \textit{redundant} heads can be safely removed with little-to-no effect, while a small set of \te...

Anh Tuấn Nguyễn, Anna Bair, Zico Kolter · 0 citations
#machine learning Preprint Oct 2026

Resisting Harmful Instructions through Training-Free Weight Editing

Instruction-following language models can comply with requests that conflict with intended behaviors and safety alignment. We introduce a training-free weight-editing approach for reducing compliance with targeted undesirable instructions. Our method needs only forward passes on a small set of prompt pairs per target b...

Ali Ebrahimpour-Boroojeny, Xiao-Cong Yang, Samuel Schapiro et al. · 0 citations
#machine learning Preprint Oct 2026

Predicting Live Birth After Fresh Embryo Transfer in IVF/ICSI: Differential ICM/Trophectoderm Morphology Encoding and a Hybrid Attention--Logistic Regression Ensemble

Predicting live birth before fresh embryo transfer could support shared decision-making in IVF/ICSI, but routine clinical data are small, heterogeneous, and contain pathway-dependent variables prone to outcome leakage. We aimed to construct a curated single-centre IVF/ICSI dataset and develop a prediction model suited...

Abdul Raheem Shahzad, Jin-Xin Chen, Ming Zhang et al. · 0 citations

From tech blogs

See all →
MIT News · Artificial Intelligence Oct 23, 2026

Q&A: How AI came to model the “messy” human mind

Assistant Professor Benjamin Lindquist studies how engineers deliberately injected randomness into early AI, giving rise to deep-learning models that sometimes hallucinate.

Microsoft Research Blog Oct 22, 2026

Flint-interactive: A language for creating interactive data visualizations

Flint-interactive gives AI agents a language for creating interactive visualizations. Click a chart, filter data, highlight related content, and explore beyond the view the author originally chose. The post Flint-interactive: A language for creating interactive data visualizations appeared first on Microsoft Research.

We use cookies to run the site and, with your consent, for analytics and to show ads. See our Cookie Policy.