Skip to content

Category

small language model

3,629 papers

#natural language process... Preprint Aug 2026

Large Language Models Systematically Favor Popular Options: Evidence and Mitigation Across MCQs

A lightweight inference-time correction that estimates and removes a popularity prior from model predictions is proposed, which requires no fine-tuning, is label-free at test time, and adds negligible computational cost.

Abdelrahman Abdallah, Mohammed Ali, Bhawna Piryani et al. · 0 citations
#natural language process... Preprint Aug 2026

RouteSparse: Input-Conditional Pattern Routing for Budgeted Long-Context Prefilling

RouteSparse is introduced, which routes each head and prompt segment among a small library of GPU-efficient sparse patterns, andlations confirm that input-conditional routing, hardware profiling, and selective dense fallback each contribute to the quality--latency tradeoff.

Chao Zhang, Yi Ji, Zi-Yan Zhang et al. · 1 citation
#artificial intelligence Preprint Aug 2026

A Unifying Perspective on Language Model Representations: From Filler-Role Structure to Mechanistic Interpretability

This work proposes using Tensor Product Representations (TPRs) as a unifying hypothesis, and shows that TPRs can unify several prior interpretability methods: additive analogies, linear probing, sparse autoencoders, and activation patching.

Enshang Zhang, R. Thomas McCoy · 0 citations
#machine learning Preprint Aug 2026

TopoCompress: Long Context Compression via Graph-Wired Semantic Trajectories

TopoCompress is introduced, a training-free and model-agnostic framework that compresses long contexts by selecting coherent semantic spans by selecting coherent semantic spans and achieves performance comparable to the strongest baseline while using a 4x smaller compression budget.

Daniel Agyei Asante, Yang Li · 1 citation
#artificial intelligence Preprint Aug 2026

Evaluating Tiny Recursive Models Across Training for Code Generation

This work compares a ~28M-parameter autoregressive Tiny Recursive Model (TRM-AR) on natural-language-to-Python code generation against parameter-matched and depth-matched controls, tracking fit and generation across 40 epochs and three seeds.

Anjani Sirivella, Aanisha Newaz, Glaucia Melo · 0 citations
#machine learning Preprint Aug 2026

Brain-Language-Action (BLA) Models: Language-Conditioned EEG for Robotics Control

A proof-of-concept Brain-Language-Action (BLA) model for drone control using motor-imagery EEG from the BCI Competition IV 2a dataset is developed, providing an initial demonstration that language conditioning can expand the effective control range of EEG-based robotic interfaces without requiring a corresponding incre...

Alexandr Plashchinsky · 0 citations
#machine learning Preprint Aug 2026

Deploying DeepSeek 175B Locally on a Single Consumer-Grade RTX 4060 Laptop with 32GB RAM for 200k-Scale Protein-Ligand Virtual Screening

This work validates the engineering feasibility of running industrial-scale trillion-parameter LLM-driven biomedical computing tasks on consumer hardware, establishing a new low-barrier paradigm for AI-powered early stage drug discovery.

Rui-Ya Xiao, Yi-Li Xu · 0 citations
#artificial intelligence Preprint Aug 2026

CoMPASS: Collaborative Molecular Property Prediction via Adaptive Small-Large Model Synergy

CoMPASS is presented, a retrieval-calibrated framework for small-large model collaboration that retains a graph attention network as the predictive anchor, retrieves locally relevant training molecules, provides attention-grounded evidence to an LLM, and converts its proposal into a bounded correction through an agreem...

Wen-Tao Li, Jiang-Jie Qiu, Yi-Jun Li et al. · 0 citations
#machine learning Preprint Aug 2026

Context Staircase: Signature-Aligned Dynamics of Token Embeddings under Small Initialization

A dynamic explanation of how data statistics and architecture jointly shape token embeddings in language models is provided, and an implicit bias in the space of data statistics is revealed: training proceeds from simpler, low-order statistical relations toward increasingly complex, context-dependent ones.

Jun-Jie Yao, Liangkai Hang, Zhi-qin John Xu · 0 citations
#machine learning Preprint Aug 2026

HalluPrism: When Multimodal Uncertainty Should Diagnose, Not Decide

HalluPrism, a behavioral diagnostic that re-runs an answer after visual degradation, blank-image replacement, and grounding or relation checks is proposed, a behavioral diagnostic that separates failure diagnosis from abstention scoring.

A. Prakash, Sourish Dasgupta, Tanmoy Chakraborty · 1 citation

From tech blogs

See all →

We use cookies to run the site and, with your consent, for analytics and to show ads. See our Cookie Policy.