Skip to content

Category

small language model

813 papers

#small language model Preprint Aug 2026

Label-Free Foundational Model Selection for Medical Image Classification under Distribution Shift via Pseudo Label Discrepancy

This work proposes a label-free selection criterion built on SUDO, a framework for evaluating clinical AI systems without ground-truth annotations, and shows that AURCC can be used to rank a variety of vision-language models on chest X-ray classification across three inter-hospital shift scenarios, under zero-shot and MLP-probe regimes.

Juan Iñaki Larrea, L. Mansilla, Enzo Ferrante · 0 citations
#small language model Preprint Aug 2026

Distinct dynamics of conceptual and referential disruptions in human reading and large language model processing

Linguistic meaning is grounded in conceptual content, from which reference to particular entities emerges as words enter discourse. To examine the processing dynamics associated with these two dimensions of meaning, we selectively disrupted conceptual or referential information in short narratives and traced the resulting effects in human self-paced reading and in the predictive and representational processing of large language models. In human reading, conceptual disruptions produced a strong but localized processing cost, emerging immediately after the distorted word, reaching an early maximum, and then declining rapidly. Referential disruptions produced weaker effects, which decreased more gradually across subsequent words, and were more strongly modulated by sentence boundaries. In the language model, both disruptions emerged immediately at the manipulated word. Contextual model surprisal showed a pattern closely paralleling human reading: conceptual disruption produced a larger, more locally concentrated effect that decayed rapidly, whereas referential disruption produced a smaller and more gradual downstream effect. Output-layer representations showed a different pattern: referential disruption produced a larger initial displacement, while both distortions were subsequently characterized by power-law decay. Together, these results provide convergent evidence for distinguishable processing dynamics of two types of meaning: conceptual information imposes a more locally concentrated integration cost, whereas referential information engages a more distributed process of maintaining discourse-level identity.

Rui He, N. Altay, W. Hinzen · 0 citations
#small language model Preprint Aug 2026

MoganBert-TR: A Turkish Encoder Foundation Model Trained from Scratch with a CLM-to-MLM Curriculum

MoganBert-TR, a 149M-parameter Turkish encoder foundation model trained from scratch on a language-specifically filtered corpus, together with an embedding model derived from it (MoganBert-Embed) that ranks first among student models on the MTEB(Turkish) overall average and outperforms all compared Turkish tokenizers on compression and fertility across two independent test sets.

Furkan Yilmaz, Habibe Aleyna Tasdemir, Muhammed Faruk Gozay · 0 citations
#small language model Preprint Aug 2026

Vulnerable Code Search: Transferable Attack for Code Language Models

This paper introduces a programming language-agnostic, transferable, adversarial attack that exploits this CLM vulnerability and demonstrates that this attack, even when computed using smaller code embedding models, is highly effective and transferable to larger, closed-source embedding models.

Kaicheng Wang, Liyan Huang, Jesse Thomason et al. · 0 citations
#small language model Preprint Aug 2026

Retrieve, Match, Escalate: Accurate and Scalable Product Linking with VLM-Distilled Cross-Encoders and Agentic VLMs

This work presents a production retrieve-then-match cascade that spends computation in proportion to difficulty: retrieval surfaces plausible matches, a lightweight text cross-encoder auto-resolves the high-confidence majority, and an agentic multimodal vision-language model settles the ambiguous remainder.

Jian Wang, Steven Xu, Sanjyot Thete et al. · 0 citations
#small language model Preprint Aug 2026

Belief Cascades Drive Persuasion in LLM Agent Networks

This work introduces a controlled testbed for studying how goal-directed persuaders shift elicited stances in networks of LLM agents grounded in real-world ego-network topologies, and argues for evaluating multi-agent persuasion as a trajectory- and exposure-level process.

Haoyi Qiu, Genglin Liu, P. Venkit et al. · 0 citations
#small language model Preprint Aug 2026

The Von-Neumann State-Space Transformer for neural decoding

A von-Neumann inspired hypothesis of efficient computation as an alternative for neural decoding, a memory-augmented Transformer whose feed-forward block is a low-rank instruction bank: a shared base operator plus a small set of learned low-rank instructions, from which a per-token code synthesizes the weight matrix actually used at that token.

Morteza Sarafyazd · 0 citations
#small language model Preprint Aug 2026

SHIFT-LLM: Distribution Shift Correction in Depth-Pruned LLMs

SHIFT-LLM, a training-free post-pruning correction framework that inserts a Linear Residual Adapter at each pruning site, consistently recovers accuracy lost to depth pruning across most configurations, achieving gains up to +15.7 points on Llama-3.1-8B-Instruct.

Ali Bahri, Hang Li, Hongliang Li et al. · 0 citations
#small language model Open access Aug 2026

Estimated Prevalence of Developmental Language Disorder Risk in West Virginia Schools.

PURPOSE Developmental language disorder (DLD) is linked to long-term academic and social difficulties. Despite these lasting impacts, state-level prevalence estimates are limited. This study estimated prevalence among early-elementary students in West Virginia who may be at risk for DLD and examined variation by grade, locale, and assessment instrument. METHOD A cross-sectional screening battery was administered to 252 students in Grades 1-2 across six Title I schools. Children with Kaufman Brief Intelligence Test-Second Edition Matrices standard scores ≤ 70 were excluded prior to analysis. A dual-criterion definition of DLD risk was used: ≤ 80 on the Clinical Evaluation of Language Fundamentals-Fifth Edition (CELF-5) core composite and ≤ 92 on the Test of Narrative Language-Second Edition (TNL-2). Descriptive statistics and mixed-effects logistic regression (random intercept for school) tested effects of grade, gender, and locale. RESULTS Using the dual-criterion definition, 72 of 252 students (28.6%) met criteria for being at risk for DLD. Adjusted models showed higher odds of DLD risk in town versus urban schools (odds ratio = 3.06, 95% confidence interval [1.31, 7.15], p = .010), whereas grade and gender were not significant predictors after accounting for school-level clustering. Instrument-specific classification rates diverged substantially, with the TNL-2 identifying a higher proportion of students relative to the CELF-5. CONCLUSIONS Although based on a relatively small yet representative sample of West Virginia students, prevalence estimates substantially exceed commonly cited large-scale population estimates. Results underscore the influence of instrument choice on suspected prevalence and the need for locally calibrated screening protocols and targeted service planning.

Megan Israelsen-Augenstein, Michelle Moore, Tracy Toman et al. · 0 citations
#small language model Preprint Aug 2026

Bolt-on, Verifiable Provenance for LLM-Powered Data Processing

BLIP is presented, a bolt-on framework for efficiently inferring a small-sized verifiable provenance for any LLM-powered data processing task, with any LLM, and eight strategies, each guaranteed to find a minimal verifiable provenance are introduced.

Yiming Lin, Sepanta Zeighami, Aditya G. Parameswaran · 0 citations
#small language model Preprint Aug 2026

Query Expansion Is More Than Generation: Improving Dense Retrieval through Better Integration

This work introduces AnchorQE, a training-free method that separately encodes the original query and its expansion before interpolating them, and shows that AnchorQE improves retrieval effectiveness by up to 12.89% when compared to widely-used expansion-only or text-level concatenation baselines across TREC-DL, LoTTE, and BEIR.

Sixia Sun, Mihai Surdeanu · 0 citations
#small language model Preprint Aug 2026

SMART: MLLM-guided Temporal Alignment for Unifying Sign Language Recognition and Spotting

This work proposes SMART, an MLLM-guided temporal alignment framework for joint sign recognition and spotting that incorporates CSFormer, a CSLR-guided spotting module that injects recognition-derived gloss evidence into a boundary-aware spotting network.

Eunjee Choi, J. Sung, Seongwhan Cho et al. · 0 citations

From tech blogs

See all →

We use cookies to run the site and, with your consent, for analytics and to show ads. See our Cookie Policy.