Skip to content

Category

small language model

3,629 papers

#computer vision Preprint Sep 2026

Investigating Temporal Motion Features for Pose-to-Text Indian Sign Language Translation

We investigate the effect of pretrained T5 model scale and explicit motion features on pose-to-text Indian Sign Language Translation (SLT) for the WSLP 2026 Shared Task. Pose sequences are projected into the embedding space of T5 through a lightweight pose encoder, with the complete model fine-tuned to generate English...

Manav Dhamecha, Praveen Kumar Chandaliya, Pruthwik Mishra · 0 citations
#natural language process... Preprint Sep 2026

SAS: Simple Attention Sparsification via End-to-End Optimization of Context Ranking

Across reasoning, long-context understanding, and agentic tasks, SAS consistently outperforms trainable sparse attention baselines across attention budgets, with especially large gains under tight budgets, demonstrating more effective context ranking for downstream tasks.

Zhi-Wei Li, Lei Zhu, Hao Gu et al. · 1 citation
#natural language process... Preprint Sep 2026

SWARM: A Multilingual Human-Annotated Dataset for Russian Propaganda Detection in Search Engine Results

SWARM (Search-Web documents Annotated for Russian propaganda, Multilingual), a dataset of 2,183 search engine results across nine languages and diverse web domains, each annotated by trained coders for whether it supports a recurring Russian propaganda narrative is introduced.

Manuel Tonneau, A. Dubey, Farhan Shaikh et al. · 0 citations
#natural language process... Preprint Sep 2026

Zipbench: Low-Cost Framework for Compressing Comprehensive Benchmarks of Large Language Models

This work presents ZipBench, a simple and low-cost BCM with theoretical error and rank-consistency guarantees that reduces the cost of both LLM evaluation and compact benchmark construction, lowering the barrier to broad LLM research for compute-constrained researchers.

Zhong-Zhan Huang, Jun-Xin Li, Guo-Ming Ling et al. · 0 citations
#natural language process... Preprint Sep 2026

SynthSentry: Detecting Synthetic Data Contamination in Language Model Training Data

This work introduces SynthSentry, a corpus-level, model-agnostic contamination signal requiring no access to the generating model, no generation history, and no synthetic labels, and frames screening as a data-curation defense rather than a post-hoc diagnosis and releases the scoring toolkit.

Praveen Kumar Myakala, Ravichandra Namburi, S. Jayaramu et al. · 0 citations
#machine learning Preprint Sep 2026

ExpertHTR: Unified Handwritten Text Recognition with Multi-Task Learning and Sparse Mixture-of-Experts

Experiments on seven heterogeneous handwriting benchmarks show that complementary supervision consistently improves training with page transcription alone, while joint multi-source training provides further gains on most datasets, and the proposed sparse expert model further improves the dense baseline on six of the se...

Nam Hoai Dang, H. Nguyen, Quang Huu Hieu et al. · 0 citations
#artificial intelligence Preprint Sep 2026

Tasks over Application Manuals: Revealing Gaps in Long-Horizon Procedural Reasoning for Language Models

This paper introduces Tasks over Application Manuals (TAM), a benchmark for evaluating long-horizon procedural reasoning and shows that current benchmarks may overestimate LLM reasoning ability and miss a key challenge: reliably following long, rule-based procedures.

Utkarsh Soni, S. Murtaza, Yi-Fan Nie et al. · 0 citations
#artificial intelligence Preprint Sep 2026

LLM-Enhanced Dual-Branch Learning for Large-Scale Multi-Label Text Classification

DualMLC is proposed, a dual-branch framework that processes the same document through an autoregressive decoder-only language model and a bidirectional encoder, achieving state-of-the-art results on three widely used large-scale multi-label text classification benchmarks.

Hui Ye, Jing Zhang, Xiu-Long Yang et al. · 0 citations
#artificial intelligence Preprint Sep 2026

Evaluating Context Segmentation in Locally Deployable SLMs for Cybersecurity CTF Tasks

This work introduces context segmentation, a two-level agentic framework that divides complex exploitation tasks into manageable, contextually isolated sub-problems and demonstrates that for the E4B model, the strategy acts as an intelligent search, achieving competitive rewards with superior token efficiency compared...

Sebastiano Nordio, Michele Lotto · 0 citations
#artificial intelligence Review Sep 2026

ORQA: An Occupation-Realistic Question and Answer Framework for LLM Professional Knowledge

We present ORQA, a method for testing occupation-level knowledge in large language models. Prior methods either map abstract LLM skills to occupations via task definitions or utilize expert knowledge which is difficult to obtain at scale and expensive. ORQA complements both of these methods by connecting O*NET occupati...

Shreyas R. Krishnan, Serina Chang, Abhishek Nagaraj · 0 citations
#artificial intelligence Preprint Sep 2026

Breaking the Token Ceiling: Distilling Smaller, Stronger Byte Models

Small models are made more capable through distillation from a larger one that shares their tokenization scheme. However, do distilled byte and token models behave similarly in terms of scaling trends as compute and data increases? To enable this comparison, we introduce two variants to efficiently convert token logits...

Kalyani Marathe, Artidoro Pagnoni, Tomasz Limisiewicz et al. · 0 citations

From tech blogs

See all →

We use cookies to run the site and, with your consent, for analytics and to show ads. See our Cookie Policy.