Skip to content

Category

small language model

3,656 papers

#small language model Preprint Aug 2026

Distilling Black-Box Machine Learning into a Small, Self-Explaining Language Model for Learning Analytics

A two-stage fine-tuning pipeline is proposed that distills a fitted black-box estimator and its post hoc interpretation into a small, open-weight large language model (LLM) that returns an individual-level estimate and explains in natural language that predicts and explains offline on a commodity laptop, so student rec...

Chenguang Pan, Airui Meng, Youmi Suk · 0 citations
#small language model Preprint Aug 2026

COEC: Calibrated Orthogonal-Equivalence Compensation for Structured Pruning of Large Language Models

COEC (Calibrated Orthogonal-Equivalence Compensation), a training-free compensation framework that applies alternating left and right orthogonal rotations to the retained weight, improves perplexity on every model and zero-shot accuracy in most settings over existing compensation methods.

Peiqi Yu, Nam Ling, Wei Wang et al. · 1 citation · ⚡1
#small language model Preprint Aug 2026

BF1: A Causal Dyadic Sparse-Attention Retrofit for Efficient Long-Context Transformers

The results establish BF1 as a reproducible sparse operator and selective retrofit primitive with real long-context systems value, and evaluates numerical correctness, selected-interaction scaling, kernel performance, partial-model inference, and matched next-token language modeling.

Hina Dixit · 0 citations
#small language model Preprint Aug 2026

Jokes Aside: Measuring the Semantic Distance of Double Meanings

Large language models have significantly enriched the toolkit for computational humor research, particularly in the automated generation of jokes and puns. A key innovation, contextual embedding vectors, offers new opportunities to revisit and refine earlier hypotheses. Notably, Petrovic and Matthews (2013) proposed a...

Fabio De Ponte · 0 citations
#small language model Preprint Aug 2026

Vibe Coding and Web Application Security: A Twin-Prompt Study

This work studies six functionally distinct web applications, each generated in two prompt variants that are identical except for an appended security-requirements section: a baseline (A) and a security-aware (B) variant.

Darko Andročec · 0 citations
#small language model Preprint Aug 2026

Interaction Effects Between Learner Characteristics and Dialogue Format in TTS Dialogue-Based Lessons

The results suggest that dialogue format should be selected according to learner characteristics in TTS dialogue-based lessons, with a significant interaction between learner characteristics and dialogue format for ARCS-based motivation.

Fumie Watanabe, Tota Suko, Takashi Ishida et al. · 0 citations
#small language model Preprint Aug 2026

Minimax Optimality of Score-Entropy Discrete Diffusion

This work establishes a minimax lower bound under the score-entropy loss, and proposes an MLE-based thresholding estimator that matches this lower bound up to constant and polylogarithmic factors that depend on neighboring density ratios.

Chol-Kyoon Cho, Yuchen Wu · 2 citations
#small language model Open access Aug 2026

Identify, Locate, Link: End-to-End Key-Value Extraction from Document Images

SmolDocling, a compact 256M-parameter vision-language model (VLM), is fine-tune to perform end-to-end key-value extraction directly from document images, jointly solving identification, localization, and association in a single pass without OCR preprocessing.

A. Gurbuz, Ahmed S. Nassar, Christoph Auer et al. · 0 citations
#small language model Open access Aug 2026

GRASSP: RNA language model–enhanced graph attention with adaptive gating for RNA–small molecule binding site prediction

GRASSP provides a competitive framework for integrating pretrained RNA representations with spatial structural context while reducing reliance on additional handcrafted structural annotations, and is demonstrated to outperform state-of-the-art baselines.

Thi Lan Nguyen, N. Le · 0 citations
#small language model Open access Aug 2026

Co-designed yoga nidra targeting anxiety in autistic children: A mixed methods feasibility study.

The feasibility of yoga nidra as a complementary intervention for autistic children is supported and directions for future research are suggested, including larger trials and further co-design with the autistic community.

T. Loftus, Shu H. Yau, Sophia Soares et al. · 1 citation

ResonatorLM: Causal Resonant Field Mixing for Efficient Long-Context Language Modeling

Contemporary language models are dominated by the transformer architecture, which leverages self-attention mechanisms to enable more efficient, parallelized training across a wide set of documents and corpora. This has allowed transformers to effectively model data across a wide range of modalities and contexts. Howeve...

Archie Chaudhury · 0 citations

From tech blogs

See all →

We use cookies to run the site and, with your consent, for analytics and to show ads. See our Cookie Policy.