Skip to content

Category

small language model

845 papers

#small language model Preprint Aug 2026

COEC: Calibrated Orthogonal-Equivalence Compensation for Structured Pruning of Large Language Models

COEC (Calibrated Orthogonal-Equivalence Compensation), a training-free compensation framework that applies alternating left and right orthogonal rotations to the retained weight, improves perplexity on every model and zero-shot accuracy in most settings over existing compensation methods.

Peiqi Yu, Nam Ling, Wei Wang et al. · 0 citations
#small language model Preprint Aug 2026

BF1: A Causal Dyadic Sparse-Attention Retrofit for Efficient Long-Context Transformers

The results establish BF1 as a reproducible sparse operator and selective retrofit primitive with real long-context systems value, and evaluates numerical correctness, selected-interaction scaling, kernel performance, partial-model inference, and matched next-token language modeling.

Hina Dixit · 0 citations
#small language model Preprint Aug 2026

Jokes Aside: Measuring the Semantic Distance of Double Meanings

Large language models have significantly enriched the toolkit for computational humor research, particularly in the automated generation of jokes and puns. A key innovation, contextual embedding vectors, offers new opportunities to revisit and refine earlier hypotheses. Notably, Petrovic and Matthews (2013) proposed a joke generation model based on the scheme"I like my X like I like my Y, Z"(e.g."I like my ice like I like my dreams, crushed"). They suggested that joke hilarity increases with: a) frequent association of Z with X and Y, b) rarity of Z, c) ambiguity of Z, and d) meaning distance between X and Y. Building on this, Winters et al. (2019) proposed a set of metrics, based on Google Ngrams and Word2Vector. In this work, three out of their five metrics are revisited with word embeddings: obviousness, compatibility, and comparison. Another measure, symmetry, defined as closeness of Z to both X and Y, is introduced here for the first time. Two models were used to collect the embedding vectors (OpenAI text-embedding-3-small and MiniLM all-MiniLM-L6-v2) on three datasets: JokeJudger, Expunations, and rJokes. The last two datasets, Expunations, and rJokes, were expanded by adding paired sentences that captured the ambiguous expression at the core of each joke in its two different meanings. Results revealed that models trained on the proposed metrics performed poorly in predicting humor ratings: on JokeJudger, the best model achieved 57.1% accuracy, below the 61.5% baseline, while performance on Expunations and rJokes was even lower. Nevertheless, the symmetry metric seems consistently associated with higher-rated jokes, suggesting it may capture a necessary -though not sufficient- property of humor.

Fabio De Ponte · 0 citations
#small language model Preprint Aug 2026

Vibe Coding and Web Application Security: A Twin-Prompt Study

This work studies six functionally distinct web applications, each generated in two prompt variants that are identical except for an appended security-requirements section: a baseline (A) and a security-aware (B) variant.

Darko Andročec · 0 citations
#small language model Preprint Aug 2026

Interaction Effects Between Learner Characteristics and Dialogue Format in TTS Dialogue-Based Lessons

The results suggest that dialogue format should be selected according to learner characteristics in TTS dialogue-based lessons, with a significant interaction between learner characteristics and dialogue format for ARCS-based motivation.

Fumie Watanabe, Tota Suko, T. Ishida et al. · 0 citations
#small language model Preprint Aug 2026

Minimax Optimality of Score-Entropy Discrete Diffusion

This work establishes a minimax lower bound under the score-entropy loss, and proposes an MLE-based thresholding estimator that matches this lower bound up to constant and polylogarithmic factors that depend on neighboring density ratios.

Chol-Kyoon Cho, Yuchen Wu · 0 citations
#small language model Preprint Aug 2026

Identify, Locate, Link: End-to-End Key-Value Extraction from Document Images

SmolDocling, a compact 256M-parameter vision-language model (VLM), is fine-tune to perform end-to-end key-value extraction directly from document images, jointly solving identification, localization, and association in a single pass without OCR preprocessing.

A. Gurbuz, A. Nassar, Christoph Auer et al. · 0 citations
#small language model Open access Aug 2026

GRASSP: RNA Language Model-Enhanced Graph Attention with Adaptive Gating for RNA-Small Molecule Binding Site Prediction.

GRASSP provides a competitive framework for integrating pretrained RNA representations with spatial structural context while reducing reliance on additional handcrafted structural annotations, and is demonstrated to outperform state-of-the-art baselines.

Thi Lan Nguyen, N. Le · 0 citations
#small language model Open access Aug 2026

Co-designed yoga nidra targeting anxiety in autistic children: A mixed methods feasibility study.

The feasibility of yoga nidra as a complementary intervention for autistic children is supported and directions for future research are suggested, including larger trials and further co-design with the autistic community.

Tundi Loftus, Shu H Yau, Sophia Soares et al. · 1 citation
#computer vision Preprint Aug 2026

CodeAssay: A Multi-Metric Benchmark with Audited Ground Truth for LLM Code Generation

These findings show that reliable evaluation of LLM-generated code requires validated ground truth, protected tests, and multiple explicitly interpreted measures, and that CodeAssay provides a reproducible basis for evidence-based model evaluation in AI-augmented software development.

Shahbaz Siddeeq, Muhammad Waseem, Umar Subhan Malhi et al. · 0 citations
#machine learning Open access Jun 2026

Unified heterogeneity-aware benchmark of drug synergy prediction: a cross-study analysis of traditional machine learning and graph deep learning models.

The first comprehensive benchmarking framework specifically designed to accommodate inter-dataset heterogeneity is presented, finding that well-designed small datasets can match or even surpass the performance of larger benchmarks, suggesting that different metrics are applicable to different datasets/testing scenarios.

Yingjuan Cheng, Qing Ye, Linlong Jiang et al. · 0 citations

From tech blogs

See all →

We use cookies to run the site and, with your consent, for analytics and to show ads. See our Cookie Policy.