Skip to content

Category

small language model

813 papers

#small language model Preprint Aug 2026

DAMP: Decay-Aware Mixed-Precision Recurrent-State Quantization

DAMP uses both quantization-error energy and decay-based persistence to identify high-risk channels during offline calibration and stores these channels at higher precision and the remainder in INT8, the first to study post-training quantization of recurrent states in GDN and KDA based language models.

Tao Zhang, Jian-Chao Tan, Ping-Wei Sun et al. · 0 citations
#natural language process... Preprint Aug 2026

AI Writers Have a Consistent Stylometric Footprint, but AI Editors Do Not

It is shown that AI generation leaves a consistent ``stylometric footprint'': a small subset of features, primarily entropy and lexical diversity, consistently separates AI-generated text from human writing across 8 LLMs and 5 domains, while the remaining features depend heavily on the domain and generator.

Zhengyang Shan, Yukyung Lee, Sophie Hao · 0 citations
#natural language process... Preprint Aug 2026

Speculative Probing: LLM Monitoring at Speculative-Decoding Cost

It is found that the speculative-decoding module in recent LLMs can be repurposed for efficient high-quality classification by appending a trained soft prompt at the end of the target sequence, which can repurpose the speculative-decoding module into a sequence classifier.

Collin Zhang, Tingwei Zhang, Vitaly Shmatikov · 0 citations
#small language model Preprint Aug 2026

Training-Free Temporal Abstraction for General Video Understanding

STITCH is presented, a training-free method that divides a video into semantically meaningful temporal chunks that are computed once per video and reused across tasks, suggesting that reusable temporal abstraction is a promising direction for general video understanding.

Etienne Casanova, S. Brodjian, Pietro Perona · 0 citations
#small language model Preprint Aug 2026

NumBench: Diagnosing Counting Failures in Text-to-Image Models

The Confidence-Weighted Numeric Precision Score (\cwnps), which aggregates three calibrated detectors and discounts uncertain proposals, is proposed for scalable evaluation and developed a process model in which requested instances compete for a finite set of resolvable image regions.

Sandeep Wadhwa, M. Vatsa, Richa Singh et al. · 0 citations
#small language model Preprint Aug 2026

Biologically Inspired Mechanisms for Facilitating Grokking in Multilayer Perceptrons

The results support the common principle that explicit regulation of neuron utilization and effective connectivity can improve the emergence of generalizable internal computation and motivate broader investigation of biologically inspired activity regulation and adaptive sparsification, including in large language models.

Florin Leon · 0 citations
#small language model Preprint Aug 2026

From Documents to Reasoning: A Validated Synthetic Data Pipeline and Semantic-Aware Fine-Tuning for Financial Numerical Reasoning

This work proposes a comprehensive pipeline for improving financial QA systems through high-quality synthetic data generation and fine-tuning of smaller language models (SLMs) using Quantized Low-Rank Adaptation (QLoRA).

Lokendra Birla, Milind Savagaonkar, Visnu Srinivasan et al. · 0 citations
#small language model Preprint Aug 2026

uMOF: A Universal Database, Benchmark, and Machine Learning Interatomic Potentials for Metal-Organic Frameworks

Foundation machine learning interatomic potentials (MLIPs) deliver near-ab-initio accuracy at a fraction of the computational cost, yet their promise for Metal-organic Frameworks (MOFs) remains largely unrealized as large unit cells make first-principles training data expensive to generate, fine-tuned models are scarce, and experimentally grounded benchmarks are scarcer still. We introduce uMOF, a three-part contribution addressing this gap. First, we release the largest and most accurate density functional theory dataset for MOFs to date, computed at the r$^2$SCAN-D4 level of theory across 85524 configurations spanning 19950 unique frameworks and 79 elements, covering empty and gas-loaded structures, geometry optimizations, equations of state, and finite-temperature molecular dynamics. Second, we release a literature-mined benchmark of 3986 verified property values (3146 experimental) extracted from 626 papers by a seven-stage, checkpointed multi-pass large language model pipeline, linked to more than 650 crystallographic information files. Third, we release two universal MLIPs for MOFs, uMOF-MH and uMOF-POLAR, fine-tuned from two architecturally distinct MACE foundation models on the uMOF dataset. On near-equilibrium, ``Tier-1''properties (bulk modulus, phonon-derived heat capacity) the uMOF models perform comparably to existing foundation and fine-tuned baselines. On harder, dynamics-sensitive properties like gas adsorption enthalpies via Widom insertion and adsorption isotherms, the uMOF models outperform every baseline we test, including MOF-specialized gas-capture models trained on datasets up to three orders of magnitude larger, cutting error by more than 80% to within experimental uncertainty. We trace this advantage to the physical diversity of the training data and to level of theory where a small (1.7%) fraction of MD simulations is decisive for MLIP stability.

T. J. Inizan, Prathami Divakar Kamath, A. Elena et al. · 0 citations
#machine learning Preprint Aug 2026

TACIT-Switch: Cost-Aware Model Escalation for LLM Agents from Censored Supervision

This method learns permanent handoff policies from accumulated trajectory evidence and Teacher-Annotated Censored Intervention Times (TACIT) and represents each annotation as an interval-censored observation on a cumulative-risk scale and achieves the highest held-out success among learned policies on both ALFWorld and DABench.

Ji'an Lei, Jian-Hao Huang · 0 citations
#small language model Open access Aug 2026

Evaluating teacher-supervised small medical language models for institution-constrained clinical QA.

For institution-constrained clinical QA, in-domain teacher-supervised fine-tuning was the most reliable evaluated adaptation path, and public-benchmark transfer was not a dependable proxy for local utility in the primary internal comparison.

Gao-Yuan Du, D. Dimitriadis, Xueping Li · 0 citations
#small language model Open access Aug 2026

Balancing privacy and performance: the impact of facial defacing on AI in medical imaging

Summary Background Recent NIH Data Management and Sharing (DMS) policy updates and NIH controlled-access data security requirements have increased attention to facial anonymization and controlled-access handling of shared head imaging data. This is particularly relevant for datasets submitted to or hosted by the Cancer Imaging Archive (TCIA), where NCI Cancer Imaging Program/TCIA implementation practices address imaging data containing potentially reconstructable facial anatomy. While intended to protect patient privacy and strengthen public trust, defacing can distort craniofacial geometry and alter image statistics, potentially compromising the fidelity and reproducibility of artificial intelligence (AI) models trained on such data. Existing studies primarily validate visual anonymization quality, but few have quantified its downstream impact on deep learning-based medical imaging tasks. Understanding this privacy-utility trade-off is crucial for responsible data sharing and compliant AI development. Methods We systematically evaluated three representative defacing algorithms, two invasive (QuickShear and Py-Deface) and one less destructive, facial replacement (mri_reface), across MRI and CT datasets from 600 subjects spanning three institutions. Model performance was assessed on three clinically relevant applications: (1) brain segmentation and Evans ratio biomarker quantification in normal pressure hydrocephalus (NPH) MRI using SLANT and FreeSurfer; (2) representative-slice selection and diagnostic reasoning for brain tumour MRI using vision-language models (VLMs); and (3) automated emergency head CT report generation using a fine-tuned Otter-based vision-language model. Each method’s impact was quantified using Dice similarity, correlation metrics, reasoning accuracy, and natural-language generation scores (BLEU, METEOR, ROUGE, CIDEr). Findings Invasive algorithms caused significant degradation across all tasks. QuickShear reduced mean Dice scores by up to 9% and introduced 14–19% failure rates during quality control, while PyDeface induced smaller but measurable performance losses. mri_reface maintained 100% success without any failures and achieved segmentation, diagnostic, and report-generation accuracy within 3–5% of the original data. Evans ratio distributions remained statistically consistent between mri_reface and original images (p > 0.05), whereas invasive methods introduced broader variance. Across all VLM tasks, mri_reface preserved high correlation with radiologist-selected slices (r = 0.979) and stable report-generation quality (BLEU-4 = 0.11 ± 0.06 vs. 0.12 ± 0.07 for original). Interpretation Facial anonymization introduces a measurable privacy-utility trade-off that must be explicitly considered in the design of AI-ready medical imaging datasets. Invasive defacing compromises geometric and statistical integrity, reducing downstream model accuracy even outside facial regions. Facial replacement anonymization methods, such as mri_reface, effectively reconcile patient privacy with reproducibility, offering a practical path to NIH-compliant open data. Future regulatory and institutional policies should integrate quantitative privacy-utility assessment and mandate transparent reporting of anonymization pipelines to ensure that shared imaging data remain both ethically safe and scientifically valid under emerging digital health frameworks. Funding This work was partially supported by the 10.13039/100000968American Heart Association (Award No. 25IPA1454088), the 10.13039/100000002National Institutes of Health (Award No. 1R03CA286693-01A1 and Award No. 1R01CA291826-01A1), the 10.13039/100000005U.S. Department of Defense (Award No. HT94252510807), and the 10.13039/100000001National Science Foundation (Award No. 2545071).

Yuli Wang, Yuwei Dai, Haoyue Guan et al. · 0 citations

From tech blogs

See all →

We use cookies to run the site and, with your consent, for analytics and to show ads. See our Cookie Policy.