Skip to content

Category

natural language processing

9,568 papers

#machine learning Preprint Open access Oct 2026

Who Sets the Pace of Judge Hacking? Judge Capability versus the Policy-Judge Gap

When a policy is trained with GRPO against a frozen, reference-free LLM judge, the judge accepts more of its answers, but in a post-hoc analysis we detect no gain in held-out greedy accuracy: in all 12 (policy, judge) cells, the seed-mean gain never exceeded 0.04. A gold-reward control with the same recipe raised the 1...

Haosen Wang · 0 citations
#machine learning Preprint Oct 2026

Preserving Updates in Low-Bit Recurrent State Quantization

Linear attention and recurrent models compress past tokens into a fixed-size state that is updated at every step. However, storing these states in high precision incurs substantial memory and bandwidth costs at large batch sizes, motivating low-bit state quantization. Unlike weights and key--value caches, a recurrent s...

Hao-Tian Li, Li-Ran Dong, Kai-Cheng Xiao et al. · 0 citations
#machine learning Preprint Open access Oct 2026

Generalizable Model Editing Could Be More Than Direct Generalization

A central desideratum in model editing is generalization to complex tasks, which prior work formalizes under a direct generalization paradigm: treating and evaluating LLM behavior as an input-output mapping and expecting a factual update to alter outputs on reasoning tasks. Yet despite substantial empirical effort, the...

Xiaotian Ye, Chunyao Yang, Xiaohan Wang et al. · 0 citations
#machine learning Preprint Oct 2026

The Gradient That Top-k Distillation Never Computes

It is proved that it removes the visibility of tail mass, not the freedom to move it, in the top-k distillation pipeline: keep the top-k probabilities, renormalize, and minimize a divergence on what remains.

Rasul Khanbayov, Hasan Kurban · 0 citations
#machine learning Preprint Oct 2026

Compute-Optimal Scaling Laws for Spiking Language Models

Spiking neural networks target energy-efficient language modeling through sparse, binary activations, but the efficiency-accuracy tradeoff of spiking language models is uncharacterized at scale. We present the first Chinchilla-style IsoFLOP scaling study of a spiking language model, sweeping SpikeGPT-family models (RWK...

Daniel J. Saunders, Benjamin Ren, Catherine He · 0 citations
#machine learning Preprint Open access Oct 2026

HodgePhaseNet: Discrete Hodge Decomposition of Semantic Flows in Transformer Attention

WavePhaseNet introduced frequency-resolved semantic representations, QuantumPhaseNet introduced gauge-covariant phase connections, and ModularPhaseNet made a finite cyclic phase channel and its cycle holonomy computable in an ordinary Transformer. These constructions are complementary, yet they do not by themselves sep...

Kiyotaka Kasubuchi, Kazuo Fukiya · 0 citations
#machine learning Preprint Open access Oct 2026

HiPAD: A Unified Gait, Audio-Visual, and Personality Fusion Network for Depression Detection, with a Hierarchical Personality Ablation and the Calibration Limits of Domain-Adapted LLMs

We present HiPAD, a single fusion network submitted to all six sub-tracks of the MPDD-AVG 2026 depression-detection challenge, covering the Elder and Young cohorts and three modality sets (gait, audio-visual, and both), each combined with personality descriptions. The network encodes each available modality with a shar...

Karol Chlasta, Piotr Struzik · 0 citations
#machine learning Preprint Oct 2026

Reward-Aligned Distillation

Reward-Aligned Distillation (RAD), a bilevel framework that weights distillation signals based on their predicted effect on the student's post-update reward, supports reward-guided weighting of teacher supervision across model families and tasks.

Yilan Chen, Zian Su, Manish Prajapat et al. · 0 citations
#machine learning Preprint Open access Oct 2026

Query-Adaptive Neuron Steering for LLMs

Effectively eliciting the capabilities of large language models (LLMs) at inference time remains an important challenge. Existing studies address this challenge through context engineering, which shapes user queries, or fixed steering, which directly intervenes in model computation. However, predefined contexts and fix...

Yanxi Pan, Jingwen Wu, Yan Ma et al. · 0 citations
#artificial intelligence Preprint Open access Oct 2026

Measuring Concept Content in Text from LLM Activations: ESG Evidence from Concept Vectors and Linear Probes

Recent work shows that Large Language Models (LLMs) encode more information in their activations than they show in their responses. We ask whether the information contained in activations could be used as a source of measurement for the presence of concepts. We compare two algorithms for extraction of concepts, i.e., R...

Luc Hazenoot, Zhaochun Ren, Amirhossein Zohrehvand · 0 citations
#artificial intelligence Preprint Open access Oct 2026

LLM-OSDA: An Optimal-Stopping Dynamic Auction for Native Advertising in Multi-Turn LLM Conversations

LLM-native advertising embeds sponsored content directly into model-generated responses, shifting the unit of sale from a fixed slot to a moment within an evolving conversation. Existing LLM ad-auction mechanisms primarily operate within a single response, settling the winner but not the timing. The extension is nontri...

Yan Fang, Jialin Chen, Chun Gan et al. · 0 citations
#artificial intelligence Preprint Open access Oct 2026

Activation Steering with Jungian Cognitive Functions

Controlling combinations of personality-related tendencies is important for adapting large language models to diverse interaction needs. But successful single-trait steering does not establish how to construct interventions for joint targets, or whether existing trait directions adequately represent them. We introduce...

Liu Zai, Yumeng Wang, Junchen Fu et al. · 0 citations

From tech blogs

See all →
Microsoft Research Blog Oct 22, 2026

Flint-interactive: A language for creating interactive data visualizations

Flint-interactive gives AI agents a language for creating interactive visualizations. Click a chart, filter data, highlight related content, and explore beyond the view the author originally chose. The post Flint-interactive: A language for creating interactive data visualizations appeared first on Microsoft Research.

We use cookies to run the site and, with your consent, for analytics and to show ads. See our Cookie Policy.