Skip to content

LoRA-generating hypernetworks for efficient on-device LLM generative personalization

Sep 2026 · 0 citations · 47 references
Computer Science

TL;DR

A novel method for personalizing on-device large language models by training a hypernetwork to map a user's context tokens to a low-rank adaptation well-suited to that user well-suited to that user is presented.

Abstract

On-device large language models (`LLMs'), e.g. running on mobile phones, are ripe for improvement via personalization. The limited compute resources of mobile devices impose limits on model scale and thus model quality, making any realizable quality gains highly impactful. At the same time, their personal nature (i.e., the close coupling to a particular user) means that a given on-device LLM tends to be used in similar, predictable patterns over the course of time. This paper presents a novel method for personalizing on-device LLMs. It trains a hypernetwork to map a user's context tokens to a low-rank adaptation (`LoRA') well-suited to that user. Once the trained common artifacts are deployed to users'devices, each user uses the hypernetwork to synthesize (entirely on device) a personalized LoRA. This approach blends the benefits while avoiding the drawbacks of two existing approaches to LLM customization: in-context learning (`ICL') and parameter-efficient fine-tuning (`PEFT'). Like ICL (and unlike PEFT), the on-device phase of our approach is computationally feasible, requiring only forward passes through neural networks. Like PEFT (and unlike ICL), our approach modifies the `target'base LLM via weights (the LoRA), avoiding negative consequences (e.g. increased latency) associated with extending the input sequence. Our approach is particularly well-suited to the mobile device regime. Apart from the on-device compute and latency benefits mentioned, it also requires minimal additional storage, as internally its architecture partly leverages the same LLM weights as belong to the target LLM to be personalized. We demonstrate the benefits of LoRA-generating hypernetworks on several representative personalization datasets, comparing against baselines like ICL and PEFT. Of note, our personalization experiments focus on more challenging and less studied long-form text generation tasks.

View source

Similar papers

#artificial intelligence Preprint Oct 2026

Fine-Tuning a 3B-Parameter LLM on a Smartphone: Characterizing Sustained Training

Multi-billion-parameter LLMs now run on phones for inference, and training them on the device would personalize them without user data leaving the phone. Prior work has measured individual training steps of such models on phones, but not complete training runs, and not whether adapters trained on the device improve per...

Andrew Geyko, Marius Mosbach, André Brinkmann · 0 citations
#machine learning Preprint Sep 2026

Extending FunctionGemma for Practical On-Device Mobile Function Calling

This work extends FunctionGemma 270M-it to practical Android workflows by introducing MOBILEACTIONSEXTENDED, a synthetic, schema-validated dataset of ~9,500 conversations covering fifteen device-control categories, highlighting compact local function calling as a practical path towards low-latency and privacy-preservin...

Ali Rezagholizadeh, Soheila Samiee · 0 citations
#artificial intelligence Preprint Sep 2026

Samsone: A Family of Open Small Audio Language Models for On-Device Inference

The core model, Samsone-134M, establishes a new state-of-the-art for its size class across multiple benchmarks, and the scaling laws of SALMs are explored by introducing Samsone-99M and Samsone-356M, a family of SALMs designed for edge computing.

Piotr Masztalski, Michal K. Grzeszczyk, Olaf Sikorski · 0 citations
Book Open access Sep 2026

GradSup: Gradient Superposition for Personalised and Scalable LLM Recommendation

Experiments conducted on movie, book and electronics benchmark datasets demonstrate that GradSup outperforms iterative fine-tuning to provide scalable and personalised recommendation that is consistently sustained above the frozen LLM backbone.

Kanishka Dandeniya, C. Dasanayaka, Daswin de Silva et al. · 0 citations
#artificial intelligence Preprint Oct 2026

Internalizer: Portable Context-to-Parameter Mapping for Very Large Language Models

Hypernetworks that map a context directly to a LoRA adapter let a large language model carry that context in its weights, but prior work has demonstrated them only on base models of up to 14 billion parameters. We present the Internalizer, a state-of-the-art, portable Context-to-Parameter Mapping hypernetwork that gene...

Peter Devine, Nick Ryan, Benjamin Sirb et al. · 0 citations

Related blog posts

Microsoft Research Blog Nov 16, 2026

TokenEconomizer: Learning from Experience to Make AI Coding Agents More Efficient

AI coding agents are increasingly help developers fix bugs and build features largely on their own, but every interaction with their models consumes tokens. In each session, an agent sends instructions, tool descriptions, and session history to its model. Most efforts to reduce this cost focus on individual tasks. Yet developers return to the same […] The post TokenEconomizer: Learning from Experience to Make AI Coding Agents More Efficient appeared first on Microsoft Research.

We use cookies to run the site and, with your consent, for analytics and to show ads. See our Cookie Policy.