FIONA is presented, an offline optimizer that selectively ternarizes weights within a given packing layout based on the estimated effect of ternary conversion on the model's performance, and fits lower-degree replacements under a cumulative accuracy budget, reducing multiplicative depth and bootstrapping.
Abstract
Fully homomorphic encryption (FHE) enables neural network inference directly on encrypted inputs, but it remains orders of magnitude slower than plaintext in- ference. Applying the server's plaintext weights to encrypted activations involves plaintext-ciphertext multiplications (PMult) and accounts for more than half of inference time in recent systems. Ternary quantization can replace these multipli- cations with additions and subtractions, but the savings rarely materialize under packed execution. A single PMult applies a weight group fixed by the packing layout and can be avoided only when all its weights share the same ternary value. Ternarizing all groups, however, largely degrades accuracy. We present FIONA, an offline optimizer that selectively ternarizes weights within a given packing layout based on the estimated effect of ternary conversion on the model's performance. FIONA encourages a shared ternary value within each weight group and retains full-precision weights for sensitive groups, so ternar- ized and full-precision paths coexist within a layer. It then compiles these hybrid operators exactly, applying common scaling factors once to accumulated inputs and reusing sums across outputs. Weight ternarization can also narrow the input ranges of downstream polynomials. FIONA fits lower-degree replacements under a cumulative accuracy budget, reducing multiplicative depth and bootstrapping. On VGG11, ViT, and BERT, FIONA reduces PMult operations by 53.4-79.5% and accelerates end-to-end encrypted inference by 2.38x, 1.68x, and 1.84x, re- spectively, with less than 1% accuracy loss across all three models.
Fully Homomorphic Encryption (FHE) enables computation on encrypted data, preserving privacy throughout analysis. While its privacy is very strong, FHE is much slower to execute than the original computation. In particular, due to the recent success in accelerating its compute, the performance bottleneck shifts to the...
A. W. B. Yudha, Erwin Eko Wahyudi, R. Rajagede et al.· 0 citations
This work introduces Homomorphic Encryption-Aware Training (HEAT), a fine-tuning method that makes the per-nonlinearity iteration counts learnable, enabling them and the model weights to co-adapt during training.
Alessandro Zirilli, Davide Marincione, E. Kornaropoulos et al.· 0 citations
Odin is the first open-source end-to-end GPU CKKS implementation of Llama-3, an FHE inference system that co-designs ciphertext packing and model execution for Llama and uses a feature-major cross-layer layout to unify residual connections and layer interfaces.
Yu-Hang Fan, Yu-Si Chen, Kan-Yu Ye et al.· arXiv.org· 1 citation
This work presents a framework that reformulates HE-aware model design as a constrained neural architecture search problem, where the objective is to identify architectures that are both cryptographically feasible and computationally efficient while preserving task performance.
Reeshav Chowdhury, Anoop Mishra, Deepak Khazanchi et al.· ACM Transactions on Internet...· 0 citations
In recent years, the Cheon–Kim–Kim–Song (CKKS) scheme has gained widespread popularity among practitioners due to its support for approximate arithmetic on encrypted data. However, the decrypted outputs of CKKS inherently contain approximation errors arising from homomorphic operations. If the error term in the ciphert...
Yu Zhan, Zhen Zhao, Yan-Bo Wang et al.· Electronics· 0 citations
It is shown that input DP is orthogonal to model confidentiality and that the local-DP premise required for shuffle amplification cannot hold under correctness-bounded noise, and that the leaked spectra enable fingerprinting, lineage attribution, and improved logit-based extraction, while suppressing them destroys infe...
By studying the computational foundations of language learning, Associate Professor Jacob Andreas builds intelligent systems that can communicate effectively with humans.
Microsoft Research Blog· microsoft.comNov 16, 2026
AI coding agents are increasingly help developers fix bugs and build features largely on their own, but every interaction with their models consumes tokens. In each session, an agent sends instructions, tool descriptions, and session history to its model. Most efforts to reduce this cost focus on individual tasks. Yet developers return to the same […] The post TokenEconomizer: Learning from Experience to Make AI Coding Agents More Efficient appeared first on Microsoft Research.
MIT News · Artificial Intelligence· news.mit.eduNov 13, 2026