Skip to content
Book Open access

Curvature Graph Mergeability: Predicting Model Compatibility from Per-Layer Hessian Geometry

Nov 2026 · Proceedings of the 35th ACM International Conference on Information and Knowledge Management · 0 citations · 26 references

TL;DR

Curvature Graph Mergeability is introduced, a scoring framework that predicts the interpolation barrier from only cheap, per-layer Hessian curvature features at each endpoint, and proves that the interpolation barrier is upper-bounded by a multilinear function of per-layer curvature mismatch.

Abstract

Merging independently trained neural networks --- via weight averaging, model soups, or task-vector arithmetic --- is increasingly central to large-scale model reuse, yet the success of any merge hinges on whether two parameter vectors occupy the same loss basin, a property currently certified only by expensive multi-point interpolation barriers. We introduce Curvature Graph Mergeability (CGM), a scoring framework that predicts the interpolation barrier from only cheap, per-layer Hessian curvature features at each endpoint. CGM extracts two scalar curvature statistics per layer using Hutchinson's trace estimator and the power method, organizes them into a chain-structured factor graph whose latent compatibility states encode qualitative regimes of layer alignment, and computes a mergeability score from the negative variational free energy of the chain. For deep linear networks under balanced factorization, we prove that the interpolation barrier is upper-bounded by a multilinear function of per-layer curvature mismatch, and we identify a diagonal-dominance condition under which the score is monotone in mismatch; the bound is stated as an implicit joint bound on the per-layer mismatch vector, with the regime of validity made explicit. Across four meta-training architectures and four held-out OOD architectures (ResNet-110, ViT-B/ImageNet, WRN-28-10, 3-layer LSTM), CGM attains Spearman ?=0.86 on leave-models-out splits and ?=0.81 pooled OOD, outperforming zero-order distance, CKA, Fisher overlap, REPAIR, midpoint-barrier, and architecture-agnostic curvature-MLP baselines at Holm--Bonferroni-corrected p<0.01. It is approached only by Git Re-Basin + barrier (?=0.90), which costs ~14,500× more per pair; CGM closes the gap to permutation-based merging at 35× amortized speedup over direct 11-point barrier when screening 50 candidates, with negligible memory overhead. Per-layer posterior marginals additionally pinpoint specific layers that act as merge bottlenecks.

Read PDF

We use cookies to run the site and, with your consent, for analytics and to show ads. See our Cookie Policy.