Skip to content

Graph Representational Learning: When Does More Expressivity Hurt Generalization?

May 2025 · arXiv.org · Vol abs/2505.11298 · 4 citations · ⚡ 1 influential · 87 references
Computer Science

TL;DR

A family of premetrics that capture different degrees of structural similarity between graphs are introduced and relate these similarities to generalization, and consequently, the performance of expressive GNNs are related.

Abstract

Graph Neural Networks (GNNs) are powerful tools for learning on structured data, yet the relationship between their expressivity and predictive performance remains unclear. We introduce a family of premetrics that capture different degrees of structural similarity between graphs and relate these similarities to generalization, and consequently, the performance of expressive GNNs. By considering a setting where graph labels are correlated with structural features, we derive generalization bounds that depend on the distance between training and test graphs, model complexity, and training set size. These bounds reveal that more expressive GNNs may generalize worse unless their increased complexity is balanced by a sufficiently large training set or reduced distance between training and test graphs. Our findings relate expressivity and generalization, offering theoretical insights supported by empirical results.

View source

Similar papers

#artificial intelligence Preprint Sep 2026

HARMONIA: Interpretable Graph Learning through Mixtures of Neural Bases

HARMONIA achieves stronger explanation recovery than existing interpretable graph baselines while maintaining competitive predictive performance and scaling to graphs with millions of nodes, and shows that interpretable graph learning can remain both faithful and scalable without sacrificing predictive effectiveness.

Q. D. Bùi, D. Nguyen, A. Dang et al. · 0 citations
#machine learning Preprint Sep 2026

Transferable Graph Metanetworks

A weight space network (or metanetwork) takes the weights of another neural network as input and predicts properties of it. Most prior work trains such models on input networks of one or a few fixed sizes and evaluates them in-distribution. The few attempts at out-of-distribution size generalization remain limited in s...

Yu-Xin Ma, A. Dayan, Yam Eitan et al. · 0 citations
#machine learning Preprint Sep 2026

Message Passing Does More with Less for In-Context Learning on Graphs

Achieving strong performance with graph neural networks (GNNs) typically requires training and hyperparameter tuning for each dataset, incurring repeated costs and effort. Graph in-context learning (ICL) avoids this by using a single pretrained model to predict unknown node labels directly from labeled context nodes. E...

Dooho Lee, Jin-Mo Lee, Minho Jeong et al. · 0 citations
Open access Aug 2026

Enhancing Knowledge Graph Embedding's via Logic Regularized Graph Neural Networks

In (high)-dimensional data, deep learning-based Knowledge Graph Embedding (KGE) models have shown better capacity in learning the true pattern of latent features. However, they are typically used as "black boxes", where statistical correlation is preferred over explicit logical inference. These are geometrically feasib...

Ahmed H. Mohammed · 0 citations
#machine learning Preprint Oct 2026

Graph Additive Models disentangling feature, topology, and interactions

Graph Neural Additive Networks (GNANs) have emerged as effective, interpretable-by-design architectures for graph learning. They achieve transparency by combining per-feature shape functions with simple distance-based reweighting. However, GNANs have three limitations. First, node-wise relevance scores aggregate inform...

C. Álvarez-Rodríguez, Steve Azzolin, Vincenzo Marco De Luca et al. · 0 citations

Related blog posts

We use cookies to run the site and, with your consent, for analytics and to show ads. See our Cookie Policy.