Skip to content

QUARTET: Quad-branch cross-Attention and Random-walk Traces for Enhancing Transformers on Relational Graphs

Sep 2026 · 0 citations · 53 references
Computer Science

TL;DR

QUARTET is an expressive graph transformer architecture that applies full self-attention on local subgraphs while enriching global context through cross-attention branches, and consistently matches or outperforms the current state-of-the-art graph transformer baselines (HGT and RelGT).

Abstract

Relational Deep Learning (RDL) models multi-table databases as heterogeneous temporal graphs, and graph transformers currently achieve state-of-the-art performance on benchmarks like RelBench. However, the current leading model, RelGT, suffers from two key limitations: its random local sampler yields loosely connected subgraphs that hinder message passing, and its global attention module relies on a single, seed-feature-based memory that ignores broader macro-level dynamics. To overcome these limitations, we introduce QUARTET, an expressive graph transformer architecture that applies full self-attention on local subgraphs while enriching global context through cross-attention branches. Specifically, QUARTET employs a Causal Random Walk (CRW) sampler based on recency-truncated Personalized PageRank (PPR) to extract compact, hub-robust, and densely connected local subgraphs without temporal leakage. Concurrently, a quad-branch cross-attention module integrates global context from four complementary perspectives: seed feature, seed topology, temporal dynamics, and collaborative dynamics. Across the RelBench v1 classification tasks, QUARTET consistently matches or outperforms the current state-of-the-art graph transformer baselines (HGT and RelGT). Ablation studies confirm that the CRW sampler significantly enriches local neighborhood quality, while the global branches provide essential, task-specific predictive gains.

View source

Similar papers

#artificial intelligence Preprint Aug 2026

DASC: Decay-Aware State Compression for Hybrid Linear-Attention Serving

Decay-Aware State Compression (DASC), which derives retention horizons from model weights, selects long-horizon state units, and packs them into a ragged state checkpoint layout to integrate efficiently with tensor-parallel inference engines.

Yanzhi Yu, Ping-Wei Sun, Jian-Chao Tan et al. · 2 citations
#artificial intelligence Preprint Aug 2026

Beyond Flat Netlist: Hierarchical Graph Representation Learning for Scalable Analysis of Sequential Circuits

DeepSeq3 is introduced, a novel hierarchical framework that abstracts circuits into a two-level representation: fine-grained combinational subgraphs partitioned by flip-flops (FFs) and a high-level Super-Node Graph (SNG) that models the register-transfer structure.

Jing-Yi Zhou, Zhengyuan Shi, Jiaying Zhu et al. · 0 citations
#machine learning Preprint Sep 2026

CommunityKV: Efficient Long-Context Decoding via Graph Partitioning

CommunityKV, a framework that formulates sparse attention as a community detection problem, is introduced, a framework that formulates sparse attention as a community detection problem and is evaluated on Qwen3 and Llama-3.1 models across three long-context benchmarks.

J. McKenna, Anastasios Alexandridis, Nathan Susanj et al. · 0 citations
Preprint Oct 2026

Graph-to-Table Interfaces Control Relational Generalisation

Graph-to-table systems combine three design choices: an encoder that maps relational evidence to feature rows, a context constructor that selects labelled examples, and a tabular decoder. We investigate whether these components can be evaluated independently through matched comparisons that change one component while h...

Alessandro Cagiano, Lorenzo Molfetta, Gianluca Moro et al. · 0 citations
Preprint Sep 2026

Long-Range Indirect Control-Flow Prediction in Stripped Binaries via Dual Virtual Hubs and Multi-Task Graph Learning

Recovering indirect control-flow (ICF) edges is fundamental to binary security analysis, yet existing methods struggle with long-range dependencies, isolate different ICF types, and are often evaluated under protocols vulnerable to label noise and data leakage. We present ICFlowNet, a unified framework for long-range I...

Kun Liu, Zheng-Ming Ding, Chen-Ke Luo et al. · 0 citations

Related blog posts

MIT News · Artificial Intelligence Sep 29, 2026

Who we become when we talk to machines

Professor Sherry Turkle’s new book, “Artificial Intimacy,” offers a withering critique of chatbots and the antisocial dynamics she believes they encourage.

We use cookies to run the site and, with your consent, for analytics and to show ads. See our Cookie Policy.