Curriculum-GraphLLM: Joint Optimization of Architectures, Structures and Texts for Denoised Graph Neural Architecture Search.

Abstract

Discovering optimal graph neural network (GNN) architectures for various tasks is both labor-intensive and time-consuming. To reduce human effort, graph neural architecture search (GNAS) has recently been utilized to automatically identify effective GNN architectures for specific tasks, achieving competitive or even superior performance compared to manually designed architectures. However, existing GNAS methods fail to identify optimal architectures in the presence of structural and semantic noise, where structural noise refers to missing or redundant edges within the graph structure, and semantic noise denotes inaccurate node representations derived from ambiguous node features such as textual vagueness or semantic ambiguity. In this paper, we address this problem for the first time via theoretical analyses and empirical evaluations. We discover that existing differentiable GNAS methods typically select architectures based on task-relevant information hidden in the graph, being highly sensitive to structural and semantic noise, which results in suboptimal selection of GNN architectures under noise. To handle the structural and semantic noise, we propose Curriculum-GraphLLM, a novel graphLLM framework for joint optimization of architectures, structures, and texts for denoised graph neural architecture search. The core idea is to jointly optimize GNN architectures, graph structures, and textual semantics as a unified denoising process during architecture search. Specifically, we first develop a dynamic topology updating mechanism to adaptively adjust the graph structure. Then, we jointly optimize the GNN architecture and graph structure through a curriculum-based iterative updating approach. To further deal with semantic noise on text-attributed graphs (TAGs), we introduce LLMs as an auxiliary text modeling module to refine textual semantics and guide the co-optimization of text representations, graph structures, and GNN architectures. We conduct extensive experiments to show that our proposed Curriculum-GraphLLM achieves consistently competitive or superior performance compared with existing baselines, especially under structural and semantic noise.