Aug 2026· Proceedings of the 32nd ACM SIGKDD Conference on Knowledge Discovery and Data Mining V.2· pp. 8930-8941· 0 citations· 17 references
TL;DR
This paper develops a comprehensive benchmark construction framework and constructs three diverse benchmark datasets and proposes a unified evaluation framework that assesses both linguistic quality and factual consistency, providing a comprehensive approach to evaluating generated summaries for content fidelity and accuracy.
Abstract
This paper introduces a novel task, graph textual summarization, which aims to generate natural language descriptions that capture both the semantic content and structural characteristics of graph data based on large language models (LLMs). Unlike traditional summarization tasks for text, images, or videos, summarizing graph data presents unique challenges due to its abstract and non-intuitive nature. We develop a comprehensive benchmark construction framework and construct three diverse benchmark datasets, including text-attributed graphs (TAGs), knowledge graphs (KGs), and heterogeneous attributed graphs (HAGs). To facilitate empirical studies, we deliver three representative benchmark datasets: SurveyGraphSum, consisting of 7835 citation graphs from survey papers across 10 major domains and 56 subdomains; BookGraphSum, containing 6175 knowledge graphs extracted from 186 books; and E-comGraphSum, with 912 heterogeneous graphs from real-world e-commerce networks, each representing a fraud group with 14 node types, 19 relation types, and 9 numerical attributes related to user behavior and interactions. In addition, we propose a unified evaluation framework that assesses both linguistic quality and factual consistency, providing a comprehensive approach to evaluating generated summaries for content fidelity and accuracy. By formalizing the task, releasing benchmark datasets, and proposing an evaluation framework, we aim to advance research in graph textual summarization and promote methods that produce accurate, informative, and human-readable summaries for complex graphs.
Text-attributed graphs (TAGs) are an important graph data form that combine relational structure with rich node text. However, real-world TAGs are often imperfect, with quality issues arising from text, structure, and labels, and typically manifesting as sparsity, noise, and imbalance. These dimensions define nine representative degradation scenarios that can substantially affect TAG learning. Although prior studies have explored specific mitigation strategies, existing evidence remains fragmented across degradation types, datasets, tasks, and model families, leaving TAG robustness insufficiently understood. To address this gap, we present OpenRTAG, a robustness benchmark for text-attributed graph learning. OpenRTAG organizes TAG quality issues into a unified 3 * 3 taxonomy and supports standardized evaluation across nine TAG datasets and three downstream tasks. It systematically evaluates scenario validity and model sensitivity, compares traditional GNNs, LLM-GNNs, and a representative GFM, investigates the effectiveness, efficiency, and robustness of scenario-matched baselines, and further examines model behavior under composite degradation scenarios. OpenRTAG provides a standardized testbed for understanding robustness in TAG learning under realistic low-quality settings.
Yuze Dai, Zhihan Zhang, Yan Zhao et al.· 0 citations
Hypergraphs effectively model higher-order groupwise relationships beyond pairwise interactions, while pretrained language models (PLMs) and large language models (LLMs) provide rich semantic understanding from textual attributes. However, research on combining language models with hypergraph learning remains limited due to the lack of public text-attributed hypergraph benchmarks. To address this limitation, we present TAHB (Text-Attributed Hypergraph Benchmark), the first public benchmark integrating hypergraph structures and raw textual attributes. TAHB contains 10 real-world datasets from four domains - e-commerce, academia, movies, and politics networks - enabling systematic evaluation of text-aware hypergraph representation learning. Experimental results show that TAHB preserves key structural properties of real-world hypergraphs and consistently reproduces performance tendencies observed in existing benchmarks. Furthermore, experiments under both LLM-as-Enhancer and LLM-as-Predictor settings demonstrate that LLM-enhanced textual semantics improve hypergraph learning performance, while structural and textual information jointly provide the best setting for LLM-based prediction. Our benchmark provides a foundation for future research at the intersection of hypergraph learning and language models.
D. Y. Kang, JungHyun Kim, Juhyun Jeon et al.· 0 citations
The rapid expansion of scientific publications has significantly increased the complexity of traditional literature review processes. While recent advances in AI-assisted screening reduce manual effort, they fail to provide an actionable organization of findings beyond thematic clustering. We propose an integrated pipeline that transforms raw bibliographic data into queryable Knowledge Graphs (KGs), combining: (1) automated collection via the OpenAlex API, (2) LLM-assisted screening, (3) hierarchical semantic clustering using state-of-the-art embeddings (Qwen3-Embedding-4B), and (4) multi-relational KG construction in Neo4j with GraphRAG. We validate this methodology on a corpus of 50 K articles on Artificial Intelligence from the computer science literature. Our hierarchical clustering identifies 7 macro-clusters and 117 microclusters with Fused Gromov-Wasserstein (FGW) coherence. The resulting KG integrates 34,200 nodes across 9 entity types, revealing temporal evolution patterns, cross-institutional collaborations, and foundational knowledge pillars through citation analysis.
Le Van Olivier, Dardouillet Pierre, Boutalbi Karima· Annual International Compute...· 0 citations
This survey presents a systematic review of 121 references spanning 2002 to 2026, tracing the evolution of TextRank-based approaches into hybrid LLM pipelines and advancing three qualified arguments.
Ahmed J. Jabur, Asmaa Abdul Azeez Dakhil, Israa Saad Mohammed et al.· Iraqi Journal for Computers...· 0 citations
Assessing scientific novelty is inherently complex, requiring evaluation of both a paper's internal structure and its contribution within the broader research landscape. Existing large language model (LLM) approaches often rely on surface-level similarity or citation retrieval, overlooking the integration between a paper's content and its contextual grounding in related literature. To address this gap, we introduce SciNova, a benchmark containing 3,063 papers from both ICLR and NeurIPS, with full content, bibliographies, and peer review scores for novelty prediction. Building on this benchmark, we propose GraphMind, a model that jointly processes micro- and macro-level structures for novelty prediction and rationale generation. It represents each paper as a hierarchical graph that captures its claims, methods, and experiments (micro-level), while its related paper graphs encode both cited and semantically similar works (macro-level). Experiments show that by incorporating this bi-level graph, GraphMind significantly outperforms baseline LLMs in both novelty score prediction and rationale generation.
Italo Luis da Silva, Hanqi Yan, Lin Gui et al.· Proceedings of the 32nd ACM...· 1 citation
A five-stage semi-automatic framework for constructing complex graph reasoning benchmarks that serves as a challenging and diagnostic benchmark for graph reasoning and provides empirical guidance for future enhancement methods is proposed.
Fali Wang, Ali Al-Lawati, Iliyas Bektas et al.· 0 citations