GraphDecisionBench is introduced, a model-independent benchmark that combines structural task profiles, matched graph-text input contrasts and heuristic-proposal controls to diagnose graph decision performance and illustrates the benchmark's central distinctions.
Abstract
Large language models (LLMs) are increasingly explored for graph understanding and decision-making, while System One models such as Jev select directly from supplied options. However, the capabilities of System One models on graph-related tasks remain unclear. We introduce GraphDecisionBench, a model-independent benchmark that combines structural task profiles, matched graph-text input contrasts and heuristic-proposal controls to diagnose graph decision performance. We evaluate Jev and related choice-based models alongside language-model baselines, covering fourteen model-interface configurations. Jev's results illustrate the benchmark's central distinctions: accurate adjacency recognition does not guarantee broader structural correctness, joint graph-text input does not consistently improve prediction, and feasible construction does not establish high solution quality. Its task contracts, candidate interfaces and scoring rules support comparison across native selectors and language-model adapters. Code and aggregate results are available at https://github.com/VictorYXL/GraphDecisionBench.
These results should be interpreted as a comparison of graph-language interaction designs rather than a fully input-matched ablation, but they suggest that exposing task-relevant structural information is an important factor for urban spatial QA.
Rachid Oucheikh, A. Mansourian· ISPRS Annals of the Photogra...· 0 citations
The Graph Theory Agent (GTA), which pairs a preference-trained representation selector with plan-and-decompose scaffolding around a frozen executor LLM, is proposed, which lifts Phi-4 from 53.5% to 69.1% on the benchmark's easy split and from 33.0% to 41.5% on its hard split.
Zi-Xiang Xu, Yan-Bo Wang, Chenxi Wang et al.· 4 citations· ⚡1
Jev answers MMLU's calculation-heavy questions more accurately than other MMLU questions (94% vs. 91%), whereas both open models, and all three on C-Eval, find them harder.
Tobias Deußer, Rajkumar Ramamurthy, L. Sparrenberg et al.· 33 citations· ⚡7
Can a pretrained graph model replace training and tuning a separate predictor for each dataset? Answering this requires evaluating prediction quality alongside computational cost. We present NodeGround, a node classification benchmark that puts graph foundation models (GFMs) and dataset-specific supervised learning und...
Jin-Mo Lee, Dooho Lee, Minho Jeong et al.· 0 citations
The results show that JSON, although widely used in practice, is not always the most efficient structured output format, and output formats and constraint methods should be selected according to task shape, latency budget, model capability, and tolerance for downstream repair.
Chao Zhang· Applied and Computational En...· 0 citations
Large language models (LLMs) have demonstrated strong performance on coding and reasoning benchmarks; however, their ability to solve graph-structured machine learning problems remains largely unexplored. In particular, no benchmark currently evaluates whether LLMs can autonomously solve end-to-end Graph Neural Network...
M. Hossen, Tasneem Selim, Gurur Gamgam et al.· Proceedings of the 2026 Conf...· 0 citations