Are Temporal Graph Based Intrusion Detection Results Trustworthy? A Dataset Audit and Evaluation Framework

Abstract

Temporal Graph Neural Networks (TGNNs) have been increasingly applied to network intrusion detection (NID), with some studies reporting accuracy exceeding 99%. This paper argues that such performance can be an artifact of dataset construction flaws rather than genuine model capability. We conduct an empirical audit of two NID datasets and identify three categories of dataset flaws when tabular NID datasets are converted to temporal graphs: node identity leakage, temporal concentration of attack traffic, and class imbalance interacting with graph structure. We further propose an evaluation framework comprising a node identity leakage detection protocol and an attack-aware chronological split strategy. We demonstrate the node identity leakage detection protocol empirically through a controlled experiment across two TGNN architectures - T-GCN (RNN-based) and A3T-GCN2 (attention-based). We also compare the attack flow rate change between a standard dataset split method and our split strategy. Together, this proposed evaluation framework provides a more reliable and rigorous basis for future TGNN-NID research.