SpectraDL: A Historical Issue-Driven, Test Specification-Assisted Transfer Testing Approach for Deep Learning Frameworks via LLMs

Abstract

Deep learning (DL) frameworks provide diverse fundamental algorithmic units as operators, which are critical infrastructure for constructing various intelligent software. Since mainstream frameworks widely adopt the open-source development paradigm, bugs may recur across operators and even across frameworks. Recent studies leverage large language models (LLMs) and historical issues to generate cross-framework test cases. However, existing approaches still suffer from two limitations. First, their test cases have low fault detection capability because they mainly reuse inputs or contexts from historical issues without considering the underlying root causes. Second, an effective mechanism for determining the appropriate transfer scope within a target framework is lacking. To overcome these limitations, we propose SpectraDL, a historical issue-driven, test specification-assisted transfer testing approach for DL frameworks. SpectraDL first extracts rigorous test specifications for each operator from official documentation, and then extracts and transforms historical issues and associated pull requests into structured fault representations (i.e., bug patterns). SpectraDL uses a dual retrieval mechanism based on semantic intent and structural input-space features to transfer these bug patterns to related operators across frameworks. Experiments on four mainstream DL frameworks show that SpectraDL detected 125 previously unknown bugs, 107 of which developers confirmed. The results confirm that SpectraDL delivers a promising transfer testing approach for DL frameworks.