Examining Attribution Reliability Through Cutting-Speed Classification as a Benchmark Task: Comparing 1D-CNN, LSTM, and Fully Connected Networks on 17-4PH Milling Force Signals

Abstract

Deep learning has achieved high accuracy in machining condition monitoring, yet accuracy alone cannot reveal whether a model relies on physically meaningful evidence or on dataset shortcuts. The difficulty is that common tasks such as tool-wear and surface-roughness estimation lack a known ground-truth basis for discrimination, so attribution outputs cannot be validated. This study therefore constructs a benchmark task—three-class cutting-speed classification (Vc = 60, 80, 100 m/min)—in which, for a fixed tool diameter, the spindle-speed-dependent spectral content is an a priori known correct basis, enabling the reliability of explainable AI (XAI) methods to be examined under controlled conditions; the task is deliberately simple and serves as a validation framework for attribution methods rather than as a demanding monitoring problem. Cutting force signals were acquired from 3 × 3 full-factorial milling experiments on 17-4PH stainless steel using a Kistler 9170A rotating dynamometer; the torque Mz and resultant force Fa were segmented by sliding windows into 2970 single-channel samples (15 records × 99 windows × 2 channels), each represented by 50 time segments × 6 amplitude-independent statistics. A 1D-CNN, an LSTM, and a fully connected network (NN) were evaluated under five-fold StratifiedGroupKFold cross-validation. The 1D-CNN achieved 93.97% accuracy with roughly one-seventh of the parameters (NN 90.24%, LSTM 89.12%), the smallest fold-to-fold standard deviation, and 84.5% on the most difficult fold; as a single random seed was used, performance conclusions are descriptive only. Holding out whole cutting conditions reduced the mean accuracy to 83–86% (one feed level: 54–57%), so the accuracies describe generalization across records of the studied conditions, not to unseen conditions. Fold-wise application of LRP, LIME, and SHAP showed that the 1D-CNN and NN distributed importance across the entire signal (normalized entropy ≥ 0.97), whereas for the LSTM the gradient × input attribution used as the LRP implementation placed 9.4 times the uniform share on the final time segment, consistently across all five folds, a concentration that SHAP (expected gradients referenced to background training samples) and the perturbation-based LIME did not reproduce. Gradient-free occlusion and resampling reproduced it, and retraining showed that it follows the last input position while removing the final segments did not reduce accuracy; it therefore reflects positional sensitivity rather than reliance on the sequence end. In this run, cross-fold Spearman correlations of LIME rankings lay within −0.05 to 0.05, versus 0.20–0.70 for LRP and SHAP; because exploratory recalculations with alternative aggregation choices gave markedly different LIME correlations, this is reported as a run-specific observation rather than as a property of LIME. All three architectures relied predominantly on frequency-domain features; because the tooth-passing frequencies (159.2–265.3 Hz) exceed the 100 Hz Nyquist frequency of the 200 Hz sampling rate yet lie below the 300 Hz low-pass cutoff of the signal conditioner, they are aliased into the analyzed band, and the dominant-frequency feature reflects the spindle rotation frequency and aliased tooth-passing components, both proportional to spindle speed and hence consistent with the physical definition of cutting speed. The contributions are: a controlled validation framework that cross-validates attribution credibility through importance concentration, inter-method agreement, and cross-fold reproducibility; and the observation that the end-of-sequence concentration of the LSTM, revealed by gradient × input but not by SHAP or LIME, reflects positional sensitivity rather than the basis of classification, highlighting the need to verify attribution concentration with independent analyses rather than relying on a single attribution method.