Bias Evaluation Framework in AI-Powered Resume Classification Using DistilBERT with SHAP Explainability and Automated Fairness Flagging

Abstract

This paper proposes a multi-layer bias audit framework for AI-powered resume screening combining DistilBERT classification, SHAP explainability, automated fairness flagging, and locally-deployed LLaMA 2 interpretation. The framework achieved 74.8% of accuracy, 76.6% of precision, 74.8% of recall, and 74.7% of F1-Score at the resume-level across 24 occupation classes. Fairness evaluation identified 6 occupations failing the Disparate Impact threshold (DI < 0.80), with cross-occupation selection rate parity not met (σ = 0.1845). However, per-class accuracy parity metric shows that this unfairness manifests primarily as under-recall rather than a cross-class false positive leakage. SHAP proxy analysis detected gender and seniority-coded tokens influencing predictions across 9 occupations, with 75.9% of test resumes triggering at least one fairness flag. Results demonstrate that algorithmic bias in resume screening can be systematically detected and surfaced before reaching hiring decisions.