Evaluation of Speaker Verification Models in Arabic Datasets

Abstract

This study presents a systematic benchmark of six state-of-the-art, pre-trained Speaker-Verification (SV) models, ECAPA-TDNN, ECAPA2, WavLM-Large-SV, Pyannote-Embedding, TitaNet, and WeSpeaker, on Arabic speech, a language for which modern SV models remain largely unevaluated. All models were trained on predominantly English corpora (e.g., VoxCeleb) and are assessed here on two Arabic datasets, the broadcast corpus QASR-SV (3,728 valid speakers, ~531 hours) and a newly collected in-house Arabic dataset for attendance systems (678 valid speakers, ~50 hours), with the English VoxCeleb1 test set as a reference baseline. Performance is measured primarily by Equal Error Rate (EER), complemented by False Acceptance and False Rejection Rates (FAR/FRR). We further quantify the effect of audio duration (1–3 s vs. 3–6 s segments) and of background noise (music, ambient noise, and competing speech) injected from MUSAN at a 10 dB signal-to-noise ratio. WeSpeaker achieved the lowest EER on Arabic speech (3.42% on the in-house dataset and 4.23% on QASR-SV at 3–6 s). Across all models, 3–6 s segments reduced EER substantially relative to 1–3 s, and competing background speech was the most damaging interference type. The results give practical guidance, including accuracy-versus-model-size trade-offs, in deploying speaker verification in Arabic attendance scenarios, and the openly released evaluation pipeline serves as a reproducible Arabic SV benchmark.