A transfer learning-based pronunciation assessment method for low-resource French: a study on Mandarin L1 learners

Abstract

This paper proposes a pronunciation assessment system that uses transfer learning with the multilingual XLSR-53 model to aid low-resource French pronunciation assessment for Mandarin L1 speakers. The paper’s methodology is based on a two-stage transfer learning approach involving cross-lingual fine-tuning on English data and progressive unfreezing on French data, along with a multi-level scoring system. Evaluations were conducted on a self-assembled corpus consisting of 50 hours of data from 200 Mandarin L1 speakers. The results showed that the proposed system achieved a Pearson Correlation Coefficient (PCC) of 0.78, reflecting an improvement of 21.9% over baseline systems without cross-lingual fine-tuning. Additionally, it achieved a PCC of 0.69 when only 10% of the training data is used. Error analysis indicates that the pronunciation difficulties of Chinese learners include front rounded vowels (/y/→/i/, 34.2%) and the uvular fricative (/ʁ/→/l/, 28.7%).