GigaAM Multilingual: Foundation Model for Underrepresented Languages
This work addresses the challenge of building robust foundation models for underrepresented Central Asian languages by presenting GigaAM Multilingual, a Conformer encoder pre-trained on 2M hours of audio using a HuBERT-style objective, and introduces a cluster-level data balancing strategy during pre-training and a domain-aware sampling method during fine-tuning to mitigate head-language dominance.