Preprint
Jul 2026
GigaAM Multilingual: Foundation Model for Underrepresented Languages
This work addresses the challenge of building robust foundation models for underrepresented Central Asian languages by presenting GigaAM Multilingual, a Conformer encoder pre-trained on 2M hours of audio using a HuBERT-style objective, and introduces a cluster-level data balancing strategy during pre-training and a domain-aware sampling method during fine-tuning to mitigate head-language dominance.
Andrei Kuzmenko, A. Maximenko, Aleksandr Kutsakov et al.
· 0 citations