It is demonstrated that small models are not yet polyglots: while leading systems perform well on high-resource languages, performance drops sharply on low-resource varieties, and comprehension degrades significantly when transferring across non-English language pairs.
Patrícia Schmidtová, Ekaterina Artemova, Seth Aycock et al.· Proceedings of the Eleventh...· 0 citations
This work addresses three challenges: finding conversational events in multilingual speech, evaluating their timing without reliable word-level alignment, and judging responses across languages with IndicFDB, which extends it to ten languages spoken in India with 12,350 samples.
Rajarshi Roy, Shobhit Banga, Jonathan Raiman et al.· 1 citation
This work evaluates state-of-the-art LLMs as pointwise and pairwise judges of conversational success on CANDOR, finding pointwise scoring correlates moderately with human ratings, while pairwise comparison suffers from long transcripts and positional bias.
Maike Zufle, Patrícia Schmidtová, Vilém Zouhar et al.· arXiv.org· 1 citation
We use cookies to run the site and, with your consent, for analytics and to show ads.
See our Cookie Policy.