Oct 2026· Proceedings of the Second BabyLM Workshop· 0 citations· 36 references
Computer Science
TL;DR
It is suggested that prosodic structure is a key component of phonemic language modelling in the small-data regime most analogous to infant learning.
Abstract
Human children learn language from continuous, prosodically structured speech long before they can read, yet almost all language models train on orthographic text. Building on previous phonemic BabyLM training (Goriely et al., 2024; arXiv:2410.22906), we ask whether supplying an estimate of the prosody infants exploit improves phonemic language modelling at BabyLM scale. We compare an orthographic baseline to three conditions forming a ladder of phonological richness. All conditions are trained as BERT MLMs at 10M and 100M words. Addressing the ecological validity of existing metrics in the small-data regime, we build a lexical decision task and two semantic organisation tasks. Our new lexical decision task offers improved insights into the distinction between wordlikeness, token familiarity, and lexicality as factors. Triphone phonotactic control and pseudo-log-likelihood scoring shows that these scores reflect phonotactic familiarity rather than wordhood: every model sits near or below chance on an adversarial inverted condition, revealing prior results of this kind as partly artefactual. On our new semantic tasks, a prosodically annotated phoneme stream gives the strongest taxonomic organisation at both the 10M and 100M word scales. Phonemic models with boundary structure are competitive with the orthographic baseline. These findings suggest that prosodic structure is a key component of phonemic language modelling in the small-data regime most analogous to infant learning.
Cross-linguistic effects are a central topic in bilingual first-language acquisition. Artificial learners can help investigate L1-L2 interactions by enabling controlled comparisons across language combinations and learning conditions. Recent work explores this direction by training bilingual language models under devel...
Nikitas Theodoropoulos, Maria Lymperaiou, Giorgos Filandrianos· Proceedings of the Second Ba...· 0 citations
German is a morphologically rich language whose syllable structure is exceptionally well-predicted by the Knuth--Liang hyphenation algorithm. We ask whether phonologically informed tokenization can serve as a competitive target for end-to-end speech recognition. We compare three tokenizer families on the Omnilingual AS...
Christopher Witzl, Tobias Bocklet, Korbinian Riedhammer· Conference on Empirical Meth...· 0 citations
Phoneme-level models transcribe or generate speech as a sequence of phonemes, the smallest sound units that distinguish words. These models enable fine-grained pronunciation control and understanding, yet often fail on input that does not match any single training language, such as speech alternating between two langua...
Comprehending spoken language requires extracting syntactic structure and meaning from a continuous acoustic signal. Psycholinguistic evidence points to a key role for prosody in this process. However, because interactions between prosody and syntax are typically demonstrated using carefully controlled sentence constru...
Cas W. Coopmans, Andrea E. Martin· bioRxiv· 0 citations
Rhyme, defined as the phonological overlap from the primary stressed vowel to the end of a word, is a central component of phonological awareness, a foundational skill in literacy acquisition. While the core principles of rhyming are shared across languages, it remains unclear whether rhyme judgments are made over rime...
Speech foundation models are primarily evaluated with task-level metrics, which do not establish whether language-specific phonological patterns are encoded or faithfully reproduced. This dissertation asks what evidence is required to distinguish recoverable acoustic or distributional information from phonological gene...
S. Barman· The 12th ISCA-SAC Doctoral C...· 0 citations
We use cookies to run the site and, with your consent, for analytics and to show ads.
See our Cookie Policy.