This work presents LAION-BVD, a large-scale open video dataset for multimodal learning, which contains 1.3B platform-specific video URLs collected from CommonCrawl, and explores video frames as an alternative source of image-text data by extracting scene-changing frames.
Andreas Hochlehnert, Marianna Nezhurina, M. Cherti et al.· 0 citations
On surrogate benchmarks ScAn-Bench-LLM and ScAn-Bench-VLM, the first systematic evaluation of both data acquisition and extrapolation methodology for scaling analysis across different data modalities is performed.
Artin Sermaxhaj, Nastaran Alipour, Donat Sinani et al.· 0 citations
This work shows that Kimi Delta Attention (KDA) can realize 2D rotations by combining a single delta-rule transformation with a second reflection supplied by its channel-wise gate, and characterize the expressivity of CKDA and prove that every orthogonal diagonal-plus-rank-one matrix is exactly a CKDA transition matrix...
This work introduces MultiSynt/MT, an open synthetic parallel corpus with approximately 4.8 trillion target-language tokens across 36 languages, produced by translating 100 billion high-quality Nemotron-CC tokens with Tower+ and OPUS-MT/HPLT-MT systems, and releases the corpus to support controlled research on multilin...
Maximilian Idahl, Jörg Tiedemann, S. Pyysalo et al.· arXiv.org· 0 citations
This study establishes a first baseline and scaling procedure for the development of future OpenEuroLLM models, and characterize the dependence of loss on model capacity and dataset size, evaluating recently proposed scaling forms that explicitly model their interaction.
Niccolò Ajroldi, Diana-Alexandra Onutu, Haider Al-Tahan et al.· 1 citation
We use cookies to run the site and, with your consent, for analytics and to show ads.
See our Cookie Policy.
