InvPT applies semantic-preserving transformations to the corpus and combines masked language modeling with multi-positive supervised contrastive learning that treats all augmentations of the same source function as positives, mixing self-contrast pairs with invariant-contrast pairs for positives of varying difficulty.
Yifeng He, Yun-Di Xu, Christopher Castro Gaw Gonzalo et al.· Proceedings of the 2nd ACM S...· 0 citations
This work shows that observation contract compliance is an emergent, regression-prone capability: it is neither guaranteed by general tool-use ability nor consistently improved by larger or newer models.
Ji-Cheng Wang, Yifeng He, Zi-Li Wang et al.· arXiv.org· 1 citation
An end-to-end system for large-scale online draft co-training that achieves strong scaling at 256K tokens with significant memory savings over prior work, and the PP transport incurs modest overhead.
This work introduces Farseer, a novel and refined scaling law offering enhanced predictive accuracy across scales, and provides new insights into optimal compute allocation, better reflecting the nuanced demands of modern LLM training.
Houyi Li, Wen-Zheng Zheng, Qiufeng Wang et al.· Neural Information Processin...· 4 citations· ⚡1
A novel Continual Audio-Visual Segmentation (CAVS) task, aiming to continuously segment new classes guided by audio, and a Collision-based Multi-modal Rehearsal (CMR) framework, designed to address challenges of multi-modal semantic drift and co-occurrence confusion.
Yuyang Hong, Qi Yang, Tao Zhang et al.· arXiv.org· 3 citations· ⚡1
We use cookies to run the site and, with your consent, for analytics and to show ads.
See our Cookie Policy.
