Pixel-based language models process rendered text without relying on a fixed subword vocabulary, offering a promising approach to noisy and multilingual inputs. However, their practical value depends on whether they can be reliably adapted to downstream tasks. To this end, we systematically examine this question throug...
Hao-Wen Tang, Wei Xia, Rao-Yuan Zhao et al.· 0 citations
On-policy distillation can transfer the reasoning capabilities of a strong teacher to a smaller student. However, distillation from a stronger teacher model faces two mismatches: its solutions may be difficult for a smaller student to imitate, and its token-level probabilities may be incompatible with, or even unavaila...
Run-Sheng Chen, Yihong Liu, Rao-Yuan Zhao et al.· 0 citations
LLM annotation at scale outperforms human-supervised classifiers at roughly one-tenth the cost, for both a closed-source and an open-weight LLM, and the advantage is robust under soft-label evaluation.
Ahmad Dawar Hakimi, Lea Hirlimann, Isabelle Augenstein et al.· 0 citations
We use cookies to run the site and, with your consent, for analytics and to show ads.
See our Cookie Policy.