AlignSAE is introduced, a method that aligns SAE features with a predefined ontology through a "pre-train, then post-train"curriculum and achieves disentanglement improvements of $9$--$\infty\times$ over standard SAEs, and further supports multi-hop reasoning and a mechanistic probe of grokking-like generalization dyna...
Minglai Yang, Xin-Yu Guo, Zheng-Liang Shi et al.· arXiv.org· 4 citations
In-context learning (ICL) allows language models to adapt their behavior from instructions and demonstrations in a prompt, but accuracy alone does not reveal how contextual information controls their predictions. We study the acquisition and trainability of ICL with a controlled diagnostic that separates copying a near...
Minglai Yang, Xin-Yu Guo, Ren-Xiong Wang et al.· 0 citations
This work proposes Rubric Dropout, a one-line fix borrowed from neuron dropout that randomly drops a subset of the rubric's criteria before computing the reward, so the policy never optimizes the same rubric twice.
Minglai Yang, Xinyu Guo, Utkarsh Tyagi et al.· 0 citations
We use cookies to run the site and, with your consent, for analytics and to show ads.
See our Cookie Policy.