Large language models (LLMs) are typically pretrained in high precision but increasingly deployed with low-precision post-training quantization (PTQ). Recent studies have shown that using weight averaging during pretraining can improve PTQ performance compared with learning-rate decay, suggesting that it might provide...
Han Wang, Tianqi Shen, Zong-Lin Liu et al.· 0 citations
It is shown that persistent sycophancy reduction does not guarantee stronger direct refusal, while identifying recovery under user pressure as a distinct, conditional benefit of training intervention.
A family of chunk-level SAEs that encode mean-pooled activations over chunks, each a contiguous span of tokens: Mean-Chunk reconstructs the observed chunk, Cross-Chunk predicts an independently processed neighbor, and Joint-Chunk combines both targets are introduced.
Xu Wang, Yi-Fan Yang, Ting-Ting Yu et al.· 0 citations
This paper proposesNeighbor Integrated Feature Selection (NIFS), a plug-and-play strategy that leverages representation similarity to improve feature selection for steering and demonstrates consistent performance gains over conventional top-$k$ selection.
Yutang Liu, Xu Wang, Di-Fan Zou· 1 citation
We use cookies to run the site and, with your consent, for analytics and to show ads.
See our Cookie Policy.