This paper derives a closed-form optimal linear operator from a small calibration set to reconstruct the activation discrepancy introduced by the pruned layers, and shows that this solution corresponds to the unconstrained optimum of the alignment objective, whereas LinearPatch is restricted to a fixed basis symmetric...
Vincent-Daniel Yun, Junhyuk Jo, Sai Praneeth Karimireddy et al.· arXiv.org· 3 citations
It is argued that it is difficult to predict the alignment state of a model after training by inspecting the training data alone, which highlights the importance of comprehensive post-training alignment evaluations.
Yavuz Faruk Bakman, D. Yaldiz, Baris Askin et al.· 1 citation
Results show that HeteroFold enables efficient cross-family KV reuse without receiver prefill, and achieves the best cache-transfer performance on all four long-context benchmarks and most short-context settings.
OptR is proposed, an output-aware rotation method that minimizes post-W_O$ attention-output error and applies an attention-equivalent key reparameterization to reduce large channel-wise offsets without changing the softmax distribution.
Speech Generation Speaker Poisoning (SGSP), a task that seeks to prevent a model from synthesizing targeted speaker identities while maintaining performance on all other speakers, is formulated and an evaluation framework based on AUC analysis and a proposed metric, Forget Set Similarity (FSSIM).
Thanathai Lertpetchpun, Thanapat Trachu, Sai Praneeth Karimireddy et al.· 0 citations
Self-Anchored Consensus (SAC), a fully decentralized filter-and-refine protocol in which agents iteratively exchange responses, locally evaluate and filter unreliable messages, and refine their own outputs, is proposed.
Haejoon Lee, Vincent Yun, Hyeonho Oh et al.· arXiv.org· 3 citations
We use cookies to run the site and, with your consent, for analytics and to show ads.
See our Cookie Policy.