It is found that embedding-based alignment metrics do not reliably indicate whether alignment will improve or degrade downstream performance, and that the compatibility between alignment and downstream objectives should be considered when designing/evaluating alignment methods.
Yana Veitsman, Yihong Liu, Hinrich Schütze· arXiv.org· 0 citations
The Last Translation Benchmark is introduced, a collection of human-authored and peer-reviewed examples that break leading machine translation models and a new evaluation approach: each example comes with handcrafted verification rules describing concrete failure cases on that example, therefore allowing reliable and a...
Vilém Zouhar, Niyati Bafna, Mukund Choudhary et al.· 0 citations
Findings show that successful MoE-block restoration does not necessarily imply localization to a single expert, as it is shown that successful MoE-block restoration does not necessarily imply localization to a single expert.
Yuetian Lu, Ali Modarressi, Yihong Liu et al.· arXiv.org· 0 citations
This work proposes a pipeline for automatically generating step-by-step linguistic reasoning traces from Universal Dependencies treebanks, dictionaries, and grammar-rule banks and shows that linguistic reasoning traces are most effective as inference-time guidance in ICL, which substantially improve translation perform...
