We have 3 of 72 papers
We haven’t gathered this author’s papers yet. Follow them and we’ll fetch their work.
Not the right person? Other researchers publish under this name.
The Last Translation Benchmark is introduced, a collection of human-authored and peer-reviewed examples that break leading machine translation models and a new evaluation approach: each example comes with handcrafted verification rules describing concrete failure cases on that example, therefore allowing reliable and a...
The results indicate that while the general purpose LLMs are better than the machine translation systems, there are systematic issues with the generation and evaluation of prose translation in both languages across systems.
The results emphasize the need for calibrating the LLM judges with a sample with reference-aware evaluation before using them in reference-free setups reliably, and the methodology provides a blueprint for researchers and practitioners in doing such calibration of LLM judges for other tasks.
We use cookies to run the site and, with your consent, for analytics and to show ads. See our Cookie Policy.
