A century of psychology has found that the trait words people use to describe one another vary, but the relational structure among those traits, which ones go together and which oppose, is strikingly consistent across raters and cultures. We test whether the LLM (Qwen 2.5-7B-Instruct) reproduces this structure in its i...
Yi-Lin Geng, Omri Abend, Eduard H. Hovy et al.· 0 citations
This work systematically investigate the effect of anchor selection by evaluating 22 different anchors on the Arena-Hard-v2.0 dataset, finding that the choice of anchor is critical: a poor anchor can dramatically reduce correlation with human rankings.
Shachar Don-Yehiya, Asaf Yehudai, Leshem Choshen et al.· Annual Meeting of the Associ...· 6 citations· ⚡1
Harnessing naturally occurring feedback from user interactions offers a promising learning signal for Large Language Models (LLMs). However, recent studies suggest this feedback is inherently noisy and difficult to leverage effectively. We challenge this conception by demonstrating that user feedback is a highly action...