Two Emojis of Difference: What Multilingual Affective Generation Benchmarks Actually Measure
We audit a multilingual affective generation benchmark eight instruction-tuned LLMs producing emoji summaries for 17,100 Bangla, English and Hindi sentences, with 6,960 human judgements and find its headline conclusions to be artefacts of the measurement instrument rather than properties of the systems. Treating annota...
Fardeen Sadab, Adib Sakhawat
· 0 citations