Script Fragmentation and Format: What Drives the English-Bengali Performance Gap in Open LLMs?
Beyond confirming a substantial gap, it is shown that part of it is a measurement artifact: exact-match accuracy conflates correctness with format adherence and because format failure is asymmetric across languages it distorts the apparent gap for some models three to five fold and even reverses its sign for one reason...
Shimanto Bhowmik, Tawsif Tashwar Dipto, Md Sazzad Islam et al.
· 11 citations
· ⚡3