Beyond confirming a substantial gap, it is shown that part of it is a measurement artifact: exact-match accuracy conflates correctness with format adherence and because format failure is asymmetric across languages it distorts the apparent gap for some models three to five fold and even reverses its sign for one reason...
Shimanto Bhowmik, Tawsif Tashwar Dipto, Md Sazzad Islam et al.· 11 citations· ⚡3
This work introduces BANGLAWILD, a benchmark of 2,535 Bengali scene text images, each paired with a verbatim gold transcription, two categorical axes, four diagnostic attributes, and an orthographically standard form where the in-image text deviates from canonical spelling.
Sadab Shiper, Tawsif Tashwar Dipto, M. Inzamam et al.· 0 citations
This work introduces MemeCULT-1K, a multilingual benchmark of 1,000 South Asian memes in Bengali, English, and Hindi, where each meme is paired with a cultural context note and three human-written explanations, along with a supplementary set of 54 Bengali regional dialect memes.
Tawsif Tashwar Dipto, Mehedi Ahamed, Radib Bin Kabir et al.· 0 citations
We use cookies to run the site and, with your consent, for analytics and to show ads.
See our Cookie Policy.