Current cultural evaluations for large language models (LLMs) often reduce culture to single-turn factual recall via MCQs, failing to capture a common use case: users seeking practical help over multiple turns in culturally grounded scenarios. We introduce CultureConverse, a scalable, multilingual simulation and evalua...
Bryan Chen Zhengyu Tan, Wei-Hua Zheng, Thong T. Doan et al.· 0 citations
Camellia is introduced, a benchmark for evaluating entity-centric cultural biases in nine Asian languages, spanning six Asian cultures, and it is found that LLMs can struggle with context understanding in some Asian languages, creating performance gaps between cultures in entity extraction.
Tarek Naous, Anagha Savit, Carlos Rafael Catalan et al.· arXiv.org· 2 citations· ⚡1
Role-playing large language models (LLMs) are expected to adopt a character's style while also respecting that character's knowledge boundaries. Prior evaluations detect character hallucination but rarely distinguish whether errors arise from failure to recognize a boundary or from failure to comply despite recognition...
Su-Hyun Han, Nahyeon Park, Gaeun Seo et al.· 0 citations
It is highlighted that effective cultural alignment requires context-conditional modeling rather than uniform debiasing, and a new direction for mitigating entity-centric cultural bias in LLMs is established.
CultureConverse is introduced, a scalable, multilingual simulation and evaluation harness for culturally grounded assistant dialogue that covers 10 East and Southeast Asian regions, 58 subgroup identities, and 7 domains and performance gains from fine-tuning on 27,860 high-quality CultureConverse-DS samples improve in-...
Bryan Chen Zhengyu Tan, Weihua Zheng, Thong T. Doan et al.· 0 citations
We use cookies to run the site and, with your consent, for analytics and to show ads.
See our Cookie Policy.
