Skip to content

CIBuzzBench: A Benchmark for Cross-Lingual Understanding of Chinese Internet Buzzwords

Sep 2026 · 0 citations · 17 references
Computer Science

TL;DR

The results show that LLMs continue to struggle with the cross-lingual understanding of Chinese internet buzzwords, particularly in fine-grained non-literal interpretation, robust equivalent matching under option perturbations, and calibrated harmfulness detection, which highlights the persistent challenges posed by culturally grounded language phenomena for multilingual LLMs and safety-oriented evaluation.

Abstract

Chinese social media has generated a vast and continually evolving lexicon of internet buzzwords whose meanings are often non-literal and deeply rooted in local cultural and pragmatic contexts. Existing research has primarily focused on interpreting these buzzwords within Chinese, leaving largely unexplored whether LLMs can transfer such culturally grounded knowledge across languages and accurately convey the intended meanings in English. This cross-lingual capability is also critical for safety, as harmful expressions may obscure their offensive content through culture-specific homophony, euphemism, irony, or coded language. In this paper, we investigate the ability of advanced LLMs to understand Chinese internet buzzwords across languages. To this end, we introduce CIBuzzBench, the first benchmark for cross-lingual Chinese-to-English understanding of Chinese internet buzzwords. CIBuzzBench comprises 3,001 Chinese internet buzzwords annotated with English meaning explanations, English equivalents, category labels, and harmfulness labels. Based on these annotations, we design three evaluation tasks: Meaning Explanation, Cross-lingual Equivalent Matching, and Culturally Grounded Harmfulness Detection. We evaluate representative state-of-the-art proprietary and Chinese LLMs under both English- and Chinese-prompting settings. Our results show that LLMs continue to struggle with the cross-lingual understanding of Chinese internet buzzwords, particularly in fine-grained non-literal interpretation, robust equivalent matching under option perturbations, and calibrated harmfulness detection. These findings highlight the persistent challenges posed by culturally grounded language phenomena for multilingual LLMs and safety-oriented evaluation. The dataset and code are available at https://github.com/SuperYFan/CIBuzzBench.

View source

Similar papers

Open access 2026

The English Translation of Chinese Internet Buzzwords and Its Dissemination and Reception in Mainstream British and American Media

Based on Fairclough's three-dimensional discourse analysis model and the News on the Web (NOW) corpus, this study is going to explore the dissemination and reception of Chinese internet buzzwords in mainstream British and American media from 2019 to 2024. The research will focus on high-frequency buzzwords such as "nei...

Zi-Yuan Dai · 0 citations
Review Sep 2026

Mapping translanguaging research in global Chinese language education: a systematic review

Abstract Over the past decade, translanguaging has emerged as a significant framework in language education, yet its application in Chinese language teaching remains underexplored. This systematic review examines 52 empirical studies published between 2014 and 2024 to map how translanguaging has been conceptualized and...

Xin-Yue Lu, Zhong-Feng Tian, Dan-Ping Wang · 0 citations
#natural language process... Preprint Oct 2026

Lost in Translation: Investigating Systematic Discrepancies between Parallel English and Chinese Names of American Chinese Restaurants

Given that the Chinese are one of the oldest ethnic groups to immigrate to the US, their cuisine have become central to the US dining scene and an interesting avenue to study how Chinese immigrants have culturally adapted into American life. In that regard, previous research has focused on analyzing the English names t...

Hang Jiang, Nan-Xi Liu, Tian-Tian Wang et al. · 0 citations
Open access Oct 2026

Translating ambiguity: A comparative evaluation of AI Chatbots in rendering Arabic full homonyms into English

The study concludes that AI systems require substantially more training, particularly in Arabic-language contexts, and recommended further research on homonym translation, focusing on different contexts as well as the rendition of homonyms outside of pairings.

A. Al-Momani, M. Al-Taher, Sultan Alali · 0 citations
#natural language process... Preprint Oct 2026

Social Bias in Chinese Large Language Models under Social Media Expressions

It is demonstrated that expression form is not a transparent carrier in bias evaluation and that lower scores under sarcasm should not be interpreted directly as bias mitigation, and the proposed paired design offers a measurement approach for Chinese social-media contexts while also revealing the construct boundary an...

Mo Zhou · 0 citations

Related blog posts

We use cookies to run the site and, with your consent, for analytics and to show ads. See our Cookie Policy.