While current evaluations of LLM cultural alignment predominantly rely on static benchmarks in Western contexts, their ability to navigate generative, high-context socio-pragmatic demands in non-Western environments remains critically underexplored. This study investigates how multilingual LLMs adapt to the Japanese workplace—a stringent stress-test environment characterized by strong high-context communication norms and rigid honorific conventions—using Hofstede’s six cultural dimensions as a heuristic framework. We evaluated five state-of-the-art LLMs (LLM-jp, Phi, Llama, Qwen, and GLM) through a large-scale crowdsourced human evaluation. Based on 1,718 valid evaluator sessions, native Japanese raters assessed model outputs to generate a holistic Japanese Workplace Cultural Alignment Score (JWCAS). To dissect the underlying communicative strategies, we paired this with a three-layer diagnostic sub-score analysis (Linguistic Form, Socio-Cultural Values, and Social Action). Our results reveal that leading multilingual models (Phi and GLM) achieved overall JWCAS scores comparable to, or significantly higher than, the native Japanese model (LLM-jp). Crucially, our sub-score analysis demonstrates that holistic evaluation metrics can obscure deep pragmatic deficits: while LLM-jp overfits to surface-level linguistic politeness (Layer 1), it shows critical weaknesses in socio-cultural values (Layer 2) and context-aware social strategies (Layer 3). In contrast, leading multilingual models demonstrate balanced competence across all layers. These findings suggest that true cultural competence requires moving beyond native linguistic mastery, highlighting the necessity of multi-dimensional diagnostic frameworks for cross-cultural AI alignment.
Retrieval-augmented Generation (RAG) has emerged as an effective paradigm for enhancing large language models (LLMs) with external knowledge, delivering substantial performance gains without costly parameter updates, particularly in low-resource settings. However, existing RAG systems still face several open challenges. Many approaches rely on static or heuristic retrieval strategies, limiting their ability to adapt retrieval strategies to different tasks with different reasoning requirements. Some methods build RAG systems based on independently optimized components, making joint optimization across all components infeasible. Moreover, empirical evidence shows that stronger retrievers do not necessarily lead to better end-to-end RAG performance, revealing a misalignment between retrieval quality and generation effectiveness and leaving considerable room for improvement in how retrieval is integrated into the RAG pipeline. Motivated by these challenges, we propose ARF-RAG, an Adaptive Retriever-Friendly Retrieval-Augmented Generation framework. ARF-RAG dynamically determines when to retrieve and further generates retriever-friendly queries that faithfully reflect the generator's current information needs while being optimized for the behavior of the given retriever. ARF-RAG adopts a role-unified mechanism, in which a single LLM simultaneously performs all retrieval-related and generation actions, including retrieval decision-making, query generation, and answer generation, enabling coherent optimization across all components. To effectively align the retriever and generator, we formulate RAG as a unified reinforcement learning (RL) problem. We design comprehensive reward functions that jointly account for answer correctness, retrieval effectiveness, and retrieval cost, enabling fine-grained policy optimization via proximal policy optimization (PPO). Extensive experiments. The codebase for reproducing the proposed framework is publicly available at: https://github.com/ii-research/ARF-RAG on three question-answering benchmarks demonstrate that ARF-RAG consistently outperforms strong non-RL and RL-based baselines on both simple and complex QA tasks. Further analyses show that ARF-RAG adaptively learns when to retrieve based on question complexity and generates more effective, retriever-aligned queries that resolve what to retrieve, highlighting the advantages of unified RL optimization for building robust and efficient RAG systems.
Yubo Fang, Hai-tao Yu, Hideo Joho et al.· International Conference on...· 0 citations
