Jul 2026· Annual International ACM SIGIR Conference on Research and Development in Information Retrieval· 0 citations· 15 references
Computer Science
TL;DR
This paper tests whether prompting the same (frozen) SLM in three typologically diverse languages and aggregating the outputs can improve classification without retraining or translation, and suggests that cross-lingual diversity rather than surface-level input variation drives the gain.
Abstract
With recent advancements, Small language models (SLMs) are increasingly used as preprocessors to handle query classification, routing, and candidate selection in retrieval pipelines, but they are nearly always prompted in English, even when users search in Hindi, Bengali, or code-mixed forms. We test whether prompting the same (frozen) SLM in three typologically diverse languages and aggregating the outputs can improve classification without retraining or translation. Nine decoder-only models (1B--9B parameters) evaluated on four public benchmarks show that confidence-weighted fusion of English, Hindi, and Bengali predictions raises macro-F1 by 3--5 points over English-only baselines, with the strongest gains on binary and coarse intent tasks. Parallel execution keeps latency within 1.2--1.4× of the single-language baseline. A paraphrase-only ensemble under identical conditions reaches only +1.4~F1 on average, suggesting that cross-lingual diversity rather than surface-level input variation drives the gain. Because no additional data, training, or translation services are required, our method may be useful when scaling to larger models is out of reach.
This work evaluates a fixed-list routing strategy that keeps stronger languages on a direct multilingual path and selectively sends weaker languages through translation into English before zero-shot classification, and reports routing through tier-level quality gains and tier-level latency rather than a single global efficiency score.
MTEB-PT is presented, a Portuguese benchmark constructed from a subset of MMTEB, comprising 14 existing datasets across Semantic Textual Similarity (STS), classification, retrieval, and reranking, and shows that language-specific fine-tuning still improves model performance in Portuguese, especially on task types that match the adaptation data most closely.
Lucas H.T. Okamura, Alexandre Alcoforado, A. H. R. Costa· 1 citation
Large language models (LLMs) are increasingly used as general-purpose translation systems, but their behavior is usually evaluated under a single prompt shape: translate one source sentence into one target language. In practice, users may ask for one target language, for several related languages at once, or for translations conditioned on examples. This paper studies prompt scope and demonstration selection as experimental variables for local LLM machine translation. We evaluate English-to-Romance and English-to-Germanic translation on the full FLORES devtest split for nine official European Union languages. We compare three local instruction-tuned LLMs, llama3.2:3b, mistral:latest, and qwen2.5:14b, against dedicated MT baselines from OPUS-MT and NLLB-200. We test zero-shot prompting and k=5 few-shot prompting with random, lexical-similarity, and embedding-similarity demonstration selection. We also compare single-target prompts with JSON-formatted family-scope prompts that request all languages in a family at once. Results show that dedicated MT systems remain strongest overall, especially for Germanic languages. Few-shot prompting helps mistral:latest and qwen2.5:14b, but hurts llama3.2:3b; embedding retrieval is best on average for the stronger LLMs, but its advantage over random and lexical examples is modest. Family-scope prompting is feasible for stronger local LLMs but exposes structured-output failures in smaller models. These findings motivate evaluating LLM translation not only by language pair and metric, but also by prompt scope, retrieval strategy, and multi-target compliance.
MGAL is the first multilingual, granularity- and position-aware long-context benchmark, constructed from United Nations reports spanning 8K to 128K tokens across the six official UN languages, and finds that LLMs perform well at word-level tasks but struggle with coarser-grained ones.
Chunhan Li, Chenglin Xu, Zongyang Zhang et al.· 0 citations
AssistEM, a framework for efficient LLM adaptation to EM via principled data selection, demonstrates that selective fine-tuning not only accelerates adaptation but also improves training efficiency (requiring fewer GPU hours), enabling open-source LLMs to rival–and in some cases outperform–closed-source models.
John Bosco Mugeni, S. Lynden, Toshiyuki Amagasa et al.· International Journal of Dat...· 0 citations
A PMI-based translation metric is proposed, which is less dependent on the target language and correlates strongly with chrF, and finds that CLA with English predicts translation quality comparably to or better than source-target CLA.
Adnan Al Ali, Kathy Hämmerl, Jindrich Libovický et al.· 0 citations