Skip to content
Open access

Low resource word sense disambiguation in Oromo with fine tuned small transformers.

Jul 2026 · Scientific Reports · 0 citations
Medicine

TL;DR

Results show that contextual transformer representations are quite successful for low-resource WSD, although there is still a significant class imbalance that limits performance.

Abstract

A key task in natural language processing is word sense disambiguation (WSD), which attempts to determine the accurate meaning of ambiguous words based on their context. While transformer-based designs have achieved significant results in high-resource languages, WSD for low-resource languages such as Oromo remains hard due to inadequate annotated corpora and lexical resources. Contextual representation learning has been greatly enhanced by recent advancements in transformer-based language models, allowing for more reliable disambiguation in situations with limited input. This study uses a manually created dataset from the Oromo-English Dictionary to examine the efficacy of transformer-based models for lexical-sample WSD in Oromo. The dataset contains sentences annotated by two native speakers, attaining an inter-annotator agreement of 0.82, indicating good annotation reliability. The dataset was filtered for experimental usage following preprocessing, normalization, and elimination of noisy cases. 472 training sentences, 71 validation sentences (15%), and 140 test sentences made up the final dataset. The dataset has a highly unbalanced long-tail distribution and encompasses 43 sense classes. BERT-base-cased gets the best performance with an accuracy of 0.862 and a macro-F1 score of 0.2897, according to an experimental evaluation of transformer-based models, including BERT, RoBERTa, DistilBERT, Davlan/afro-xlmr-base, and multilingual variations. Significant differences between models with χ2 = 34.03 and p = 4.0 × 10-1 are confirmed by statistical analysis using the Friedman test. BERT-base-cased performs much better than most transformer variations and classical baselines, according to post-hoc Wilcoxon signed-rank tests. These results show that contextual transformer representations are quite successful for low-resource WSD, although there is still a significant class imbalance that limits performance.

Read PDF

Similar papers

Aug 2026

A semi-automated LLM-based framework for word sense disambiguation in Serbian

Word Sense Disambiguation (WSD) remains a challenge in natural language processing, particularly for morphologically rich and less-resourced languages, e.g., Serbian, where complex inflectional systems intensify lexical ambiguity. Existing WSD approaches often struggle because of scarce annotated corpora, inadequate sense inventories, and linguistically complex languages. This paper presents a semi-automated framework for Serbian WSD resource development that employs Large Language Models (LLMs) and a constrained JSON-formatted output. The approach combines LLM-assisted sense assignment with a Serbian WordNet-based custom inventory, iterative inventory expansion, and expert validation, with the aim of supporting the practical construction and refinement of sense-annotated resources in a low-resource setting. In WSD experiments, LLMs significantly outperformed the Serbian-specific TeslaXLM , the multilingual embedding model intfloat/multilingual-e5-large , and the sentence transformer all-MiniLM-L6-v2 . The expanded sense inventory improved the accuracy, particularly for GPT-4.1 , which achieved 92.4% for single-word and 95.2% for multi-word spans. These results position LLMs as potentially practical for scalable sense-annotated data creation in less-resourced settings. The proposed methodology addresses key limitations in existing WSD pipelines for Serbian, and contributes to the broader effort of adapting LLMs for semantic processing in linguistically diverse and data-scarce environments.

Saša Petalinkar, R. Stanković, Milica Ikonić Nešić et al. · 0 citations
Open access Aug 2026

Enhancing Hindi Word Sense Disambiguation Using Supervised Logistic Regression and Contextual Features

Natural languages are ambiguous by nature. Word sense ambiguity is one of the numerous layers of ambiguity. In many applications of natural language processing, sense ambiguity resolution is essential. In this work, word sense ambiguity is addressed, and a supervised method for Hindi word sense disambiguation has been suggested. A supervised method for Hindi word sense disambiguation is employed, incorporating contextual feature modeling and systematic preprocessing to effectively resolve ambiguity. After applying tokenization, POS tagging, stop-word removal, and Lemmatization, a context window is constructed over open-class words. A logistic regression model is used, and to extract feature TF-IDF technique is used. An average accuracy of 78.93% is shown by experimental findings on 20 polysemous Hindi words, which is higher than previous published work on the same dataset.

Vinto, Neeru Mago, Raj Kumari · 0 citations
Open access Jul 2026

Minimal-Supervision Part-of-Speech Tagging for Assamese Language: An Evidence-Gated Cascade with Classifier Backfill

Part-of-speech (POS) tagging for low-resource languages is limited more by the expense of generating labeled training data than by algorithmic constraints. This study evaluates the extent to which a length-stratified, minimally-supervised pipeline can recover tagging accuracy for the Assamese language using a labeled seed of only 3 to 5 sentences per distinct sentence length (142 to 228 sentences in total), compared to a weak-supervision classifier trained on a conventional 80% corpus split (2,840 sentences). An evidence-gated cascade is constructed, comprising Brown-style word clustering, a seed-ambiguity audit with context-based disambiguation, confidence-gated cluster labeling, and empirically validated rule fallbacks. This approach achieves 94.9% to 95.0% accuracy but only 19.8% to 20.0% token coverage from its minimal seed. To address the coverage gap, every remaining token is backfilled using a classifier trained on the pipeline’s accumulated evidence, resulting in 100% coverage at 84.7% to 86.0% accuracy (depending on seed size), as evaluated against a fixed, shared test set for direct comparison with the larger-budget baseline (90.8% accuracy, same coverage, same test set). Three negative results are reported alongside the positive findings: Brown-style clustering provides negligible benefit at this scale, Viterbi sequence decoding does not transfer to the backfill classifier from the larger-budget setting, and naive self-training fails due to confirmation bias. Additionally, through five replicated random seed draws per condition, it is demonstrated that the residual seed-size effect (3 versus 5 sentences per length, closing approximately 21% of the accuracy gap to the larger-budget baseline) is statistically significant rather than the result of a single favorable draw.

B. Sarma, Rupam Baruah, Diganta Baishya · 0 citations
Jul 2026

Parsing Middle High German: exploring cross-lingual NLP for treebank construction in low-resource historical languages

Building syntactically annotated corpora, such as treebanks, for historical languages is a challenging yet vital task in digital humanities, as it underpins linguistic analysis and facilitates a range of interdisciplinary research. However, the scarcity of annotated data and the need for extensive expertise in historical linguistics make this process particularly demanding. In this study, we explore the potential of cross-lingual natural language processing (NLP) techniques as a semiautomatic solution for treebank construction in low-resource historical languages. We use Middle High German (MHG) as a case study. Leveraging the linguistic continuity and structural similarities between MHG and Modern German (MG), we effectively utilize the extensive MG treebank resources to develop a constituency parsing system tailored for MHG. Specifically, to design a semiautomatic system that integrates automatic annotation with manual validation, we explore two cross-lingual transfer techniques: zero-shot transfer and delexicalization; the latter removes lexical information to focus on syntactic structure. In our experiments, we first train parsers on MG treebanks, and then transfer them to MHG using the two cross-lingual transfer techniques. The delexicalization method achieves a parsing performance of 67.3 per cent in terms of F1-score. This performance significantly surpasses the zero-shot cross-lingual method by a margin of 28.6 percentage points. These investigations validate the effectiveness and feasibility of cross-lingual transfer techniques for historical language treebank construction. This study highlights the potential of NLP tools to streamline the semiautomatic annotation process, reducing the reliance on extensive linguistic expertise and manual effort, and paving the way for broader applications in digital humanities research.

Ercong Nie, Siyao Peng, Helmut Schmid et al. · 0 citations