Alation studies reveal a strong synergy between POS features and CRF decoding, with the combined approach yielding the best single-model performance, and ensembling models trained with different objectives improves both overall F1 score and discontinuous MWE scores, demonstrating the importance of training diversity for capturing non-adjacent syntactic patterns.
Part-of-Speech (POS) tagging is a foundational task in Natural Language Processing (NLP), yet it remains challenging for low-resource and morphologically rich languages such as Amazigh. This paper proposes a hybrid ensemble framework for Amazigh POS tagging that integrates three complementary models: a Bidirectional Long Short-Term Memory network (BiLSTM), a Conditional Random Field model (CRF), and a rule-based morphological analyzer (RBMA). Rather than treating prior results obtained on different corpora and tag inventories as directly comparable, the study evaluates all proposed components under a common 54-tag experimental setting based on the publicly available Amazigh Linguistic Dataset. Three ensemble strategies are examined: majority voting, validation-weighted voting, and logistic-regression stacking. An additional late-fusion ablation applies hard and soft RBMA constraints to CRF and Stacking outputs; hard masking degrades performance substantially, whereas soft masking is more robust but remains below unconstrained decoding. The best micro-level performance is obtained by the stacking ensemble, which reaches 98.51% Micro-F1/accuracy, whereas the boosting-like weighted ensemble obtains the strongest Macro-F1 among the ensemble variants, reaching 74.24%. These results show that hybrid ensemble methods can improve token-level accuracy in low-resource POS tagging, while also revealing a trade-off between frequent-tag accuracy and rare-tag robustness. The findings highlight the usefulness of combining neural, probabilistic, and rule-based information for Amazigh POS tagging, and point to class-balanced meta-learning and character/subword representations as important directions for improving rare and out-of-vocabulary categories.
Abdelouahed Moussaoui, Nor-Eddine Azalmad, Said Bahassine et al.· Information· 0 citations
One of the basic Natural Language Processing (NLP) tasks is Part-of-Speech (POS) tagging, which helps in various applications like sentiment analysis and information retrieval. However, creating accurate POS taggers for low-resource African languages continues to be difficult due to the scarcity of linguistic resources that are annotated. Its contribution is a deep learning method for POS tagging of Dholuo, a less-resourced Western Nilotic language, spoken by about four million people in Kenya and Tanzania. The suggested system uses DistilBERT, a small transformer model, in addition to FastText and Word2Vec vector representations of words that are used to capture the context and meaning of a word. The KenCorpus Dholuo POS dataset was carefully preprocessed, normalized, and standardized with the Universal POS tags and balanced using a hybrid resampling strategy to bring about class representation. The proposed method combines contextual transformer representations with complementary word representations and a training strategy that is optimized for the linguistic features of Dholuo, while previous studies primarily used multilingual transformer models or traditional sequence-labeling methods. The framework developed is a computationally efficient one that is well-suited for low-resource language processing. The results of the experiments reveal that the proposed DistilBERT-based model outperforms the baseline Conditional Random Field (CRF) and Bidirectional Long Short-Term Memory (BiLSTM) models with an accuracy 79.05%, precision 80.63%, recall 79.05% and F1-score 79.24%. To our best knowledge, these results are the best reported for Dholuo POS tagging, and for under-resourced languages in Africa in general, highlighting the suitability of lightweight transformer architectures.
Maureen Otieno, L. Wanzare, Calvins Otieno· International Journal of Com...· 0 citations
A PMI-based translation metric is proposed, which is less dependent on the target language and correlates strongly with chrF, and finds that CLA with English predicts translation quality comparably to or better than source-target CLA.
Adnan Al Ali, Kathy Hämmerl, Jindrich Libovický et al.· 0 citations
In highly specialized and terminology-dense scientific and technical texts, existing word sense disambiguation (WSD) models struggle to adequately model the contextual semantic dependencies of polysemous words, especially in engineering domains where the same term may carry different technical meanings across contexts. To address this issue, this paper proposes a robust WSD model that integrates a bidirectional long shortterm memory network (BiLSTM ) with an attention mechanism, specifically designed for Chinese patent texts. First, a two-layer BiLSTM is used for bidirectional context modeling to capture long-range dependencies. Then, a multi-head attention mechanism uses dynamic weighting to highlight key semantic components, generating highly discriminative context vectors. Finally, a paraphrase alignment mechanism employs bilinear matching to align the context vectors with candidate paraphrase embeddings, thereby reducing semantic confusion. Experiments show that the model achieves a Top-1 accuracy of 88.6 % on high-frequency words, with an average paraphrase alignment similarity of 0.920. In perturbation tests, the average robustness index is 0.159, representing reductions of 62.7%, 53.9%, and 39.5% compared to Word2Vec+CNN, BiLSTM, and BERT, respectively. The method presented in this paper helps to enhance the accuracy and stability of word meaning recognition in technical texts, providing reliable support for knowledge mining and intelligent text processing in technical domains. Its terminology alignment is also useful for engineering corpora where antenna, wavepropagation and materials terms require context-sensitive interpretation.
L. X. Gao, T. Dong, M. H. Yang· Advanced Electromagnetics· 0 citations
Pretrained language models (PLMs) have established state-of-the-art performance across diverse natural language understanding (NLU) tasks. This study reveals that seman-tic-rich explanations of lexical units can effectively guide PLM learning processes. We propose a novel language understanding enhancement method with token interpretation (LUETI) that addresses two critical limitations in conventional PLMs: Incomplete token semantics caused by isolated contextual learning and insufficient semantic encoding in embedding matrices. LUETI operates through dual mechanisms, augmenting token represen-tations by integrating hidden states with corresponding token interpretations and refining embedding spaces using interpretation-derived semantic vectors for token prediction. LUETI, which is implemented as a plug-in module for standard architectures, demonstrates significant improvements on BERT and GLM, achieving average performance gains of 3.36% and 4.87% respectively on the SuperGLUE benchmark with equivalent parameters and training data. Note that LUETI-equipped models attain comparable performance to baseline PLMs using only 60% of pretraining data. Findings establish token interpretation as a computationally efficient but semantically powerful enhancement strategy for language model pretraining.
Tianyi Chen, Yashen Wang, Huan Chang et al.· IEEE/CAA Journal of Automati...· 0 citations
Hate speech detection in low-resource and informally written languages remains a significant challenge due to the lack of annotated corpora, orthographic variability, and complex code-mixing. Roman Urdu a non-standardized variant of Urdu written in the Latin script exemplifies these linguistic hurdles. In this paper, we propose LEXF-ATT-XLM, a novel hybrid deep learning architecture that synergizes contextual language modeling with explicit domain knowledge. Our model leverages XLM-RoBERTa for deep contextual embeddings, passed through a two-layer Bidirectional Gated Recurrent Unit (BiGRU) and a multi-head attention mechanism to capture both sequential and salient linguistic patterns. Crucially, the pooled representations are fused via a learnable gating layer (Linear + tanh) before final classification. Furthermore, we integrate a domain specific Roman Urdu hate lexicon as an auxiliary regression supervision signal within a multi task learning framework to guide the model’s focus. Evaluated on the RU-HSD-30K dataset using 3-fold stratified cross-validation, the proposed model achieves an average accuracy of 88.83% and a weighted F1 score of 88.83%, with fold wise weighted F1-scores of 88.58%, 89.19%, and 88.71%. Extensive ablation studies confirm that the lexicon-guided auxiliary supervision significantly enhances the model’s ability to handle lexical variations, negations, and informal spelling. These findings demonstrate the robust effectiveness of our approach in addressing the unique linguistic challenges of Roman Urdu hate speech detection.
Jaweria Jalil Awan, Muhammad Hamid, T. Alshalali· PLoS ONE· 0 citations