2026· Journal of Advances in Information Technology· Vol 17, pp. 1321-1330· 0 citations· 28 references
TL;DR
Test set results show that Decoding-Enhanced Bert with Disentangled Attention (DeBERTa) achieves the highest macro F1 − Score of 85.48%, surpassing the previously top-ranked Multi-Task Learning (MTL) system, which attains a macro F1 of 83.07%.
Abstract
—Improved fluency in large language models has intensified the need for accurate detection of machine-generated text. This study evaluates transformer-based models using an improved version of the Conference on Computational Linguistics 2025 (COLING 2025), Generative Artificial Intelligence (GenAI) Content Detection Task 1 dataset, which was carefully preprocessed to enhance label quality and balance. All models were trained under a unified protocol to ensure fair comparison and robust evaluation. Test set results show that Decoding-Enhanced Bert with Disentangled Attention (DeBERTa) achieves the highest macro F1 − Score of 85.48%, surpassing the previously top-ranked Multi-Task Learning (MTL) system, which attains a macro F1 of 83.07%. These results highlight the effectiveness of advanced transformer architectures for distinguishing human-written and machine-generated text. Despite these gains, performance degradation under domain shift and highly paraphrased inputs remains a challenge.
Large Language Models (LLMs) have been rapidly evolving lately, resulting in the need for strong, explainable models to detect the difference between human-generated and machine-generated articles. Existing approaches which are mostly based on fine-tuned transformers suffer from several drawbacks such as rapid obsolescence, paraphrasing attacks, and lack of interpretability. To improve their ability to detect, this paper proposes a novel paradigm called Human vs. LLM Identification (HLI) which introduces a Retrieval-Augmented Generation (RAG)-inspired evidence-based detection strategy alongside a fine-tuned transformer classifier. Our core model, DeBERTa-Sentinel, is built on top of a fine-tuned Microsoft DeBERTa-v3-small model, which uses a disentangled attention mechanism to better capture subtle syntactic and stylistic deviations characteristic of AI-generated text. We evaluate our framework on a balanced dataset of 43,456 text samples, curated from the OpenGPTText corpus and covering AI-generated and human-authored content across diverse domains including news, education, and creative text. The experimental results show improved performance over the selected baselines, with our framework achieving an accuracy of 97.53%, precision of 95.89%, recall of 99.34%, and ROC-AUC of 99.53%. In addition, explainability is integrated into our framework through Local Interpretable Model-agnostic Explanations (LIME) analysis, providing token-level insight into classification decisions. This study establishes a benchmark for scalable, explainable AI text detection, with implications for academic integrity, content moderation, and combating misinformation.
Ibtasam Ur Rehman, Muhammad Islam, Muhammad Yousaf Rehman et al.· Knowledge· 0 citations
The rapid growth of academic literature has intensified the need for effective automatic text summarization techniques capable of producing concise and informative representations of scientific documents. While extractive methods are widely used, they are limited in their ability to generate coherent and semantically rich summaries. Recent advances in Transformer-based architectures have enabled significant progress in abstractive summarization; however, their effectiveness on domain-specific datasets, such as scientific articles, remains an open challenge. In this study, we investigate the performance of three pre-trained Transformer-based models—T5, BART, and GPT-2—on the task of abstractive summarization using the CL-SciSumm 2019 dataset. A total of 19 experimental configurations are conducted to analyze the impact of generation parameters, including beam size, length penalties, and n-gram constraints, on summarization quality. The models are evaluated using ROUGE metrics, with a focus on ROUGE-2.To complement content-based evaluation, this work incorporates linguistic acceptability assessment using the Corpus of Linguistic Acceptability (CoLA), a benchmark dataset for evaluating grammatical correctness. The results show that BART achieves the best performance with an ROUGE-2 F1-score of 0.40664, while T5 demonstrates superior grammatical acceptability, achieving 93.36%, but BART achieves a very near performance to T5. Ultimately, these findings demonstrate the potential of pre-trained neural networks, particularly the BART architecture, to drive the future of complex, generative NLP applications, transforming how academic research is processed and understood.
Emad Nabil· Islamic University Journal o...· 0 citations
Part-of-Speech (POS) tagging is a foundational task in Natural Language Processing (NLP), yet it remains challenging for low-resource and morphologically rich languages such as Amazigh. This paper proposes a hybrid ensemble framework for Amazigh POS tagging that integrates three complementary models: a Bidirectional Long Short-Term Memory network (BiLSTM), a Conditional Random Field model (CRF), and a rule-based morphological analyzer (RBMA). Rather than treating prior results obtained on different corpora and tag inventories as directly comparable, the study evaluates all proposed components under a common 54-tag experimental setting based on the publicly available Amazigh Linguistic Dataset. Three ensemble strategies are examined: majority voting, validation-weighted voting, and logistic-regression stacking. An additional late-fusion ablation applies hard and soft RBMA constraints to CRF and Stacking outputs; hard masking degrades performance substantially, whereas soft masking is more robust but remains below unconstrained decoding. The best micro-level performance is obtained by the stacking ensemble, which reaches 98.51% Micro-F1/accuracy, whereas the boosting-like weighted ensemble obtains the strongest Macro-F1 among the ensemble variants, reaching 74.24%. These results show that hybrid ensemble methods can improve token-level accuracy in low-resource POS tagging, while also revealing a trade-off between frequent-tag accuracy and rare-tag robustness. The findings highlight the usefulness of combining neural, probabilistic, and rule-based information for Amazigh POS tagging, and point to class-balanced meta-learning and character/subword representations as important directions for improving rare and out-of-vocabulary categories.
Abdelouahed Moussaoui, Nor-Eddine Azalmad, Said Bahassine et al.· Information· 0 citations
The rapid proliferation of digitally distributed news has made large-scale automated verification an urgent research priority, as deceptive content spreads across social platforms faster than manual reviewers can evaluate it. This work introduces a two-stage deep learning pipeline in which a pretrained BERT encoder generates rich contextual token embeddings that are subsequently processed by a Bidirectional Long Short-Term Memory (BiLSTM) network, enabling the classifier to exploit both sentence-level semantics and document-level narrative flow simultaneously. Experiments on the WELFake benchmark [1] following a standardised cleaning pipeline—lowercasing, URL stripping, tokenisation, stop-word removal, and WordNet lemmatisation— yielded 98.7 % accuracy and an F1-score of 0.986. This surpasses a BERT-only baseline by 3.1 percentage points and exceeds several previously published state-of-the-art results. The gains confirm that coupling transformer-based semantic representations with recurrent sequential modelling produces a measurable and reproducible improvement in misinformation detection. Planned extensions include multilingual evaluation and knowledge-distilled encoder replacements targeting real-time throughput constraints.
Ardra P Namboodiri, Archa P S, Honey Mol O· 2026 6th International Confe...· 0 citations
Plain Language seeks to enhance the clarity and comprehensibility of legal and administrative communication; while Natural Language Processing (NLP) offers promising tools for assessing text complexity, most Plain Language classification studies focus exclusively on English, leaving low-resource languages underexplored. This study presents the first systematic evaluation of transformer-based models for sentence-level Plain Language classification in Hungarian tax administrative texts. We benchmarked zero-shot prompting with GPT-4o against fine-tuned open-weight and proprietary models, including huBERT, XLM-RoBERTa, GPT-4o-mini, and Gemini 1.0 Pro, and contextualized these results against previously established lightweight machine learning baselines based on term frequency-inverse document frequency with a support vector machine (TF-IDF + SVM) and fastText. To address data scarcity, we applied translation-based data augmentation using parallel Hungarian–English corpora. The best-performing model achieved a macro-average F1-score of 0.79. Mid-sized models also delivered competitive results, combining accuracy with feasible inference speed and deployment flexibility. Beyond classification performance, we conducted local and aggregated interpretability analysis based on Shapley-values to identify linguistic patterns influencing model decisions. This revealed alignment with known Plain Language features, such as nominalizations and syntactic complexity, as well as biases introduced by frequent domain-specific terms. Our findings demonstrate that Plain Language classifiers can be effectively adapted to low-resource legal–administrative domains. The results support the development of real-time feedback tools that promote linguistic accessibility and contribute to the broader goal of Access to Justice.
Transformer-based models such as BERT, RoBERTa, DistilBERT, and DeBERTa have achieved remarkable performance across a wide range of natural language processing (NLP) tasks. However, their decision-making processes remain difficult to interpret, particularly in high-risk applications such as hate speech detection, where unreliable explanations may undermine model transparency, trust, and accountability. This study investigates whether explainability methods remain faithful and stable under domain shift in transformer-based text classification. Four transformer architectures were fine-tuned and evaluated on two linguistically distinct datasets: IMDb Movie Reviews and Hate Speech Offensive. Model performance and explanation quality were assessed using classification accuracy, macro F1-score, top-k token-removal faithfulness analysis, and cross-domain Spearman rank correlation. Experimental results show that DeBERTa achieved the highest classification performance, reaching accuracies of 95.6% on IMDb and 91.3% on Hate Speech. Across all evaluated models and datasets, SHAP consistently produced higher faithfulness scores than attention-based explanations. Cross-domain analysis further revealed reduced agreement between SHAP and attention-based explanations under domain shift, indicating lower explanation consistency across linguistically distinct domains. Qualitative error analysis further showed that implicit sentiment, sarcasm, and domain-specific slang remain major sources of prediction errors. Overall, the results demonstrate that superior predictive performance does not necessarily correspond to higher explanation faithfulness or stronger cross-domain stability. These findings highlight the importance of jointly evaluating predictive performance, explanation faithfulness, and explanation robustness when developing trustworthy transformer-based NLP systems.
Dony Bahtera Firmawan, B. Darnoto· Journal of Computing Theorie...· 1 citation