2026· Proceedings of the 11th Social Media Mining for Health Research and Applications (SMM4H-HeaRD 2026) Workshop and Shared Tasks· pp. 211-215· 1 citation· 15 references
TL;DR
This paper addresses Task 1 on adverse drug event (ADE) detection and Task 8 on Mul-tiClinNER at SMM4H-HeaRD 2026 and targets clinical NER for English text.
Abstract
In this paper, we address Task 1 on adverse drug event (ADE) detection and Task 8 on Mul-tiClinNER at SMM4H-HeaRD 2026. ADE detection is formulated as a multilingual binary classification problem over social media posts spanning German, French, Russian, English, Mandarin and Japanese, with zero-shot on Farsi. Using XLM-RoBERTa-Large with a dual-pooling head, combined with stratified sampling, language-conditioned inputs, translation-based augmentation, and calibrated ensembling, our model achieves a macro F1 score of 0.6088, surpassing both the competition mean (0.5465) and median (0.5798). Our work in MultiClinNER targets clinical NER for English text. Using GLiNER-large with sliding-window inference, query augmentation, and calibrated thresholds, it achieves strict F1 scores of 0.7591 (Disease), 0.7263 (Procedure), and 0.6733 (Symptom), outperforming a Pub-MedBERT baseline across all entities.
This study investigates the English-to-Turkish clinical translation performance of three large language models (Aya-23, Seed-X-Instruct, Gemma-3) using zero/few-shot prompting and QLoRA fine-tuning. Our three-stage setup evaluates: (1) general translation after FLORES-devtest training, (2) medical domain adaptation on i2b2 via a mixed dataset, and (3) generalization capacity on the independent FLORES-test set after mixed-data fine-tuning, using SacreBLEU, chrF, and COMET. Results show that Seed-X-Instruct achieves the strongest fine-tuned performance, reaching 32.70 BLEU, 64.34 chrF, and 90.65 COMET on the general FLORES test set, and 22.40 BLEU, 54.61 chrF, and 79.70 COMET on the medical i2b2 set. After mixed-data training, Seed-X improves from 11.48 to 32.39 BLEU on the unseen FLORES test set, whereas Aya-23 drops from 19.23 to 10.83 and Gemma-3 largely preserves its strong zero-shot baseline. These findings indicate that QLoRA effects are model-dependent and that domain adaptation does not yield the same generalization behavior across models.
Muhammetalp Erdem, Ş. Bora, Oguz Dikenelli· Signal Processing and Commun...· 0 citations
Abstract Background Adverse drug events (ADEs) pose significant public health challenges and economic burdens. While substantial ADE information is documented in unstructured clinical notes, its extraction remains difficult due to semantic complexity. Large language models (LLMs) offer promising text comprehension capabilities but are often hindered by domain-specific hallucinations. Objective This study aims to evaluate the effectiveness of retrieval-augmented generation (RAG) in improving the identification of ADEs using LLMs from Chinese clinical narratives and to establish a paradigm for this task. Methods We collected and preprocessed 19,983 Chinese clinical notes, retaining 18,432 high-quality records. Following a rigorous annotation and deduplication process, we established a gold-standard reference dataset (n=2510) and an ADE knowledge base (n=5144) using a standardized JSON schema. We evaluated 3 state-of-the-art LLMs (DeepSeek-V3 [DeepSeek], ERNIE 3.5-8K [Baidu], and GPT-4o [OpenAI]) under 3 prompt strategies: nonaugmented generation (NAG), static-augmented generation (SAG), and RAG. Performance was comprehensively assessed using precision, recall, and F1-score across 3 recognition matching levels (L1 exact, L2 sentence, and L3 overlap) via 1000 bootstrap resamples. Model robustness was further validated from real-world clinical progress notes, reflecting real-world ADE prevalence. Results We successfully constructed and publicly released the first Chinese ADE corpus derived from clinical notes. Across the tested LLMs, RAG yielded higher F1-scores than NAG and SAG at the L3 level. The optimal configuration, DeepSeek-V3 with RAG, achieved an overall L3-level F1-score of 0.9638 (95% CI 0.9541‐0.9727). Notably, the RAG approach increased the recall of GPT-4o from 0.6419 under NAG to 0.9241 under RAG (FDR P=.003). Evaluation on real-world datasets demonstrated clinical utility, with the RAG prompt maintaining high discriminatory capability (specificity: 0.9821; F2-score: 0.8885). Error analysis revealed that RAG successfully resolved common identification errors, both omissions and commissions, that were intractable for nonaugmented models. Conclusions Synergizing a curated domain-specific knowledge base with LLMs via a RAG architecture is an effective strategy for accurately identifying ADEs in unstructured Chinese clinical notes. This approach can mitigate hallucinations in LLMs, providing a foundational open-source benchmark and a robust technical framework to advance pharmacovigilance, drug safety research, and clinical decision support.
Junlong Ma, Xuehong Wu, Zeying Feng et al.· Journal of Medical Internet...· 0 citations
Alation studies reveal a strong synergy between POS features and CRF decoding, with the combined approach yielding the best single-model performance, and ensembling models trained with different objectives improves both overall F1 score and discontinuous MWE scores, demonstrating the importance of training diversity for capturing non-adjacent syntactic patterns.
Yunus Karatepe, Mert Sülük, Zeynep Tu˘gçe Kırımlı et al.· 0 citations
Overall, the proposed reformulation module produces more natural and clinically relevant Arabic questions and can be plugged into Arabic clinical QA pipelines as a measurable, tunable front-end.
Walid Ounachad, M. Khenchouch, Imad Zeroual et al.· Language Resources and Evalu...· 0 citations
Distractor selection rates in multiple-choice questions (MCQs) provide a behavioral proxy for distractor plausibility, yet current evaluation methods are reactive and resource-intensive. We investigated whether domain-specific fine-tuning of pre-trained embedding models can predict distractor selection rates from textual features before test administration. Using 6000 medical MCQs across eight clinical disciplines, we evaluated five general-purpose and five medical domain-specific embedding models under a unified 5-fold cross-validation protocol. Fine-tuning produced substantial improvements across both model categories: among medical models, SapBERT improved from
r
= 0.403 to
r
= 0.644 (+59.9%), while BGE-large improved from r = 0.467 to
r
= 0.626 (+34.0%) within the general group. Compared with lexical baselines where TF-IDF with string overlap features achieved the best performance (
r
= 0.546), the proposed transfer learning with fine-tuned contextual models showed meaningful improvement. Meanwhile, compact models also performed competitively: MiniLM (22 M parameters) reached
r
= 0.627 and MedEmbed-small (33 M) reached
r
= 0.629. These results establish the technical feasibility of text-based distractor selection rate prediction and characterize the performance landscape across model categories. This article offers a methodological investigation of a plausibility proxy, with potential application scenarios requiring future validation.
Zhehan Jiang, Tianpeng Zheng, Jiayi Liu et al.· npj Digital Medicine· 0 citations
A morphological barrier is shown: while pivot-based adaptation facilitates transfer for distant languages through script alignment, it induces negative transfer in morphologically complex agglutinative languages like Gondi.
R. Mundotiya· Annual International ACM SIG...· 0 citations