This system for SemEval-2026 Task 5 is reported, which predicts graded plausibility scores for target word senses in narrative context and shows that distribution modeling better aligns with human plausibility judgments than single-score prediction.
Abstract
Here, we report our system for SemEval-2026 Task 5 (Gehring et al., 2026), which predicts graded plausibility scores for target word senses in narrative context. We explore embedding-based similarity, transformer fine-tuning, and a three-stage curriculum combining WiC pretraining, Wasserstein distribution learning, and KL-based calibration. Our best model, DeBERTa-xLarge with curriculum training, achieves 78% accuracy within one standard deviation and a Spearman Correlation of 0.70 , with an overall test score of 0.74. Results show that distribution modeling better aligns with human plausibility judgments than single-score prediction.
This study explores the semantic competence of Italian BabyLMs, focusing on their sensitivity to semantic violations, using a minimal pair benchmark targeting semantic violations to evaluate the semantic abilities of BAMBI, a family of small-scale models trained on progressively larger and more complex datasets.
A. Suozzi, Luca Capone, Gianluca Lebani et al.· 0 citations
Automatic Speech Recognition (ASR) is typically evaluated using Word Error Rate (WER), which poorly reflects semantic similarity. While embedding-based metrics correlate better with human judgments, the respective roles of encoder and decoder-based Large Language Models (LLMs) remain underexplored. This paper presents a comparative study of both families for ASR evaluation. We analyze BERTScore and SemDist across different LLMs, layers, and pooling strategies, showing that both metrics can achieve strong correlation with human judgments when properly configured. For decoder models, we investigate generative LLMs in two settings: pairwise hypothesis selection via prompting and direct qualitative error classification. Our results show that encoder-based metrics remain highly competitive, while generative LLMs perform strongly in hypothesis comparison and improve the interpretability of ASR evaluation.
Thibault Bañeras-Roux, Shashi Kumar, Driss Khalil et al.· 0 citations
Legal concepts in statutes are often expressed using vague terms, and practitioners frequently turn to case law to interpret them. We study the task of ranking case-law sentences by their usefulness for explaining a concept or target statutory term, using an established dataset of 26,959 sentences covering 42 U.S. Code concepts labeled into four explanatory-value categories. We compare two families of methods: (i) supervised fine-tuning of encoder-only models (ModernBERT) and (ii) zero-shot prompting of decoder-only models. We show that across all concepts and standard NDCG cutoffs, ModernBERT largely matches earlier BERT-family baselines. In contrast, prompting decoder-only models achieves the strongest overall effectiveness, with our best system surpassing all previously reported state-of-the-art results on this task.