2026· SemEval@ACL· pp. 1477-1481· 1 citation· 9 references
Computer Science
TL;DR
NCL’s system for SemEval-2026 Task 8B, the generation track for multi-turn retrieval-augmented dialogues, achieves a harmonic mean score of 0.5973 and ranks 19th out of 26 teams on the leaderboard.
Abstract
We present NCL’s system for SemEval-2026 Task 8B, the generation track for multi-turn retrieval-augmented dialogues. Our submission follows a compact and reproducible RAG pipeline: (1) global and local question rewriting with LLM-based multi-turn relation control, (2) passage reranking with BGE-M3, (3) context-level answerability filtering with strict binary LLM judgments (“yes”/“no”), and (4) deterministic inference with a small-llm (Qwen2.5-1.5B-Instruct) plus post-generation quality fall-back (cleaning, bad-answer gate, one stricter retry, then an IDK fallback). On the official test set, our system achieved a harmonic mean score of 0.5973 (RB agg 0.4993, RL F 0.7235, RB llm 0.6105), ranking 19th out of 26 teams on the leaderboard.
DCASE~2026 Task~5 introduces Audio-Dependent Question Answering (ADQA), which tests whether large audio-language models answer from the audio rather than from textual priors. An Audio-Dependency Filtering (ADF) pipeline combines silent-audio probing, per-option perplexity, a large language model (LLM) commonsense check, and human review to remove items solvable from text alone. The 3000 items that pass form the ADQA-Bench evaluation set, spanning music, speech, and environmental audio. The inaugural edition draws 14 teams and 36 submissions across two tracks defined by total parameter count (up to 100B and under 10B). A Chung-Ang University ensemble of MOSS-Audio-8B-Thinking and Qwen3-Omni-30B reaches the top overall accuracy at \pct{58.33}, and a MOSS-only configuration from the same team leads the sub-10B track at \pct{57.30}. Across the 30 submissions with a comparable development score, evaluation accuracy falls by 11.91 percentage points (pp) on average (median 10.91\,pp) on the hidden evaluation split, which is designed to be harder than the development split. The most common building blocks are: the MOSS-Audio-8B-Thinking backbone (13 of 36 submissions), Low-Rank Adaptation (LoRA) fine-tuning on AudioMCQ-StrongAC, and preference or reinforcement-learning objectives -- Group Relative Policy Optimization (GRPO) in five teams, Group reward-Decoupled Normalization Policy Optimization (GDPO) in two. At test time, prompt engineering is near-universal, and majority or choice-permutation voting is common. Every system misses the same set of 233 evaluation items.
Haolin He, Renhe Sun, Zheqi Dai et al.· 0 citations
FinMMEval 2026 Task 2 evaluates short-answer financial question answering over multilingual evidence. Each final-test item pairs an English question with financial statements and news in English, Chinese, Japanese, Spanish, and Greek. Participating systems submit one concise answer per item in JSONL format. The final-test set contains 256 items, split evenly between easy and expert tiers; each tier contains four question templates instantiated over 32 company-report groups. Gold answers were withheld during submission, and systems were ranked by macro-averaged item-level ROUGE-1 F1 against organizer-held reference answers. The final leaderboard includes 12 ranked submissions. The strongest systems are closely clustered, with the top four separated by less than one percentage point in ROUGE-1 F1. The submitted system papers document retrieval-augmented generation, cross-lingual evidence handling, structured prompting, answer compression, and validation strategies.
Zhuohan Xie, Xueqing Peng, Georgi N. Georgiev et al.· 1 citation