Skip to content
Open access

IIMAS-RAG at SemEval-2026 Task 8: Hybrid Sparse-Dense Retrieval and Answerability-Conditioned Generation for Multi-Turn RAG

2026 · SemEval@ACL · pp. 2744-2753 · 1 citation · 18 references
Computer Science

TL;DR

This paper presents the IIMAS-RAG system, which evaluates multi-turn retrieval-augmented generation (RAG) conversations, a modular pipeline composed of LLM-based query rewriting to transform conversational history into standalone queries, and hybrid sparse–dense retrieval combining SPLADE and Voyage-3-large via Reciprocal Rank Fusion (RRF).

Abstract

This paper presents the IIMAS-RAG system submitted to SemEval-2026 Task 8, which evaluates multi-turn retrieval-augmented generation (RAG) conversations. Our system is a modular pipeline composed of three stages: (1) LLM-based query rewriting to transform conversational history into standalone queries, (2) hybrid sparse–dense retrieval combining SPLADE and Voyage-3-large via Reciprocal Rank Fusion (RRF), and (3) answerability-conditioned generation using GPT-4.1. In Sub-task A (Retrieval), our system ranked 4th out of 38 teams (nDCG@5 = 0.5445), demonstrating the robustness of the hybrid retrieval strategy in specialized domains. On Subtask C (Full RAG), we ranked 13th out of 29 teams (composite = 0.5397). Ablation experiments show that LLM-based query rewriting is the main driver of retrieval performance, yielding a +16.3% relative gain in nDCG@10 over the hybrid baseline without rewriting, while domain-specific prompt variants provide only localized gains on specialized corpora. Generative performance remains sensitive to low-context and partially answerable turns, where the user query lacks sufficient grounding information and the model struggles to either abstain or provide a properly qualified partial answer, explaining the performance gap between retrieval and final synthesis. Our code is available at https://github.com/PLN-disca-iimas/ mtrag_semeval2026 .

Read PDF

Similar papers

Open access Jul 2026

ADAPTIVE MULTI-STAGE VECTOR RETRIEVAL FOR RETRIEVAL-AUGMENTED GENERATION

The Adaptive Multi-Stage Vector Retrieval (AMSVR) framework is proposed, prioritising weighted, drift-resistant composition over uniform fusion, and offers tailored configurations: AMSVR-Scientific (dense + tuned hybrid) peaks at NDCG@10 = 0.7570 on SciFact, while AMSVR-Full (seven stages) targets broader, noisier corpora where Recall@100 matters most.

Samsudeen Alabi Bankole, Yakub Kayode Saheed · 0 citations
Preprint Aug 2026

SciRet: A Compute-Aware Empirical Study of Retrieval and Reranking for Scientific RAG

We introduce SciRet, a compute-aware empirical study of retrieval-augmented generation for scientific question answering over CORD-19. Rather than proposing a new model, we evaluate a fixed scientific RAG pipeline across three corpus scales: 1,034 chunks (1K papers), 5,160 chunks (5K papers), and 15,480 chunks (15K papers). The pipeline combines sentence-window chunking, BM25, BGE-M3 dense retrieval, reciprocal rank fusion, optional cross-encoder reranking, and grounded answer generation. Across these settings, hybrid retrieval is more robust than either sparse-only or dense-only retrieval in our setting, reaching Recall@10 of 1.000 at 1K and 15K. In contrast, an MS MARCO-trained cross-encoder reranker reduces precision on the scientific corpus, suggesting that domain mismatch can outweigh the benefits of stronger query-passage interaction. Generation faithfulness measured with RAGAS increases with corpus scale in our setup. Retrieval evaluation uses pseudo-relevance labels derived from the hybrid system, so we treat the results as controlled comparative evidence rather than a benchmark claim. We release code, indexes, and evaluation outputs to support replication and follow-up studies.

Kaysarul Anas Apurba, Mahade Hasan, Rofiqul Alam Shehab et al. · 0 citations
Preprint Jul 2026

TabRank: Chain-of-Thought Distillation for Table Re-Rankers

The ability to retrieve relevant tables for answering questions is a key task for structured information retrieval. Multi-stage retrieval systems rely heavily on rerankers to refine candidate lists produced by efficient first-stage retrievers. As a result, neural rerankers and LLM-based reranking methods have become increasingly important due to their superior capacity for semantic understanding and reasoning compared to conventional sparse or dense retrieval models. Recently, Large Reasoning Models (LRMs) equipped with explicit chain-of-thought (CoT) reasoning have shown strong improvements in ranking quality in unstructured passage retrieval. In this work, we present TabRank, a framework for training reasoning rerankers for Tabular Retrieval. We first present a comprehensive dataset of 6728 reasoning traces for tabular reranking on the Natural Questions Tables dataset. We then explore two variants of training a compact reasoning model on these reasoning traces: explicit CoT distillation and conditioning the student reranker on the teacher's reasoning trace within the prompt. We stress-test TabRank on several out-of-distribution generalization settings on diverse domains and multi-table scenarios. Our approach significantly improves performance across a variety of table retrieval datasets, increasing Acc@10 by 30.5% on HybridQA, 15.2% on SQA, 52.9% on TabFact, and 13.1% on TATQA subsets of the Multi-Table QA Benchmark compared to the base model. Notably, TabRank generalizes effectively to multi-table reasoning. Our code, data and models are available at https://github.com/AdarshSingh7647/TabRanker

Adarsh Singh, K. Bhandari, Jianxi Gao et al. · 0 citations