Evidence-Aware Prompting for Causal Fact Verification with Large Language Models

Abstract

Large language models (LLMs) are increasingly integrated into information retrieval pipelines to reason over retrieved textual evidence and produce final decisions in tasks such as fact verification. However, causal fact verification presents a more specialised challenge than conventional claim verification: models must aggregate multiple pieces of evidence and assess whether the evidence supports a claimed causal relation, rather than merely verifying factual consistency. This setting is captured by CHECKWHY, which, to the best of our knowledge, is currently the only publicly available benchmark specifically designed for causal ''why'' fact verification. Motivated by this specialised problem, we propose Evidence-Aware Prompting (EAP), an evidence-centric framework that separates evidence selection from decision generation to enhance LLM reasoning. EAP models query-conditioned evidence selection as an explicit intermediate step and aggregates decisions over selected evidence subsets, mitigating query-only shortcuts and reducing the influence of irrelevant context. Experiments on CHECKWHY across multiple closed-source LLMs show that EAP consistently outperforms strong prompting baselines and achieves more balanced and robust evidence-grounded decisions for causal fact verification.