ProgV-Trace: An LLM-Driven Verification Framework for Evidence-Grounded Video Misinformation Attribution
Abstract
Video-based misinformation—assembled from authentic footage, fabricated captions, and manipulated speech—spreads rapidly across platforms, yet existing multimodal detectors typically produce only a binary verdict or a post-hoc rationale without determining which specific element across video, speech, and text is responsible for making the narrative false, or to what degree. Current approaches treat misinformation analysis as a single-step classification augmented with explanations, collapsing the structured, multi-step reasoning that human fact-checkers perform into a single inference pass. We formalize LLM-Driven Evidence-Grounded Multimedia Misinformation Attribution (L-EMMA), a task that jointly requires veracity classification, fine-grained falsity grounding, evidence-backed provenance chains, and quantified interventional responsibility scores. To address L-EMMA, we propose ProgV-Trace, a framework in which a large language model generates an executable verification program—a typed operator sequence in a domain-specific language—that decomposes claims, orchestrates evidence retrieval, dynamically expands a temporal-multimodal provenance graph, and plans minimal counterfactual interventions to isolate each element's contribution to the misinformation verdict. A program-to-student distillation stage transfers the LLM's structured reasoning into a compact verifier for deployment without LLM inference. We further introduce a Program-Hard Challenge Protocol with fresh, semantically coordinated, and multi-step-verification samples that stress-test generalization beyond standard benchmarks. Extensive experiments on GroundLie360, FakeVV, and FakeSV demonstrate that ProgV-Trace achieves 87.2% F1 and 71.4% Resp-NDCG, surpassing the strongest baseline by +4.5 F1 and +24.2 Resp-NDCG points—of which a same-model ablation confirms +21.8 points stem from the interventional attribution method itself. The distilled student retains over 96% attribution quality at 196 ms per sample, and responsibility scores agree with human judgments 84.9% on average across five manipulation types.