Skip to content

BudgetVerify: Budget-Tiered Verification for Financial QA

Sep 2026 · 1 citation · 22 references
Computer Science

TL;DR

BudgetVerify is proposed, a budget-tiered generator-verifier framework that routes each generated answer to one of three verification tiers: no verification, lightweight check-and-revise, or higher-cost solve-first-then-compare verification.

Abstract

Financial question answering often requires precise numerical extraction, unit handling, and arithmetic over tables and text, but applying expensive verification uniformly wastes test-time compute. We propose BudgetVerify, a budget-tiered generator-verifier framework that routes each generated answer to one of three verification tiers: no verification, lightweight check-and-revise, or higher-cost solve-first-then-compare verification. The router is trained from offline correctness and token-cost outcomes and, at test time, selects a verification tier using information available before verification, including the question, context statistics, the generated answer, and associated generator metadata. The selected tier either returns the generated answer directly or invokes the corresponding verifier. Across six commercial and open-weight base models, BudgetVerify consistently produces more efficient accuracy-cost Pareto frontiers than fixed verification policies by selectively allocating stronger verification only when it is useful. Although absolute performance varies across models, these efficiency gains and the resulting qualitative frontier shape are consistent across generator models.

View source

Similar papers

Preprint Oct 2026

Token-Budgeted Escalation for Financial Document QA: Cost Is Predictable, Benefit Is the Bottleneck

Retrieval-augmented generation systems can route difficult queries to deeper context, but batch deployments must allocate a shared token budget across calls whose costs vary by query. We formulate selective escalation as finite-batch allocation for financial document question answering. Each of 150 FinanceBench questio...

Jun-Ru Zhu, Yi-Xin Yang, Xiao-Qing Ding et al. · 0 citations

Test-Time Scaling via Budgeted Multi-Attribute Verification

Verifying LLM-generated answers under a shared computational budget requires jointly deciding which candidates to inspect and which verification attributes to evaluate. We formulate this problem as multi-attribute good-arm identification under a global budget: each candidate is an arm evaluated along several costly att...

Bo Xue, Ji Cheng, Shenghao Lyu et al. · 1 citation

FinFIRST: Benchmarking Search Agents for Financial Information Retrieval, Sourcing and Traceability

FinFIRST is the first financial benchmark to jointly evaluate answers and supporting evidence through atomic rubrics, retaining final-answer correctness as the primary objective while making the supporting research process measurable, verifiable, and diagnosable.

Wen-Qing Wang, Hai-Tao Xiang, Xin-Yi Zhao et al. · 2 citations
Preprint Oct 2026

How Should a Prompt Optimizer Spend a Tight Budget? BudgetAPO with Noise-Adaptive Evaluation

Automatic prompt optimization (APO) has been widely employed to adapt large language models without updating their weights, yielding promising results. However, existing methods such as GEPA and OPRO assume hundreds to thousands of subject-model calls, far more than is practical behind paid, rate-limited APIs. Under ti...

Hao-Yue Liu, Zhi-Chao Wang, Huan-Yu Yan et al. · 0 citations
#artificial intelligence Preprint Sep 2026

VEX-Bench: Benchmarking Verification Complexity of LLM-Generated Misinformation

The VEX-Bench is introduced, a unified benchmark for evaluating the verification complexity of LLM-generated misinformation, as perceived during screening, across models and generation methods, and shows that no single method dominates all dimensions, underscoring the need for multi-dimensional evaluation.

Hanxun Huang, Yu-Tao Wu, Qi-Zhou Wang et al. · 0 citations
Preprint Aug 2026

Stopping and Routing LLM Judge Panels

LLM evaluation pipelines often have many candidate judges: general LLM-as-a-judge prompts, reward models, safety classifiers, confidence variants, and task-specific verifiers. The deployment question is not only which judge is best, but which judges should be called, on which examples, and when panel construction shoul...

Bin Zhu, Yi Xie, Yanghui Rao · 2 citations

Related blog posts

MIT News · Artificial Intelligence Sep 29, 2026

Who we become when we talk to machines

Professor Sherry Turkle’s new book, “Artificial Intimacy,” offers a withering critique of chatbots and the antisocial dynamics she believes they encourage.

We use cookies to run the site and, with your consent, for analytics and to show ads. See our Cookie Policy.