Existing long-document QA benchmarks rarely combine large-scale human-authored questions with detailed visual evidence annotations, limiting diagnosis of evidence localization and interpretation failures. We present VELD-QA, a large-scale benchmark for Visual Evidence grounding and question answering in Long Documents....
Zhivar Sourati, Marianne Menglin Liu, Rong Wang et al.· 0 citations
Answering questions over heterogeneous data means combining evidence from files, databases, documents and the web. Data agents automate this, but their success depends on context. Three context-engineering techniques (CETs) help by shaping it: semantic layers describe the data, orchestration splits work among agents, a...
Zhivar Sourati, Marianne Menglin Liu, Zheng Wang et al.· 0 citations
PARITY (Paired Assessment with Reused Identity), a synthetic, demographically balanced set of identity-controlled neutral/expression portrait pairs with neutral-neutral controls, is introduced, finding a clear split between refusal-first models and speculative models; in the latter, certain expressions disproportionate...
Karan Dua, Amit Agarwal, Hitesh Laxmichand Patel et al.· 0 citations
PLSQLBench is the first benchmark for evaluating whether LLMs can write executable PL/SQL programs, with correctness measured through execution-based tests, and recurring difficulties in schema grounding, PL/SQL dialect fidelity, procedural control flow, exception handling, and cross-turn consistency are revealed.
Marianne Menglin Liu, Leonid Boytsov, Daniel W. Peterson et al.· 0 citations
We use cookies to run the site and, with your consent, for analytics and to show ads.
See our Cookie Policy.
