Fnu Pramono

TRAPSBench: Vision-Language Models Encode but Fail to Express Epistemic Restraint

This paper shows that Vision-Language Models (VLMs) can internally distinguish when abstention is required, but fail to express it anyway, and introduces Penalized Epistemic Calibration Score (PECS), a new robust metric that requires models to both answer correctly when the outcome is knowable, and abstain when the out...