Model Confidence Under Answer-Preserving Attacks: An Informativeness-Manipulability Frontier
Deployed vision-language systems often gate their answers on confidence, making confidence robustness relevant to oversight. We study confidence readouts under white-box, image-only attacks constrained to preserve the generated answer byte-identically. Under a reachability assumption, an unmovable readout cannot outper...