VANTAGE-Bench is introduced, a benchmark measuring the "Infrastructure AI Gap", which spans three operational domains, unifies image and video evaluation across semantic, spatial, temporal, and spatio-temporal capabilities, and moves beyond multiple-choice to eight task formulations including dense captioning and spati...