Vision Transformers Learn Word Meaning from Images Alone
Can a vision model learn what words mean from pixels alone? We show that the two largest DINOv3 vision transformers organize words rendered in images by word meaning, despite being pretrained using self-supervision on images only, without paired captions, class labels, or other language supervision. We derive word repr...
Anders Gjølbye, S. R. Mason, Stella Frank et al.
· 0 citations