Omni-Embed-Mini: Binding Modalities Without Forgetting via Dense Distillation
This work presents Omni-Embed-Mini, a 0.9B-parameter model that maps text, speech, audio, images, video, and visually-rich documents into a single shared cosine space without updating any text-side parameter, and is competitive with the closed gemini-embedding-2, edging ahead of it on the overall-modality average.
Mohammed Irfan Kurpath, Jaseel Muhammad Kaithakkodan, Sahal Shaji Mullappilly et al.
· 0 citations