Beyond Objective Expressivity: Geometry Preservation in Multimodal Contrastive Learning
This work introduces geometry-preserving encoders (GPEs) by directly conditioning the Jacobian through regularization and demonstrating that simple modifications like LeakyReLU activations and residual paths recover these geometric benefits.