ZOMP: Zeroth-Order Multi-Modal Prompt Tuning for Vision-Language Models
This work proposes ZOMP (Zeroth-Order Multimodal Prompt tuning), a query-efficient, fully forward-only method that tunes deep prompts in both the vision and text branches of a frozen CLIP model using simultaneous perturbation stochastic approximation.