Fewer Tokens, More Self-Teaching: On-Policy Self-Distillation for Extreme Visual Token Reduction
LT-OPD, a training framework for extreme visual-token reduction, is proposed and it is shown that on-policy learning can substantially recover capabilities lost to extreme visual-token reduction.
Junxian Li, Rui-Xuan Yang, Tian-Ao Zhang et al.
· 0 citations