What Shared Prefixes Hide: Trajectory Dropout for On-Policy Distillation
Trajectory Dropout provides a simple mechanism for strengthening token-level supervision across model scales and OPD objectives and consistently improves average performance across teacher--student model pairs of different scales and six mathematical reasoning benchmarks.