Metropolis-Hastings Dominates Importance Resampling for Policy Composition
Post-training a large language model (LLM) often requires exploring trade-offs between multiple rewards, but retraining for each trade-off is expensive. Decoding-time policy composition allows these trade-offs to be adjusted by combining reward-specific policies at inference time. This composition targets a weighted pr...
A. Kurennoy, R. Yarullin, Fergal Reid
· 0 citations