Steered Not Stirred: Effective Reward Tilting using Surrogate Reward Mixing
This paper introduces Mixing And Reward Tilting for INference-time Improvement (MARTINI), which learns a surrogate composition for plug-and-play use with standard inference-time steering methods, and characterizes surrogate information limits and relates population recovery error to ideal-target KL at small tilt streng...