Rubric Rewards from Item Response Theory
Many language tasks have no single answer that can be checked automatically. Rubrics provide criteria for judging responses to these tasks. For reinforcement learning, the resulting verdicts must be combined into a scalar reward. A common approach sums the points assigned to satisfied criteria. Distinct verdict pattern...
M. Yazdani, Yaser Souri, Xi-Ren Zhou et al.
· 0 citations