Bidirectional Preference Synthesis: Learning Prompt-Conditioned Preferences from Boundary Failures
Correction-based offline preference pipelines commonly treat model failures only as rejected responses under the original prompt. This supervision is incomplete for boundary failures: responses that violate the given instruction yet coherently satisfy a nearby intent or constraint setting. We introduce Bidirectional Pr...
Jun-Bo Wang, Li-Dong Lu, Zhuo-Qun Li et al.
· 0 citations