Signed Rescue Routing: Harm-Aware Cascades for Efficient LLM Inference
Signed Rescue Routing is introduced, a budgeted routing method that predicts these two events separately and ranks requests by their difference and shows that predicting incremental value, rather than model uncertainty, is a simple and effective objective for efficient LLM cascades.