OSCAR: Order-aware Scoring and Calibration for AI Rankings
Judge-specific sensitivity is useful for aggregating pairwise LLM evaluations, but its interpretation depends on which systematic presentation effects the ranking model includes. We introduce OSCAR, an order-aware framework for scoring and calibrating AI rankings, and study position as one such effect. In released judg...
You Liu, Yue Liu, Quanchao Lu et al.
· 0 citations