Reliability of Uncertainty-Aware Listener-Rating Prediction for Piano Performances Under Distribution Shifts

Abstract

Models can predict listener ratings of piano performances, but their reliability under distribution change remains unclear. We study 19 perceptual and descriptive attributes in PercePiano using valid-only ratings, frozen MERT and MuQ representations, grouped out-of-fold evaluation, and two synthesis pipelines. The primary comparison excludes two reference-rendering categories and retrains composition-LOPO and human-performer LOPO models on the same 1072 human-performance segments. Rank association with mean listener ratings was markedly lower for held-out composition IDs than for held-out human-performer IDs (MERT: 0.203 vs. 0.599; MuQ: 0.206 vs. 0.629). Categorical rating distributions aligned more closely with observed listener dispersion than ensemble uncertainty under proxy-grouped evaluation, although this pattern did not transfer consistently across compositions. Paired renderer tests found positive transfer penalties in all four encoder–source combinations. Score-relative late fusion, conformal calibration, and labeled target-composition adaptation improved numerical error, empirical coverage, and target-specific prediction, respectively. Interpretation is limited by finite-panel disagreement and the small NeuroPiano comparison. These experiments characterize the reliability boundary of listener-rating prediction in this corpus and motivate composition-aware evaluation on broader acoustic-piano data.