Measurement Without Validity: The Compounding Reliability Problem in Agentic AI Evaluation
A three-layer compounding validity model is presented that captures multiplicative degradation across task generation, human-simulator calibration, and automated judgment and provides a tractable knowledge-based tool for diagnosing and correcting evaluation pipeline validity before deployment decisions are made.
William Caban
· 1 citation