PromptOps: Iterative Prompt Optimization for LLM-as-a-Judge Systems
This work introduces PromptOps, a closedloop framework for monitoring and iteratively recalibrating LLM-as-a-Judge (LLMaaJ) systems and establishes prompt optimization for deployed LLM judges as an inherently iterative maintenance problem and demonstrates that PromptOps provides a practical abstraction for studying and...