MODEL AND AGENT EVALUATION SYSTEM IN A MULTI-AGENT INTELLIGENT COMPETITIVE INTELLIGENCE PLATFORM: ARCHITECTURE AND METHODOLOGY

Abstract

The article addresses the problem of evaluating the quality of large language models (LLMs) and intelligent agents in multi-agent competitive intelligence automation platforms. The aim is to develop the architecture of a two-level evaluation system featuring an Evaluator Agent with reasoning chain tracing, a control dataset of 30+ use cases, and a validation mechanism on samples of real user queries. The proposed architecture includes Cell Agent, Cell Agent Models, Evaluator Agent, and a results database. A four-criteria verification model is introduced: (a) step verifiability and logical consistency, (b) source correctness and relevance, (c) result correctness, and (d) solution path optimality. The scientific novelty lies in substantiating an integrated evaluation approach unifying model benchmarking and agent tracing within a single architecture.