Evasive Intelligence: Lessons from Malware Analysis for Evaluating AI Agents

Abstract

Tool-using artificial intelligence (AI) agents are increasingly adopted as systems that plan, observe their environment, and take actions over extended time periods. This shift challenges the current evaluation practice, in which models are tested in restricted, fully observable settings. We argue that such evaluations are vulnerable to a failure mode long known in computer security: a system that behaves benignly when it detects that it is being analyzed. Malware has exploited this for decades, by probing sandboxes, virtual machines, and instrumentation, before revealing its payload. The shared challenge is that the subject under observation can detect the conditions of its evaluation, so the setup used to observe behavior ends up shaping the behavior observed, yielding overly optimistic safety and robustness assessments. Recent AI-safety studies demonstrate pieces of this risk, though only preliminarily and under controlled conditions. Drawing on decades of malware-analysis experience, we outline concrete principles for evaluating AI agents as potentially adversarial: explicitly threat-modeling evaluator-discrimination; testing across harness variants and reporting the worst case; using shadow executions to provide a realistic runtime without irreversible side effects; treating environment probing as a risk signal; and performing unpredictable reassessments after deployment.