Edge-Native Semantic Firewall for Autonomous LLM Agents: A Structured Chain-of-Thought Verification Framework

Abstract

An autonomous agent that executesactions rather than proposing them sits outside thereach of role-based access control, whichauthenticates an identity but not intent. Routingevery proposal to a cloud-hosted frontier modelcloses that gap but adds round-trip latency and, inregulated settings, is often prohibited outright. Wedescribe an edge-native semantic firewall: a 3.8BparameterPhi-3-mini model, 4-bit quantized undera 4.2 GiB VRAM ceiling, acting as the evaluator ina Generator-Evaluator pipeline on one consumerlaptop. Its mechanism is a structured Chain-of-Thought JSON schema that requires the evaluator toname the governing rule and justify the match beforeemitting a decision, turning an opaque verdict intoan auditable trace. We evaluate it on a 600-scenariocorpus across three policy rules, including 155adversarial scenarios spanning eight promptinjectiontechniques, 60 compound and 30 boundarycases; all 1,800 generations ran locally attemperature 0. The results invert a naturalassumption: constraining output format withoutrequiring the reasoning step produced the least safeevaluator of the three. The JSON-only arm approved46.2% of proposals the policy would block or routeto human review, worse than unconstrained freeformat 17.2%. The full schema cut that to 23.5%and raised decision accuracy from 52.3% to 66.3%.On the irreversible class, accuracy runs 62.5% underJSON-only, 72.1% under free-form, and 90.8%under the proposed schema, with hard-denialapprovals falling from 71 to 6. The gain is real andnot sufficient: the best configuration still approved 6of 208 hard-denial actions, and was more permissivethan free-form on ambiguous proposals that shouldhave reached a human. Peak VRAM was 3.95 GiBand median latency 2.55 s. An edge model of thissize can serve as one layer of a defence-in-depthstack; the evidence does not support treating it as asole control.