A reusable engineering pattern for turning exploratory prototypes into auditable applications with versioned source, control, and validation artifacts is presented, which reconstructs this pattern into a traceable, auditable LLM-agent architecture.
Abstract
Enterprise large language model (LLM) applications often begin as prototypes whose behavior is carried by prompts and retrieval context. Productization adds requirements for source boundaries, entity routing, answer contracts, and reproducible traces. We present a harness-engineering approach that reconstructs this pattern into a traceable, auditable LLM-agent architecture: deterministic behavior moves into code, manifests, schemas, and validation artifacts around a replaceable composition boundary, while source-backed claims remain the authority for runtime answers. We instantiate it on a public-data slice of five Korean corporate groups (25 listed companies) and evaluate three research questions. (1) The harness preserves its source-grounding, entity-routing, trace, output-hygiene, and recommendation-language contracts across the fixed validation scenarios; a fault-injection control confirms the validators flag deliberately broken contracts. (2) The checks the harness enforces held under model substitution: across three hosted models, they passed on all 270 composition-boundary runs; failures were confined to the model-composed side and were caught and recorded. (3) The code-owned guarantees are load-bearing, not reproducible by prompting alone: holding the model fixed and varying only the enforcement layer, prompt instructions alone let recommendation-language and internal-trace-leakage violations reach the reader, which the harness blocks entirely. A bolt-on external guardrail prevents such violations too but over-refuses, dropping utility to 88/120 where the harness preserves full utility (120/120); in this ablation, only code-owned enforcement preserves both safety and utility. The result is a reusable engineering pattern for turning exploratory prototypes into auditable applications with versioned source, control, and validation artifacts.
A pipeline promoting an AI system publishes records claiming the thing evaluated is the thing deployed and that the evidence licensed the transition, and measures whether those records can express that claim and whether it holds where declared.
Agent systems validate inputs, tool calls, and generated objects. The final package often escapes the same scrutiny. In one DRSS release, the ledger supported 60 points and a failed certificate; the report announced a 100-point Gold Path. Every local gate was green. The package contradicted itself. We study that failure alongside Schema Docs, where similar faults became product contracts, and Brand Shuttle GEO, where evidence is turned into repair work. The result is a candidate Schema-SIP Relational Conformance profile (SIP-RC). It models a release as a graph: claims point to evidence, decisions carry bounded authority, derived artifacts retain their execution conditions and lineage, and published bytes must match the package that was checked. Hard failures cannot be averaged away, and a validator recomputes critical decisions on a separate path. The paper establishes the failure class and shows that several mechanisms are practical. Whether the full profile performs better than existing checks remains an open experiment.
Large language model (LLM) agents in governed organizations must let the persona (instructions, tone, self-presentation) evolve freely, while keeping execution (stateful, audited work) traceable. A single trust domain does not satisfy both cheaply. We present Persona-Execution Separation (PES): persona and execution reside in different trust domains, connected by a governed contract bridge. The persona is singly-homed and may drift; execution is faceless and audited. Status summaries may return; data bodies remain in the restrictive domain except a graded data-loss-prevention (DLP) exception; identity stays continuous. An approval matrix, DLP, and audit enforce the crossing. PES follows from three goals---free drift, execution traceability, and decoupling. Under LLM representational indistinguishability, any single-domain mechanism that meets all three must re-introduce typed change objects, an external gate, and a stable audit anchor: PES rebuilt at higher coupling cost. A development/pilot case in a regulated digital-employee platform records five decisions over one month, each with a rejected alternative. A mechanism check on the shipped implementation found no execution-side re-validation under persona perturbation (five model configurations) and no persona fingerprint on hard-asserted fields. A probe of a recovered pre-separation build found the governed execution path decoupled from the persona by omission, not by construction; a later wiring change could reverse that isolation, which PES makes an audited architectural rule. The pattern applies when multi-user deployment, execution audit, and expected persona churn hold jointly.
Managing enterprise network firewalls is an ad-hoc process today, where administrators must extract policies relevant to their enterprises from thousands of natural language vendor documents and tailor them to their unique context. In this paper, we present Elastispec, a first step towards principled management of enterprise firewall policies with informal and incomplete specifications. We make three contributions: (i) LLM-assisted formalization of vendor documents into a custom domain specific language that precisely captures the rich choices and options using a multi-step consistency-preserving LLM agent; (ii) mapping the DSL to a concrete network environment by correlating diverse and possibly imperfect enterprise data sources; and (iii) an interactive auditor that cross-checks firewall configurations against the formal but potentially partial specifications and reports potential compliance gaps along with conjectures for human validation. Evaluations with real-world enterprise firewall configurations and popular enterprise application vendor documents show that Elastispec is effective in enabling operators to audit their configurations against vendor documents by producing compliance trees, enabling comparative analysis across parallel application deployments, and detecting configuration errors that permit non-compliant traffic.
Chenan Wen, Yizhan Qing, Curt P Jansen et al.· Proceedings of the ACM SIGCO...· 0 citations
This paper proposes SafeKeep, an inference-time safeguard that decouples safety judgment from tool execution: it assesses requests using flattened textual tool specifications while retaining the original schema-formatted specifications for execution.
Minghui Pan, Jiayuxuan Yang, Yuanyuan Yuan et al.· 0 citations
: Integrating Large Language Models (LLMs) into production-grade business systems requires a clear boundary between deterministic application logic and non-deterministic AI behavior. This paper introduces a runtime architecture that extends the BAB (Business Application Builder) framework on Jakarta EE to support agent-augmented applications hosted alongside ontological-driven generated cores. The core of the solution, the AgentRuntimeManager , moves beyond simple API wrappers by providing a robust environment for agent lifecycles, asynchronous event dispatching, and error handling through configurable retry policies. To ensure system stability, we decouple LLM providers (such as OpenAI and Ollama) using an abstraction layer built on LangChain4j, allowing for runtime provider switching without redeploying the core logic. We demonstrate the practical utility of this architecture through a self-healing log-monitoring chain. In this setup, agents intercept high-severity log events and, through a structured analysis service, propose corrective actions. A specialized SqlErrorHealingAgent illustrates how LLM-proposed fixes can be filtered through layered safety constraints, such as the automatic blocking of destructive SQL operations like DROP or TRUNCATE, before any action is committed. The result is an architectural pattern that lets model-driven rigor and LLM agent flexibility coexist in enterprise environments without compromising either.
Borivoj Bogdanović, M. Segedinac, Z. Konjović et al.· SINTEZA· 0 citations