The results support verifier-selected supervision for bounded, machine-checkable capabilities, not arbitrary planning, enterprise validity, or unrestricted self-improvement.
Abstract
Machine-verifiable workflows produce governance records linking a task contract, model attempt, verifier decision, accepted output, and target origin. We test whether these records can supervise bounded models, consolidating occasional or expensive capability into reliable one-shot execution. On fresh, structure-disjoint PlanBench replanning cases, Qwen3-14B thinking generated 24 plans admitted by the independently authored VAL verifier. Those plans trained the same checkpoint for non-thinking execution, without oracle targets or a stronger teacher. On 80 unopened cases, VAL-accepted plans increased from 1 to 57, with 56 paired gains and zero regressions; thinking reached 30. The adapter was schema-valid on all cases and used approximately 1/56 of thinking's mean latency. The separate paired interface-cure gate did not pass. A matched ablation fixed the source cases, 52-candidate pool, 24-target count, model, recipe, and seed while changing target selection. On 160 new cases, base, schema-selected, model-self-selected, and VAL-selected execution reached 1, 55, 69, and 102 accepted plans. VAL exceeded self-selection by paired net +33 (p=0.0000019647), with gains in both difficulty strata. Independent semantic selection is therefore load-bearing relative to matched alternatives within this band. A complementary Phi stronger-teacher arm raised base Phi-4 from 2 to 51 accepted plans and from 35 to 80 schema-valid outputs. Earlier synthetic experiments establish teachability, cumulative learning, construction robustness, and stopping boundaries. The results support verifier-selected supervision for bounded, machine-checkable capabilities, not arbitrary planning, enterprise validity, or unrestricted self-improvement.
Language models now draft, classify and criticise inside research production, yet the artifacts they help produce carry little accountable history. Rather than detecting machine involvement afterwards, we specify an auditability discipline built at production time: git sealing with an anchor lineage, hash-bound provenance, red-line gates that refuse non-compliant artifacts and log every refusal, cross-model role separation, and programmatic assembly from registered sources. Adherence is instrumented by metric cards, each carrying a pre-registered blind spot and evidential standing, frozen before the prospective case it observes. In that case the observed project's pre-registered confirmatory test was executed under seal and returned No-Go, and that project's frozen stopping rule halted the work, against its own operators. A lower-graded retrospective case covers families whose machinery predates the protocol. Current observations are provisional; we release a package from which a third party can recompute every primary metric.
Auto Research uses language-model agents to propose, implement, and evaluate machine-learning changes in a closed loop, but is usually judged by its terminal pipeline. A terminal score cannot reveal which technical decision produced a gain or distinguish a reusable discovery from a change adapted to development feedback. We introduce intervention-centered Auto Research, which validates research decisions rather than only final artifacts and makes their reliability measurable. Feature, Model, Representation, and Data axes are searched independently with inner five-fold feedback. Each axis winner is frozen before an outer-holdout matrix compares all alternatives on evidence the loop never sees. Across 701 agent-executed attempts spanning ten Matbench endpoints, outer evidence confirms the selected intervention on nine of ten endpoints and preserves 89.3\% of non-tied intervention orderings. It also rejects an aggregate Representation gain that inner feedback endorsed. The resulting matrix reveals an information-dependent hierarchy. Composition-only tasks support several routes to improvement, whereas structure-informed tasks favor local geometry features and complementary tree ensembles. A subsequent compatibility test combines already frozen Feature and Model code without further search or tuning and raises mean outer-holdout improvement from 19.0\% to 26.3\%. By validating decisions rather than only artifacts, this design turns adaptive search into reusable evidence wherever agents propose executable alternatives against a fixed evaluator.
Jingjie Ning, Xiaochuan Li, Shan Zhong et al.· 0 citations
An engineering case study of a production low-code workflow on the n8n platform that integrates twelve RSS feeds from five patent offices that identifies five design lessons: P1 (Hybrid Intelligence Architecture), P2 (Format Normalization at Boundaries), P3 (Separation of Content and Presentation), P4 (Graceful Degradation), and P5 (Configuration Externalization), each supported by differentiated within-case evidence.
H. Wang, Hao-Ren Ke· Applied Sciences· 0 citations
This state-of-the-art review assembles that evidence across a cross-disciplinary corpus spanning software engineering, human-computer interaction, labour economics, security research, governance, and education, finding the early benchmarks saturated but task-level capability uneven.
D. Michels, Mutaz Abu Ghazaleh, Francois Lazzari et al.· 0 citations
Legal AI benchmarks, citation checks, and retrieval-grounding tests primarily evaluate upstream capability: whether a model can answer, extract, or ground a legal task. Deployment asks a different question: whether a particular output remains observable enough to be deployed, reviewed, corrected, or escalated once it enters an organizational workflow. We introduce the Workflow Signal Protocol (WSP), a deployment-layer measurement method for recording workflow observability as a structured workflow-observability record. WSP encodes source status, proposition support, review state, recourse, provenance, and role-scoped disclosure. We validate WSP through controlled stress tests, public legal datasets, documented real-world failures, and a live-output pilot using three general-purpose model application programming interface (API) arms. In the main matched-vocabulary stress test, local formal/substantive routing reduced hidden-risk deployment from 93.5% under calibration-only abstention to 4.0% or below; all 192 pilot outputs were expressible as WSP records. These results support the central claim that deployment-time workflow observability is measurable within the evaluated legal-AI settings; validation in deployed institutional legal workflows remains future work.
A reusable engineering pattern for turning exploratory prototypes into auditable applications with versioned source, control, and validation artifacts is presented, which reconstructs this pattern into a traceable, auditable LLM-agent architecture.
