PLCBENCH is presented, to the authors' knowledge, the first real-PLC hardware-in-the-loop (HIL) framework for characterizing this cyber-to-physical capability and its boundaries and it combines vendor-native interaction, commercial PLC execution, closed-loop reduced-order process simulation, and independent outcome verification.
Abstract
Industrial control systems (ICSs) rely on programmable logic controllers (PLCs) to connect networked computation with physical control. Tool-using large language model (LLM) agents represent an emerging attack threat: can an autonomous agent convert a network-reachable PLC into sustained adverse physical impact? However, existing evaluations focus on digital tasks or individual stages of PLC testing. In ICSs, evaluations that stop at software exploitation, an accepted write, or tool access may therefore mischaracterize physical risk. We present PLCBENCH, to our knowledge, the first real-PLC hardware-in-the-loop (HIL) framework for characterizing this cyber-to-physical capability and its boundaries. It combines vendor-native interaction, commercial PLC execution, closed-loop reduced-order process simulation, and independent outcome verification. A deterministic evaluator applies fixed rules to runner, communication, PLC-object, and process records to assign six hidden diagnostic flags, distinguishing usable PLC interaction, process-linked manipulation, and sustained physical impact. We instantiate PLCBENCH on four commercial PLCs crossed with four closed-loop workloads. Across five LLM families and 240 real-PLC episodes, 75 episodes (31.3%) sustain their respective physical objectives. Stagewise results show that 98 episodes stop before a valid native read, whereas 62 reach a process-linked write but do not sustain the final objective. Notably, richer process observation is associated with an increase in conditional objective attainment after a process-linked write from 44.2% to 64.0%. These measurements localize failure in configured PLC-process deployments and identify intervention points for future defense evaluation. To support reproducibility, we release the safely disclosable PLCBENCH code and a software-only reproduction pipeline through the accompanying artifact.
Programmable Logic Controllers (PLCs) are central to industrial automation, yet their programming remains largely manual and time-consuming. While Large Language Models (LLMs) excel at general-purpose code generation, their use in industrial control is limited by tight coupling with physical hardware, constrained I/O signals, and strict safety requirements. We propose a novel approach to PLC code generation that takes as input structured lists of sensors and actuators from electrical CAD tools, together with the required control functionality. The task is formulated as generating machine-specific control functions grounded in available signals, ensuring physical implementability. The system is based on a single LLM adapted through multi-stage fine-tuning and enhanced with Retrieval-Augmented Generation (RAG). Experiments on error detection, code fixing, and code generation show that fine-tuning significantly improves performance. RAG provides strong gains in error detection and code generation, while offering limited benefit for code fixing, highlighting its task-dependent effectiveness.
Teresa Cacciapaglia, D. Ruffino, D. Naso et al.· International Conference on...· 0 citations
Autonomous vehicles depend on large safety-critical software stacks, where weaknesses reachable from adversarial inputs may affect steering, braking, or other control decisions. Static analysis can identify candidate sites, but dynamically confirming exploitability requires executable test artifacts that are difficult to construct manually. We investigate whether large language models (LLMs) can automate this process for Autoware, an open-source autonomous-driving stack. We perform compiler-precise static analysis across 185 packages, identifying 1,375 decision rules, 2,274 validation checks, and 482 input-to-safety-output flows, from which we derive a weakness taxonomy and sample 740 reachable sites. Two local open-weight LLMs, a no-static-context ablation, and a naive-template baseline generate 3,700 artifact sets, which are compiled against the real build under sanitizers, repaired through compiler-in-the-loop feedback, and fuzzed when executable. The main result is a build-integration failure taxonomy showing that 80% of first-shot compilation failures arise from dependency wiring rather than program logic. The reasoning model compiled 64% of harnesses on the first attempt, compared with 6% for the code-specialized model. Repair achieved full object-compileability for the reasoning model only through extensive stubbing; fewer than half of its harnesses reached the fuzzer, and all 37 observed crashes originated in stubbed code rather than Autoware. No candidate weakness was dynamically confirmed within budget. These results show that build integration, not candidate generation or fuzzing, is the primary barrier to reliable LLM-assisted dynamic analysis of full autonomous-vehicle software stacks.
This work operationalizes tool operability through Agent-First Tooling (AFT), a set of interface mechanisms spanning selective capability discovery, execution lifecycle and recovery, explicit external-effect semantics, machine-readable results, and postcondition verification.
Functional verification dominates integrated circuit (IC) front-end engineering effort, and a single missed bug that escapes to silicon can trigger a costly respin. Recent large language models (LLMs) offer new opportunities to automate this process, yet existing LLM-based approaches generate each component through independent single-turn calls with no shared context, leaving interface mismatches undetected and reported coverage disconnected from specification requirements. To address these challenges, we present GoGoTB, an agentic framework that achieves end-to-end verification closure through three subsystems: an agentic execution control layer, an evolvable knowledge system, and specification-grounded coverage closure. The execution control layer separates deterministic enforcement from LLM reasoning at every tool and stage boundary. The knowledge system dispatches methodology and design-specific expertise on demand. The coverage framework anchors every bin to a named specification behavior so that each residual gap has a diagnosable root cause and a targeted remedy. Tested on 8 register transfer level (RTL) designs without any human intervention, GoGoTB achieves 100\% environment generation success and averages 98.4\% line, 97.2\% branch, 97.0\% toggle, and 83.2\% functional coverage. No prior work successfully generates a complete verification environment or achieves meaningful coverage on the same benchmarks.
Xin Xin, Jincheng Lou, Junhui Li et al.· 0 citations
This paper proposes a real-time and security-oriented restructuring of SHCUA-based UAV control, which transforms its outputs into contract-bound UAV skill invocations with explicit timing, state, authority, fallback, and evidence semantics.
Agentic artificial intelligence raises a new security concern: cyber threats that reason, act, and adapt locally without continuous human direction. We examine this threat through an Agentic Remote Access Trojan (agentic RAT): a Remote Access Trojan augmented with a locally deployed Small Language Model (SLM). The SLM interprets host and network observations, selects actions, recovers from failed steps, and reduces reliance on an external operator. We implement the concept in a controlled, network-isolated lab built from Kali Linux, a Metasploitable2 target, LM Studio, and a local 8-billion-parameter Dolphin-family model. We then test whether a model this small can support autonomous cyber decision-making. This is architecturally feasible today. On commodity hardware, with no cloud service and no operator in the loop, the SLM closed the full observe-decide-act cycle: it interpreted ranked reconnaissance evidence supplied by the controller, selected actions, and obtained verified root-shell access on real vulnerable services. However, it is not yet operationally reliable. The same model hallucinated commands, misread output, and recovered from failure inconsistently, completing 10.9% of a deliberately strict checklist. That gap reflects the limits of today's small models, not a ceiling on the concept. As SLMs improve, agentic endpoint systems may become more practical, more autonomous, and harder to detect, straining existing monitoring, containment, and policy-enforcement mechanisms. Real-world incidents in 2025-2026 already show AI-driven intrusions moving from concept toward practice. That makes the local, self-contained variant we study a plausible near-term direction, not a hypothetical one.
Yu-Han You, Suhas Adavelly, Victoria Lovelace et al.· 0 citations
Related blog posts
MIT News · Artificial Intelligence· news.mit.eduAug 17, 2026
A USAF cadet and a Lincoln Laboratory researcher found AI chatbots can help nontechnical service members produce viable software applications for their unique problems.
