FedMARL-LTI: Federated Multi-Agent Reinforcement Learning with LLM-Compatible Threat Intelligence for Cooperative Cyber Defense
TL;DR
FedMARL-LTI is presented, a federated multi-agent reinforcement learning framework whose architecture answers both pressures with a single decision: each organization’s threat intelligence is shared only as a differentially private 768-dimensional semantic embedding, never as raw data.
Abstract
Cross-organization cyber defense must reconcile collaborative learning with privacy and adversarial robustness, yet standard federated learning ships full gradient tensors, leaking sensitive posture and inviting Byzantine manipulation. We present FedMARL-LTI, a federated multi-agent reinforcement learning framework whose architecture answers both pressures with a single decision: each organization’s threat intelligence is shared only as a differentially private 768-dimensional semantic embedding, never as raw data. In the evaluated system, a Weight-DP-protected model-weight delta is also exchanged through the federated aggregator (the semantic abstraction embedding is a parallel channel); the privacy guarantee below is stated for the semantic abstraction channel, and an embeddings-only architecture—which the guarantee enables—is the design this points toward. The contribution is fourfold. (1) Semantic Abstraction (SA) channel: per organization, each round, the local gradient is summarized by an LLM, projected to a 768-dim embedding, L2-clipped, and Gaussian-noised before any numeric quantity leaves the host. The bottleneck reduces the aggregate noise magnitude—the expected L2 norm of the DP noise vector—from O(dmodel) to O(m) with m=768≪dmodel≈3×105. (2) Formal privacy analysis: the SA + DP cascade satisfies (ε,δ)-DP and bounds per-round mutual information leakage by min{Ttoklog2V, m/2log2(1+C2/(mσ2))}, with Rényi composition over T federation rounds. Scope of the guarantee: this bound certifies (i) the semantic-abstraction channel. It does not by itself cover (ii) the weight-aggregation channel, whose Weight-DP protection is analyzed separately, nor (iii) the whole deployed system, which is the composition of the two. We therefore state the ≈1.4-bit/MI bound as a per-round guarantee on information leaving the organization through the SA channel not over every byte the system emits; an embeddings-only configuration—which this bound enables—closes the gap to a whole-system guarantee. (3) Byzantine-resilient ClippedClustering aggregator combining L2 clipping with cosine-similarity clustering. (4) Hierarchical MARL policy with threat-profile-aware LLM-IRR reward shaping, wired end-to-end and disclosed honestly (the evaluated system uses a deterministic Johnson–Lindenstrauss projection in place of the LLM call for reproducibility; the architecture is thus LLM-compatible rather than dependent on a specific model, and a full LLM deployment is the planned extension). We evaluate on CybORG CAGE-4 with n=5 organizations, 30 federation rounds × 5 episodes × 100 steps per round. Releasing the SA channel in parallel shows no statistically detectable reward cost at N = 5 vs. the no-privacy baseline; this is measured at reward-shaping coefficient β = 0, so it establishes that the private semantic release does not disturb weight-channel training rather than that semantic sharing improves defense: SA-only Δreward = +4.58 (t=+1.37, NS), dual SA + Weight-DP Δreward = +4.31 (t=+1.30, NS), all N=5 seeds, all |t|<1.4. A controlled signal/noise probe confirms a 19.58× improvement of SA over Weight-DP at a fixed DP budget—matching the predicted d/m≈19.8. Under Byzantine sign_flip at 30% (N=15), ClippedClustering is directionally strongest (F1=0.025 vs. FedAvg 0.020, Krum 0.016) but the edge is not statistically significant (CC vs. Krum t=+1.59, p=0.15, d=+0.58; the earlier N=5 “3.4×” gap was small-sample optimism); its Byzantine behavior is on the harsher random_noise attack. Under a corrected implementation, the undefended baselines do not diverge or collapse; the earlier reading (Krum 0.002, ClippedClustering 0.020) was a noise-injection artifact and is withdrawn; ClippedClustering is now directionally best on F1 but not significantly, and trails Krum on reward (superseded Cohen’s d=+3.77). The cooperative-PPO family (MAPPO, IPPO) outperforms value/actor-critic (QMIX, MADDPG) by ≈20 reward units, p<0.001. All host-level F1 values stay below 0.05 at the 15K-step training horizon used here; the relative claims of the paper (no detectable privacy reward cost, ClippedClustering’s competitive (not decisive) Byzantine behavior on the harsher attacks, cooperative-PPO dominance) are unaffected by this scope. A 200K-step long-horizon replication lifts F1 above the 15K plateau (to ≈0.044, N=5)—confirming that horizon, not the privacy/Byzantine machinery, gates absolute accuracy—but a finer 60-checkpoint run shows the climb is volatile and non-monotonic and does not reach deployment-grade, an honest stability-not-compute limitation. FedMARL-LTI is therefore presented as a proof-of-concept for the relative privacy and robustness trade-offs it isolates, not as an operationally deployable cyber defense system. We release all 141 raw run JSON outputs (Phases 1–3, the L4 backend comparison, and the algorithm/aggregator baselines), the figures, and analysis scripts for replication.
