From threat intelligence to decision theory

Abstract

A Network Intrusion Detection System (NIDS) watches network traffic and decides whether what it sees is ordinary activity or an attack. Modern systems learn that decision from labeled examples and routinely report accuracies above 99 percent. That number does not answer the question a defender actually faces, which is not how often the detector is right but what posture to run: how sensitive to make it, whether to spend a fixed budget on adversarial retraining, whether to replace an architecture. That choice is made in advance and has to hold against everything an adversary might do. Game theory cannot answer it, because computing an equilibrium assumes each side knows the other's payoffs and a defender does not. Adversarial Risk Analysis (ARA) removes that assumption by letting the defender hold a probability distribution over the attacker instead. Applied to open-set intrusion detection it becomes Adversarial Risk Analysis for Open Set Intrusion Detection (ARA-OSID), a framework of ten utility parameters covering both attacker and defender costs. All ten had been assumed rather than measured, so its recommendations could not be reproduced or audited. This thesis derives all ten from the MITRE Adversarial Tactics, Techniques, and Common Knowledge (ATT&CK) v16 knowledge base with zero learned parameters and no new data collection. Attacker effort comes from required privilege levels and sub-technique counts, detection probability from Detection, Denial, and Disruption Framework Empowering Network Defense (D3FEND) countermeasure and data source coverage, resource cost from kill-chain position, and benefit from campaign severity. Defender parameters come from group usage frequency, severity weighted by the detection gap, per-tactic disruption weights, defense-evasion sub-technique counts and mitigation portfolio size. The structured data comes from a hybrid Long Short-Term Memory (LSTM) and Markov forecaster built for this work, which predicts an attacker's next technique correctly 86 percent of the time and generates 4,849 tactic-ordered attack chains from 33 documented campaigns. Running the mapping exposed two modeling errors, both corrected here. Threat probability has to be dropped on the attacker side when scoring a specific chain, because the chain's existence already establishes that its techniques are in use, and keeping it suppresses rare but devastating techniques. An affine calibration then restores the score scale. Validated against operational severity ratings across 4,849 attack chains from 33 campaigns, the framework reaches a mean absolute error of 0.71 on a 0 to 10 scale with correlations near 0.80, moving by less than 0.02 across three data splits. A sensitivity analysis over 540 evaluations shows the result is driven by the defender's own costs and stays robust to 50 percent perturbation of every attacker-side parameter, which matters because attacker capability is the hardest quantity in security to estimate. Two preliminary studies establish why better detection alone is not enough: the most accurate architecture on ordinary traffic collapsed to 26.8 percent under a one percent perturbation, a reversal named here the False Champion Problem. Every number behind a posture recommendation is traceable to a named field in public threat intelligence that anyone can check.