Mapping Linux Auditd Telemetry to MITRE ATT&CK Techniques with a Locally Deployed Small Language Model: A Benchmark on the CAM-LDS Dataset

Abstract

Mapping host activity to the MITRE ATT&CK framework usually needs manual work or a cloud-hosted large language model, which raises privacy and cost concerns for small organisations that cannot send their logs off-host. We ask whether a small language model (SLM), running locally and without fine-tuning, can classify Linux auditd records at the technique level of MITRE ATT&CK. We built a deterministic cascade that captures, filters, scores, and aggregates auditd events into session narratives, discarding most records before any model call. Only these narratives reach a quantised SLM (Qwen 2.5 7B, served by Ollama). Candidate techniques are calibrated only on public sources (MITRE ATT&CK, GTFOBins, Sigma), never on the evaluation data. We evaluated the full pipeline on the CAM-LDS dataset, across 29 attack variants, with deterministic and reproducible model outputs under a fixed seed and a temperature of zero. The system reaches a macro-averaged incident-level F1 of 54.2%. Per-variant scores range from 19% to 78%, driven by how much of the attack auditd can observe: scripted attacks leave few visible events, while interactive ones expose far more. An initial pilot with Phi-3 Mini 3.8B produced invalid, hallucinated technique IDs and missed obvious techniques, motivating the switch to Qwen 2.5 7B, which gives the best accuracy–latency trade-off. Our central finding is where the loss happens: the deterministic cascade preserves almost all of the observable on-host activity, so the binding constraints are downstream. The model under-confirms techniques it is given 71% of missed techniques were already available to it and over-broad signatures, not hallucinations, cause most false positives. A local, fine-tuning-free SLM gives a usable technique-level baseline; the ceiling is the model and the catalogue, not the filtering stage.