Efficient Isolation Forest for Anomaly Detection in System Logs using Behavioral Feature Fusion

Abstract

Detecting anomalies in system logs looks straightforward from the outside, yet logs are voluminous, the signalto-noise ratio is low, and standard Isolation Forest (IF) scores each entry in isolation, with no sense of whose entry it is; a user repeating one command in a loop stays invisible to it. This work addresses that gap with BFF (Behavioral Feature Fusion), which adds two features to standard IF. The first, $H_{\text {cmd }}$, applies Shannon entropy to each user's command sequence over a 60-second window; repetitive loops collapse toward zero entropy, while normal admin activity typically falls in the 2-4 bit range. The second, $\rho_{\text {exfil }}$, is a binary flag combining a keyword threshold with a 90th-percentile traffic threshold, intended to support downstream attack-type triage. Both features are computed entirely from the syslog and pcap files a Linux system already produces. Evaluated on 9,900 synthetic log entries under fold-isolated cross-validation with a contamination estimate independent of any label information, BFF reached $\mathbf{F} \mathbf{1}=\mathbf{0. 6 4}$, a $\mathbf{2 8} \boldsymbol{\%}$ relative improvement over plain IF $(\mathbf{F} \mathbf{1}=\mathbf{0. 5 0})$ at $\mathbf{1 0 0 \%}$ recall on both exfiltration and privilege escalation, with $H_{c m d}$ responsible for most of that gain and the full pipeline completing in 0.33 s on a single CPU core without a GPU.