Embedded ONNX Versus Python Sidecar Inference for Anomaly-Aware Availability Response in Spring Boot Microservices: An Architectural Evaluation

Abstract

Availability-oriented monitoring may require low-latency anomaly inference, but deploying a lightweight model as a separate service adds process, serialization, and network-path overhead. This article evaluates the inference-path architecture used by a prototype availability-response loop; it does not evaluate the effectiveness of a new DoS/EDoS detector or claim production attack mitigation. The same IsolationForest model and telemetry vectors were executed through an external Python/FastAPI sidecar and through ONNX Runtime embedded in a Spring Boot JVM. The prototype also included a LSTM Autoencoder and a bounded rule layer producing SCALE_UP, RETRY, FALLBACK, or NONE. The main Kubernetes benchmark used Docker Desktop 4.73.1 (Docker Engine 29.4.3) with Kubernetes v1.34.3 on a single-node cluster. Embedded ONNX reduced request-response latency from 26.090 ms to 4.839 ms on average, from 62.065 ms to 5.367 ms at P95, and from 76.487 ms to 6.244 ms at P99; calculated sequential throughput increased from 38.3 to 206.7 requests/s. Additional checks covered concurrent load, gRPC transport, cold start, memory footprint, and limited Kubernetes actions. In an extended comparison using 15 runs of 950 s for each active configuration, CPU-based HPA, rules-only, and AI+rules produced overlapping aggregate SLA-violation rates; no aggregate benefit of the ML gate over rules alone was observed under the tested degradation proxies. The evidence supports a narrow architectural conclusion: in-process ONNX is a lower-latency and lower-footprint execution path for the tested lightweight model. Detection quality, superiority over rule-only or established autoscaling mechanisms, and effectiveness against real adversarial traffic remain open validation tasks.