LARA: A Practical Evaluation of CPU-Efficient Object Detection Using YOLOv8, OpenVINO, and INT8 Quantization

Abstract

The deployment of computer vision systems on resource-constrained computing platforms presents a practical challenge: modern deep learning models can provide strong detection capabilities, but their computational requirements may limit their responsiveness when dedicated graphics processing units (GPUs) are unavailable. LARA (Lightweight Adaptive Recognition API) is a lightweight object detection API designed to provide computer vision inference through a practical CPU-oriented deployment pipeline. This study presents an experimental evaluation of LARA, focusing on whether established model-level and application-level optimization techniques can substantially improve its performance while maintaining comparable object detection accuracy. The evaluation uses YOLOv8n as the object detection model within LARA and examines two complementary optimization levels. At the model and inference level, YOLOv8n is deployed using OpenVINO FP32 and INT8 configurations. At the application level, LARA's request-processing pipeline is evaluated using synchronous and deferred database logging. The experiments are conducted on an Intel Core i3-1115G4 CPU with 2 physical cores and 4 logical processors, without GPU acceleration. Model evaluation uses a 500-image subset of the COCO validation dataset containing 3,504 annotated object instances across 80 classes, with an input resolution of 640 × 640 pixels. In a controlled single-image latency experiment, the OpenVINO INT8 configuration reduced mean latency from 51.774 ms to 21.997 ms, representing a 57.51% reduction and a 2.35× speedup compared with the FP32 configuration. Across the 500-image evaluation, mean latency decreased from 68.579 ms to 23.527 ms, while throughput increased from 14.582 FPS to 42.504 FPS, corresponding to a 2.92× increase in throughput. End-to-end measurements within LARA, including preprocessing, inference, and postprocessing, showed a reduction in mean processing time from 77.705 ms to 34.378 ms. The performance improvement was accompanied by only small observed differences in detection accuracy on the evaluated dataset. The FP32 configuration achieved an mAP@0.50 of 0.557114 and an mAP@0.50:0.95 of 0.401498, while the INT8 configuration achieved 0.554730 and 0.401238, respectively. The application-level evaluation further showed that deferred database logging reduced LARA's mean API latency from 405.52 ms to 246.12 ms, representing a 39.31% reduction, while substantially reducing latency variability.Batch-size experiments using batch sizes of 1, 2, 4, and 8 were also conducted to examine the behavior of LARA's CPU inference pipeline under increasing workloads. These experiments provide an additional perspective on the relationship between batch processing, per-image latency, and throughput on a constrained CPU platform. Overall, the results indicate that practical optimization at both the inference and application layers can substantially improve the measured performance of LARA on CPU-only hardware. OpenVINO INT8 inference produced substantial latency and throughput improvements while maintaining similar observed detection accuracy on the evaluated COCO subset, while deferred database logging reduced application-level request latency. The findings are specific to the evaluated LARA implementation, YOLOv8n model, hardware configuration, dataset subset, input resolution, and experimental settings and should therefore not be interpreted as universal performance guarantees. Keywords: LARA, Lightweight Adaptive Recognition API, YOLOv8, Object Detection, CPU Inference, OpenVINO, INT8 Quantization, Inference Optimization, Edge AI, Low-Resource Computing