Construction of a Small Model Based on Large Model Knowledge Distillation in Anomaly Behaviour Recognition for Intelligent Connected Vehicles

Abstract

This paper proposes an end-to-end “large model annotation—small model distillation—onboard inference” framework to address a critical engineering bottleneck in the field of anomaly detection for intelligent connected vehicles: the tension between the strong reasoning capabilities of large language models and the severe resource constraints of edge computing. Specifically, the LongCat-Flash-Lite model (68.5 billion total parameters, approximately 3 billion activated parameters) is used as the teacher model, which performs three-level semantic annotation (Level 0: normal driving, Level 1: suspicious behaviour, Level 2: anomalous behaviour) on a balanced sample set pre-filtered by a TabNet coarse classifier, using few-shot prompting. Subsequently, the annotations serve as supervised signals to efficiently fine-tune the lightweight Qwen3-1.7B student model via Low-Rank Adaptation (LoRA), thereby transferring the large model’s anomaly-discrimination knowledge to the compact student model. Ultimately, the distilled student model performs inference independently on the onboard side without invoking a cloud-based large model API. In simulation experiments based on real vehicle telemetry data (originally 82.82 million records, downsampled to 1 million records), the proposed method achieved the following key results: (1) In the TabNet coarse-filtering stage, the binary classification accuracy reached 98.62% with a macro-average F1 score of 97.15%; (2) The LongCat-Flash-Lite teacher model annotated 120,000 balanced samples with an average confidence of 0.9174, and the distribution of the three annotation levels was 26.3%, 39.2%, and 34.6%, respectively; (3) After LoRA distillation, the student model’s accuracy improved from 27.50% to 32.10% (an absolute improvement of 4.60 percentage points), the macro-average F1 score increased from 0.1624 to 0.2468 (a relative improvement of 52.0%), and precision and recall improved by 7.49 and 2.83 percentage points, respectively. These experimental results validate the feasibility of transferring knowledge from large models under low-computational-power constraints and provide a new technical pathway and engineering reference for recognising anomalies at the edge of intelligent connected vehicles.