Toward Codec-Native Interfaces for Wearable Video-Language Systems

Abstract

Continuous video-language reasoning on wearable devices requires the edge to preserve useful visual evidence before the user query is known. Current pipelines often transmit sampled RGB frames, while standard codecs already organize video into I-frame appearance anchors and P-frame temporal updates. This paper explores whether such codec structure can serve as a lightweight edge-server interface for wearable Video-LM systems. We present EventRetina, a preliminary codec-native split-stream prototype that transmits low-frequency I-frame anchors and saliency-selected P-frame event deltas. The edge selects informative P-frames using codec-side motion and residual statistics under a bandwidth budget, while the server maintains the received anchors and deltas as visual evidence for later question answering. On the Perception Test dataset, EventRetina improves the bandwidth-accuracy tradeoff over I-only streaming and random or uniform P-frame selection. At a 20% P-frame budget, it reaches 46.3% accuracy while transmitting 0.80 Mbps. Under matched temporal evidence, it also reduces server-side visual processing cost compared with repeatedly encoding selected RGB frames. These results suggest that codec-native split-streaming is a promising design point for bandwidth-limited wearable Video-LM reasoning.