Fetch, Frame, and Focus: Retrieval-Augmented Panoramic-Spotlight Memory for Scalable Vector Graphics Generation
Abstract
Current autoregressive generation of scalable vector graphics (SVG) often struggles to maintain structural coherence and geometric fidelity, leading to part omission, layout drift, and repetitive strokes. This limitation arises from their purely text-conditioned decoders that decode SVGs directly from prompts without visual references. To address this, we present Fetch, Frame, and Focus (F^3SVG), a retrieval-augmented framework for decoder-side visual guidance injection in text-to-SVG generation. Specifically, given a prompt, Fetch retrieves relevant reference SVGs via CLIP-based text similarity search, decomposes each reference into ordered path groups, and extracts scene tokens and detail tokens. Frame compresses the scene tokens into a Panoramic Memory that provides persistent compositional guidance, while Focus encodes the detail tokens into Spotlight Memory slots that preserve group-specific structure and are accessed through a lightweight token-conditioned router. Both memories are injected only into the late layers of a frozen autoregressive decoder, preserving single-pass decoding and the backbone architecture. Furthermore, we conduct systematic ablations to identify principles for incorporating these memory modules into cutting-edge models like OmniSVG. Extensive experiments on MMSVGBench show that F^3 SVG consistently improves upon OmniSVG across quality metrics while adding only 257M trainable parameters (3.3% of a frozen 7.6B backbone), and remains competitive with proprietary closed-source models like GPT-5.4. These results suggest that retrieval-guided memory injection is an effective, lightweight, and architecture-preserving route to stronger autoregressive SVG generation.