SpiderPTG: Any-to-Many Multimodal LLM
Abstract
Existing Any-to-Any MLLMs (Multimodal Large Language Models) generate only pairwise modalities ('Text + X') within a single response. We propose SpiderPTG for Any-to-Many Modalities Generation (AMMG) that enables the generation of arbitrary modality combinations ('Text + Xs'). First, we introduce the PTG (Planning-Thinking-Generation) structure, where Planning decomposes complex tasks into multi-steps, Thinking generates adaptive signals to guide decoders, and Generation produces coherent multimodal outputs. Second, we design an Interference-proof architecture to address signal interference and semantic degradation in multi-decoder control. Third, we propose a Pseudo-to-Real training paradigm, which progressively transitions the model from learning basic PTG structures on pseudo data to mastering high-quality content generation on real multimodal data. We further construct a comprehensive AMMG dataset with three progressively-enhanced subsets (Any2Many-Pseudo, Any2Many-PTG-Pseudo-10k, and Any2Many-PTG-2k), enabling models to develop sophisticated planning and thinking capabilities. Extensive experiments demonstrate that SpiderPTG outperforms existing methods in Any-to-Any and Any-to-Many generation scenarios.