Multimodal models and agents have emerged to enable AI systems to learn from and interact with the real world more like humans by integrating information across multiple modalities. However, enabling multimodal agents to continually acquire useful information from complex, heterogeneous multimodal data remains a signif...
Ming-Hao Guo, Qing-Yue Jiao, Xiu-Jin Liu et al.· 0 citations
Diagram-to-graph topology extraction aims to extract a graph of entities and their connections from a structural diagram. This task remains challenging for current vision-language models because it requires both fine-grained perceptual grounding and topology-aware reasoning with global consistency. We present TopoBench...
Bang-Wei Guo, Xujiang Zhao, Yan-Chi Liu et al.· 0 citations
Structural diagrams are widely used to represent complex systems and relational information across scientific, engineering, procedural, and spatial domains. Recent vision-language models (VLMs) have become increasingly capable of recognizing diagram elements and reasoning about their content, while complete diagram top...
Bang-Wei Guo, Xu-Jiang Zhao, Sheng-Yu Chen et al.· 0 citations
Multimodal large language models (MLLMs) have shown strong visual reasoning abilities, but knowledge-intensive visual question answering often requires external textual evidence beyond the image and the model's parametric knowledge. Existing multimodal RAG systems commonly rely on Top-$K$ retrieval or reranking, which...
Hang Gao, Wu-Jiang Xu, Zhixing Zhang et al.· 0 citations
Opera is presented, a verbal critic framework that treats each correction as a persistent note, followed until the diagnosed problem is resolved, and achieves the highest mean resolve rate among competitive critic baselines on all three benchmarks.
Kai Mei, Zhi-Yuan Hu, Yu-Tong Dai et al.· 0 citations
Conditional Progressive Pruning (CPP) is proposed, a lightweight pruning framework that fully exploits multi-round MAD and is the first to fully outperform consistency methods.
Ruo-Song Ye, Caiqi Zhang, Jia-Hao Li et al.· 0 citations
This study provides new mechanistic insights into how LLMs internalize OOD challenges, and designs Sparsity-Guided Curriculum In-Context Learning (SG-ICL), a strategy that explicitly uses representation sparsity to schedule few-shot demonstrations, leading to considerable performance enhancements.
SPACE induces two-level programmatic skills from successful trajectories, where subskill boundaries serve as direct chunk-boundary supervision from trajectory-induced programmatic skills, and distilled into a primitive-chunk policy via hybrid on-/off-policy optimization.
Yan-Ting Yang, Can Jin, Jin-Man Zhao et al.· 2 citations
This work presents OpenPM, an auditable point-in-time evaluation framework for LLM portfolio-management agents, and builds a reference agent named the tiered allocator, where typed analysts score candidates, a constructor LLM proposes weights, and a deterministic critic guarantees feasibility.
Xinying Cai, Ming-Hao Guo, Jiahe Liu et al.· 1 citation
Energy-Shaped Visual Prompting (ES-VP), a novel approach that generates image-specific prompts using low-rank initialization and energy-guided dynamic adaptation, achieving superior performance with fewer parameters compared to single-prompt methods, thereby establishing a new benchmark for efficient and generalizable...
Can Jin, Ying Li, Jingchen Sun et al.· 0 citations
We use cookies to run the site and, with your consent, for analytics and to show ads.
See our Cookie Policy.