Skip to content
Book Open access

SAGE: Execution-Grounded Adaptive Planning for Egocentric Interactive Agents

Nov 2026 · Proceedings of the 34th ACM International Conference on Multimedia · pp. 17525-17532 · 0 citations · 19 references

Abstract

Egocentric interactive agents must translate first-person observations and constrained user requests into reliable tool-mediated actions. ReAct acquires branch-specific evidence but may over-rely on local observations; Plan-and-Solve decomposes tasks globally but can linearize mutually exclusive branches. We introduce \method, a Status-Aware Grounded Execution framework. \method maintains a unified task state, executes currently supported steps grounded in visual and tool evidence, and replans unresolved requirements before branch actions. The tool branch uses compact planner views, step-specific schemas, and Reporter summaries, while the visual branch applies Staggered Pair Grounding to pair hand-indicated targets with less-occluded temporal evidence. Without task-specific training, \method achieves 78.13% joint success (25/32) on retail1, outperforming the official Qwen3.5-397B-A17B baseline by 34.38 percentage points. It reaches 81.01% ground-truth tool-call recall, compared with 75.95% for ReAct and 69.62% for Plan-and-Solve.

Read PDF

Similar papers

#artificial intelligence Preprint Oct 2026

Ego2World: Compiling Egocentric Cooking Videos into Executable Worlds for Belief-State Planning

Egocentric videos capture how people carry out everyday activities, yet testing an agent requires evaluating the consequences of actions it chooses itself. We introduce Ego2World, a benchmark that turns annotated cooking activities into executable planning environments under partial observation. Its compiler links sour...

Qin-Chuan Cheng, Zhan Gong, Peng-Zhan Sun et al. · 1 citation
#artificial intelligence Preprint Nov 2026

DualWorldBench: Can Agents Plan Deliveries across Symbolic and Grounded Worlds?

DualWorldBench is introduced, a unified benchmark for evaluating agent planning under controlled symbolic--temporal and grounded--topological complexity and is established as a challenging testbed for agents with tighter perception--reasoning--planning integration.

Yang Chen, Bin Wen, Dan Peng et al. · 0 citations
Sep 2026

NavProbe: Evidence-Grounded Reasoning with Active Memory Retrieval for Zero-Shot Navigation

Long-horizon navigation requires an agent to revise its intermediate objectives as evidence accumulates. Full visual histories are costly to process, while compact summaries may omit details needed to reconsider earlier decisions. We introduce NavProbe, a hierarchical zero-shot navigation agent that couples a dynamic s...

Jing-Yang Liu, Su-Jia Yao, Jia-Yuan Gu et al. · 0 citations
Sep 2026

Structured World-State Reasoning for Agentic Robotic Search

WorLDS: World-state Observation and Reasoning for Language-guided Discovery and Search, a framework that grounds reasoning in a persistent graph initialized from geospatial priors and updated by perception, is presented.

Finley R. Holt, Luis A. Pabon, J. Alora et al. · 0 citations
Preprint Sep 2026

X-Planner: Event-Structured Task Planning for Embodied Intelligence

This work presents X-Planner, a planning front-end that addresses both the supervision and representation of embodied reasoning, and describes planning-text quality and downstream execution.

Howard Lu, Shalfun Li, P. Pan et al. · 0 citations

Seek Before You Move: Evidence Seeking for Progress Grounding in Vision-Language Navigation

This work proposes SeekVLN, an evidence-seeking framework that couples semantic progress reasoning with active acquisition of task-relevant observations and introduces Counterfactual Contrastive Policy Optimization (C2PO) for reinforcement fine-tuning.

Zhi-Min Wang, Mei-Yuan Zhu, Duo Wu et al. · 0 citations

We use cookies to run the site and, with your consent, for analytics and to show ads. See our Cookie Policy.