This work presents SemanticXR, the first device-cloud system for real-time, open-vocabulary semantic mapping and querying under XR power, bandwidth, and memory constraints, and key insight is to elevate semantically identifiable objects to first-class units of system design, governing how the system communicates, executes, and manages memory across the device and the server.
Abstract
Semantic mapping is a core service that enables grounded interactions in emerging Extended Reality (XR) applications such as AI assistants. Deploying this capability on mobile XR devices requires a system that is open-vocabulary, real-time, and low-power. Existing approaches are compute-intensive and assume server-class resources. Cloud offloading offers a practical path, but no existing system splits semantic mapping between the device and the cloud, and current approaches do not address how to manage communication, execution, and memory footprint across the device-cloud boundary. We present SemanticXR, the first device-cloud system for real-time, open-vocabulary semantic mapping and querying under XR power, bandwidth, and memory constraints. Our key insight is to elevate semantically identifiable objects to first-class units of system design, governing how the system communicates, executes, and manages memory across the device and the server. Evaluation against a new, aggressive device-cloud baseline shows that object-level system organization improves server-side mapping latency by 2.2x at equivalent semantic quality. Object-level depth-mapping co-design maintains upstream bandwidth under 2.5 Mbps. On the device, an object-level sparse local map with incremental updates and update prioritization enables sub-100 ms query latency for up to 10,000 objects even under network drops, supports tens of thousands of objects within 500 MB memory footprint, and scales downstream bandwidth with map changes rather than total scene size. The system adds only about 2% to idle device power.
Deploying large language models (LLMs) as personal assistants on mobile devices demands privacy, low latency, and offline availability, yet the computational cost of giant models clashes with strict edge-hardware budgets. We argue that this tension cannot be resolved by model compression alone; it requires decomposing on-device intelligence into complementary functional roles. We present SmartRAG, a fully on-device framework that organizes an intelligent assistant around four coordinated modules -- Perception, Memory, Focus, and Thinking. At the core of SmartRAG is EvoNER, a continually learnable named-entity recognizer that incrementally expands its label inventory through teacher-distilled updates, enabling the system to absorb previously unseen entity types without retraining the backbone LLM. Extracted knowledge is stored in MRGraph, a three-layer provenance-preserving knowledge graph, and retrieved at query time through a hybrid pipeline combining graph traversal, lexical matching, and dense semantic search. The on-device LLM is invoked only for high-value semantic operations -- labeling, planning, and answer synthesis -- keeping inference costs bounded. Experiments on four QA benchmarks (TriviaQA, Natural Questions, HotpotQA, MultiHopQA) show that SmartRAG with a quantized 1.7B-parameter backbone achieves multi-hop reasoning performance competitive with models up to 18$\times$ larger, while running entirely on commodity smartphones within practical memory and latency envelopes.
Zhihan Jiang, Meng Li, Shenghao Liu et al.· 0 citations
Esiur preserves object identity across nodes and performs incremental state synchronization without application-level coordination, enabling runtime type definition (TypeDef) discovery and efficient transmission of strongly typed and heterogeneous data without external schemas.
Ahmed Kh. Zamil, Ammar D. Jasim· IEEE Access· 0 citations
A Mobile Reasoning-as-aService (MORES) framework that treats reasoning as a computational service accessible to edge devices over wireless networks, and focuses on implicit reasoning, which achieves an approximately 18% improvement in system throughput over the baseline Soft Actor-Critic (SAC) algorithm.
Guanchen Liu, Hongyang Du, Kaibin Huang· 1 citation
Edge AI is evolving from isolated inference toward long-running services that coordinate model pipelines, data streams, state, and accelerators near users and physical environments. Cloud-native and edge-native platforms offer useful foundations, but their primary control objects–containers, nodes, links, and enrolled sites–generally do not expose the model, data, state, quality, and participation semantics required by these services. This paper presents eAI+, a vision for an edgeAI-native service platform built around three first-class control objects: AI service graphs, dynamic edge resource fabrics, and participant contracts. eAI+ aims to preserve service quality under latency, privacy, reliability, cost, and participation constraints through three coordinated mechanisms. Runtime would select safe execution adaptations based on current workload, environment, and contract signals. Deployment would map service-graph components and prepared fallbacks to heterogeneous resources. PolyLink is the participant-contract module for plug-and-play resource onboarding; it would register contributors, verified resource offers, capabilities, and participation terms. Once a resource is onboarded, it would become available to Deployment for placing eligible service-graph components under the registered contract, while PolyLink would maintain metering, reputation, rewards, and exit events. Migration would transfer only continuity-critical state or control when mobility, overload, policy changes, or contributor lifecycle events invalidate the current placement. This framing treats edge AI as a coordinated service-platform problem across models, data, state, resources, and contracts.
Jiannong Cao, Zhiyuan Hu, Mingjin Zhang et al.· International Symposium on S...· 0 citations
The proposed approach separates the research-infrastructure layer, which exposes and manages distributed resources, from the application layer, where Cyber-Physical workflows are organized according to an Edge-Fog-Cloud pattern in which placement, timing, and data provenance are treated as first-class experimental concerns.
Fabio Orazio Mirto, Giuseppe Tricomi, L. D’Agati et al.· 0 citations