From Terrestrial Laser Scans to Queryable Robot Knowledge: A VLM-Verified Framework for Incremental 3D Semantic Modeling

Abstract

Indoor service and inspection robots need environment knowledge that is queryable, reliable, and maintainable as the site changes, yet open-vocabulary 3D pipelines propagate unverified labels downstream and build-once scene models offer no maintenance path. We present TLS-SMF, a semantic modeling framework that converts registered terrestrial laser scans into a confidence-aware semantic knowledge graph and keeps it current across repeated scans. TLS-SMF combines (i) a deterministic geometric backbone whose re-runs are byte-identical on the same host, (ii) multi-view vision–language-model verification with automatic escalation that retains unresolved objects as explicitly unverified, and (iii) identity-preserving incremental updates that apply confidence-gated node-level upserts to mint-once node identities. Across two operational configurations of one industrial room, escalation lifts verification accuracy from 77.8% to 88.9% and from 78.1% to 84.4%, with 94.4–100% unanimous-vote precision against confirmed owner ground truth; on the showroom benchmark, structurally gated querying answers 83.3% of questions versus 52.1% for an unverified Stage-A LLM baseline while blocking hallucination traps; in a controlled same-quality combined-edit re-scan, selective re-verification reduces VLM cost to 3.1% of a full rebuild; and a real three-epoch study detects 2/2 controlled physical changes with zero observed identity switches on strong reference correspondences, while re-segmentation still produces false-new/false-absent cases. A cross-floor cafeteria stress scene bounds domain transfer, and an owner-in-the-loop revision path keeps the single semantic store corrigible for robot-facing use; the same store drives a physical mobile manipulator, which completes three console-issued semantic missions with a mean arrival error of 0.30 m while phantom goals are refused with zero motion.