Aug 2026· The International Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences· 0 citations· 2 references
TL;DR
Chat2Map is presented, an agent framework for automated web map generation from natural-language instructions that integrates skill and tool grounding, runtime schema construction for uploaded GeoJSON and JSON files, and an iterative generate-execute- diagnose-repair loop based on runtime and visual feedback.
Abstract
Abstract. Web map creation remains difficult for non-specialists because it requires the combined use of geospatial data understanding, cartographic design, front-end development, and platform-specific mapping APIs. Although large language models have recently improved natural-language-driven code generation, they still perform unreliably in web mapping tasks, where failures often arise from hallucinated API usage, heterogeneous input data structures, and the gap between code plausibility and rendered map quality. This paper presents Chat2Map, an agent framework for automated web map generation from natural-language instructions. Rather than treating the task as one-shot text-to-code generation, Chat2Map formulates web map authoring as a grounded workflow that integrates skill and tool grounding, runtime schema construction for uploaded GeoJSON and JSON files, and an iterative generate-execute- diagnose-repair loop based on runtime and visual feedback. To validate the framework, we implemented a working prototype with Tianditu as a reference API setting and evaluated it on 30 web map generation tasks covering both directive-based generation and data-driven visualization. The results show that Chat2Map achieved 80% code executability and a 70% task success rate on directive-based tasks, while also maintaining a 70% task success rate on data-driven tasks where baseline models failed to produce usable outputs. These findings suggest that reliable web map generation depends not only on stronger code models, but also on how model reasoning is connected to domain knowledge, explicit data understanding, and real execution environments.
Large language models (LLMs) have demonstrated growing competence in web page generation. However, existing text-driven approaches rely on complex prompts that impose substantial demands on users and offer limited expressivity for page layout and cross-page visual coherence. Image-driven paradigms, which take UI screenshots as input, align more closely with real development workflows. However, current benchmarks focus primarily on visual fidelity and lack a systematic evaluation of the interaction capabilities in generated artifacts. To address this gap, we introduce UI2App, the first benchmark targeting interaction inference, the ability to recover application behavior from screenshots alone, without any textual or behavioral guidance. UI2App comprises 327 screenshots grouped into 45 state-coherent screenshot sets for runnable multi-route web applications. We design an end-to-end pipeline that evaluates each artifact along four dimensions: executability, navigation reachability, visual fidelity, and interaction inference. The interaction metric (IIS) assesses inferred interactions by functional correctness and state-management complexity, crediting any valid implementation rather than matching a single reference. Experiments on six frontier vision-language models reveal a marked capability mismatch between visual reconstruction and interaction realization: the visual-fidelity leader scores only 7.5 on IIS, ranking fourth and trailing the IIS leader by 5.2x. High-complexity interactions such as cross-page state remain a pervasive bottleneck, with half of the evaluated models scoring exactly zero on this dimension. Overall, the results indicate that inferring complete interaction behavior from static screenshots remains a key challenge for models.
Grace Man Chen, Litao Guo, Yifan Wu et al.· 0 citations
Digital Adoption Platforms (DAPs) are embedded overlays widely used on web systems to guide users through operations inside a page, helping them get started with unfamiliar interfaces quickly. Completing a real task, however, rarely means clicking a few buttons on a single page: it takes a sequence of actions that unfolds across changing page states. Prior studies have also treated automated web agent actions and guide text generation as two separate problems, and most of them feed models textual page representations such as the DOM or accessibility trees rather than the rendered screens that humans actually operate on. In this work we introduce MAG, the first benchmark that unifies task execution and guide writing into a single Multimodal Action and Guide task, with two grounding schemes over screenshots: Set-of-Mark element selection and raw pixel coordinates. We further build a complete harness for this compound task, covering annotation with LLM assistance and human verification, training, evaluation in live environments, and joint metrics for actions and guides. With this harness we evaluate frontier API models and open multimodal models, and report detailed analyses. Finally, we design a GRPO training method augmented with expert trajectories, which nearly doubles the success rate of a supervised 9B agent (from 6.9% to 13.2%) and improves guide quality at the same time. Even the strongest model completes fewer than 40% of the tasks, leaving ample room for future research.
Chengguang Gan, Hanjun Wei, Yunhao Liang et al.· 0 citations
This paper presents the design and implementation of intelligent chatbots using Retrieval-Augmented Generation architecture to derive accurate, context-based responses from domain-specific knowledge sources such as PDFs and CSV files. The proposed system integrates information retrieval mechanisms with large language models to overcome crucial limitations of traditional chatbots: hallucination, knowledge cut-off, and inability to access private organizational data. The proposed architecture uses a six-stage pipeline consisting of document ingestion, text splitting, and embedding generation with sentence transformers, semantic retrieval, and response generation via Large Language Models (LLMs). By preprocessing the PDF and CSV documents using a RecursiveCharacterTextSplitter and converting them into 384-dimensional semantic embeddings, the system works via a similarity-based retrieval of document chunks. For implementation, the LlamaIndex framework was used for data indexing and query orchestration, and HuggingFace Transformers for language modeling. Experimental validation conducted on the smart campus chatbot system demonstrated significant improvements in the accuracy, relevance and transparency of responses across education, student services, and administrative support domains compared with a standalone language model. Because of its modular design, it supports a wide range of domain-specific topics without the combinatorial costs of re-training. This is another important application for systems that require regular updates and accurate information. Overall, the proposed framework provides a simple and practical approach for easily implementing domain-aware chatbots.
Varsha Spoorthi Shree Narravula, V. K, Yashaswini B. Suresh et al.· 2026 7th International Confe...· 0 citations
The resulting prototype confirms that a cloud-hosted multimodal LLM, when combined with a minimal and well-structured web stack, can serve as a practical foundation for next-generation digital assistants suitable for customer support, education, and personal productivity applications.
G. Vamsi, Vinay Kumar Male· International Scientific Jou...· 0 citations
Domain ontologies play a crucial role in organizing and integrating heterogeneous knowledge in the construction industry, particularly for the modeling and semantic representation of building codes and standards. Existing ontology construction approaches—including manual, semiautomatic, and automated methods—still rely heavily on domain experts, resulting in high development costs, subjective bias, and limited scalability. Recent large language model (LLM)-driven methods have shown promise for end-to-end ontology construction. However, their direct application to iterative ontology modeling remains challenging. LLM calls over long and dynamically evolving contexts often lead to degeneration and hallucination. Moreover, the inherent stochasticity of LLM outputs, together with the accumulation of structural errors across iterations, can progressively undermine the coherence and reliability of the constructed ontology. To address these challenges, we propose Text2Onto-Agent (T2OA), an agent-based, end-to-end ontology construction framework that formulates ontology modeling as a closed-loop reasoning process. T2OA enables an LLM-based agent to coordinate multiple diagnostic reasoning modules while interacting with a graph-based memory that persistently stores the evolving ontology. This memory provides structured and explicit contextual references across iterations, thereby reducing reliance on long-context prompts, mitigating output stochasticity, and supporting proactive error detection and correction during ontology evolution. A case study in the green building domain demonstrates the effectiveness of the proposed framework. T2OA achieves an F1-score of 0.77 for concept disambiguation, as well as prediction accuracies of 85.22% and 90.35% for parent–child and sibling relations, respectively. Furthermore, T2OA significantly outperforms baseline methods in terms of ontology completeness, highlighting its potential for constructing reliable and domain-adaptive ontologies for building codes and standards. The source code is publicly available at
https://github.com/pipiyapi/T2OA
.
Juntao He, Hainan Chen, Hai Yang et al.· Journal of construction engi...· 0 citations
Building a functional website traditionally demands proficiency in HTML, CSS, and JavaScript, or costly subscriptions to no-code platforms. Sketchify addresses this barrier by providing a canvas-based wireframe drawing interface that converts hand-drawn sketches directly into production-ready HTML/CSS websites using Large Language Models (LLMs). A custom spatial layout interpreter processes drawn elements and classifies them into semantic web components such as navbars, hero sections, cards, and footers. The enriched layout descriptor is forwarded to the Groq API (Llama 4 Scout) to generate single-file HTML output styled in one of six design paradigms: Glassmorphism, Skeuomorphism, Neo Brutalism, Claymorphism, Minimalism, and Liquid Glass. The system further supports multi-page generation, a natural-language post-generation chat editor, a Supabase-backed project dashboard, voice-command input, and a template gallery. Acceptance testing confirmed successful website generation within five minutes of onboarding. Generation latency is under fifteen seconds at near-zero infrastructure cost. Benchmarking against Pix2Code and Sketch2Code confirms superior output quality and design flexibility without task-specific model training.
Harshini Babu, Densy John Vadakkan· 2026 7th International Confe...· 0 citations