It is revealed that 68% of generated webpages exhibit at least one compatibility issue, underscoring the pervasive reliability concerns surrounding MLLM-generated front-end artifacts and developing XCompat, a lightweight offline compatibility issue detector that combines visual screenshots and the structural DOM tree for analysis.
Abstract
Multimodal Large Language Models (MLLMs) have been increasingly adopted to automate webpage generation from visual designs (e.g., screenshots). However, existing evaluations are limited to visual fidelity assessment under a fixed browser-device configuration. Such a setting overlooks the cross-environment rendering compatibility for real-world deployments. To address this gap, we present the first systematic empirical study of cross-environment compatibility in AI-generated webpages. Specifically, we construct WebCompat, a dataset of 2,032 annotated instances, comprising webpages generated by 8 representative AI tools, each rendered across 9 browser-and-device combinations. We analyze the prevalence of compatibility issues, their user-perceptible symptoms, and underlying code-level root causes. Our findings reveal that 68% of generated webpages exhibit at least one compatibility issue, underscoring the pervasive reliability concerns surrounding MLLM-generated front-end artifacts. The most prevalent symptoms are failures that disrupt the entire page layout (88.3%): pages shrink directly to fit the target screen with too small fonts, or exhibit scale mismatches that produce cut-off content. Failures localized to individual elements, such as image distortion or missing components, are comparatively less common (13.4%). Furthermore, although most MLLMs incorporate responsive design patterns into the generation, they fail to properly implement these codes. Guided by the findings, we develop XCompat, a lightweight offline compatibility issue detector that combines visual screenshots and the structural DOM tree for analysis. It achieves an F1 score of 0.903 on the WebCompat-test, outperforming the existing compatibility checking tools and LLM baselines. All datasets and tools are released to support future research on rendering reliability in MLLM-based front-end code generation.
ProgramDistill is introduced, a benchmark evaluating coding agents on features discovered through interaction with fully functional reference applications, and provides a scalable benchmark with controlled difficulty for evaluating and diagnosing coding agents, and a natural basis for future curriculum-based training.
Jeonghye Kim, Minseon Kim, Young Jin Kim et al.· arXiv.org· 0 citations
WebCraftBench, an interactive benchmark for evaluating web application generation from a software testing perspective, is proposed and experiments show that coverage guidance improves exploration coverage and the model rankings remain stable when the judge model is replaced.
Beyond basic tool use and coding skills, effectively incorporating visual references into agentic generation is equally essential for applying AI agents to real-world productivity tasks. We study this crucial capability through webpage development, a practical and readily evaluable setting in which existing benchmarks...
Yu-Chi Wang, Rongyao Fang, Wei-Hao Yu et al.· 0 citations
LADar is presented, an accessibility auditor that leverages Agentic AI to bridge the gap between static analysis and real browser interaction and successfully detects critical violations that traditional automated tools consistently overlook.
Forough Majidi, Marina Ribaudo, Maurizio Leotta· Proceedings of the 2026 Inte...· 0 citations
This study evaluates the performance and operational practicality of locally deployed large language models (LLMs) as organizations seek cost-efficient, privacy-preserving alternatives to cloud AI. The objective is to generate workflow-aware evidence that supports informed model selection for automation use cases. Four...
Ida Bagus Kerthyayana Manuaba, Juwono· International Journal on Adv...· 0 citations
Evaluating eight MLLMs reveals substantial gaps in generating UI code that satisfies implementation requirements, alongside distinct performance profiles in inference and realization, and shows that MLLMs'visual reconstruction ability does not necessarily imply an ability to generate code that meets these requirements.
Cheng-Wei Shi, Yun-Nong Chen, Ting-Ting Zhou et al.· 1 citation
We use cookies to run the site and, with your consent, for analytics and to show ads.
See our Cookie Policy.