Aug 2026· Journal of Visualized Experiments· Vol 234· 0 citations
Medicine
TL;DR
An automated UI prototyping framework that integrates the Faster region-based convolutional neural network (Faster R-CNN)-based component detection and CIECAM02 uniform color space (CAM02-UCS)-driven perceptual color modeling, enriched with cognitive and visual design principles is proposed.
Abstract
Effective user interface (UI) design requires a harmonious balance between visual appeal, cognitive usability, and layout consistency. However, current automatic UI generation approaches primarily focus on visual appearance or component detection and lack a unified framework that integrates cognitive principles, color intelligence, and structural reasoning. Furthermore, existing methods suffer from limited layout generalization, poor interpretability, static color selection, and inconsistent behavior across screens. In this context, this work proposes an automated UI prototyping framework that integrates the Faster region-based convolutional neural network (Faster R-CNN)-based component detection and CIECAM02 uniform color space (CAM02-UCS)-driven perceptual color modeling, enriched with cognitive and visual design principles. The Faster R-CNN is used to identify UI components and infer hierarchical structures from large-scale interface datasets. An enhanced color generation module analyzes brand or reference images to ensure perceptually uniform, harmonious, and usability-compliant color themes using CAM02-UCS. These outputs are further optimized through cognitive design rules, including Gestalt grouping, Fitts' and Hick's laws, attention-based spacing, and visual hierarchy modeling, to automatically generate refined, task-oriented UI layouts. Experiments conducted on RICO, ENRICO, and Guo's UI Color Datasets show that the proposed system achieves 92.4% component detection accuracy, improves layout clarity and reading order accuracy by 18.7%, and produces color palettes rated 24.5% more harmonious by designers compared to baseline methods. User evaluations also indicate a 31% reduction in perceived cognitive load and a 28% increase in design consistency across screens. These findings demonstrate that combining deep learning-based structural understanding with perceptually grounded color modeling and cognitive design principles produces UI prototypes that are highly efficient, aesthetically coherent, and user-friendly. This framework establishes a novel, end-to-end approach to intelligent and human-centered automated UI prototyping.
Effective color contrast in visual design is essential for content accessibility. While existing tools can identify contrast issues, they often operate in isolation from design workflows or are used as an afterthought. We present ColorA11Y, a system that supports designers in creating accessible content by providing ju...
Alexa F. Siu, R. Jain, Abhinav Kannan et al.· 0 citations
Interactive visual interfaces have become an important means of controlling generative image models, enabling users to manipulate generation through prompts, direct manipulation, and a range of interactions. However, existing techniques are typically presented as independent systems, making it difficult to understand h...
Susie S. Y. Li, Mingwei Li, Remco Chang· 0 citations
Large Multi-modality Models (LMMs) have made significant progress in visual understanding and generation, but still face challenges in visual editing, particularly in following complex instructions, preserving appearance consistency, and supporting flexible input formats. To study this gap, we introduce RISEBench, the...
Xue Yang, Pei-Yuan Zhang, Yilun Zhu et al.· 0 citations
AUV-Bench is introduced, developed in collaboration with professional UI designers around 1,395 executable web interfaces and four tasks: aesthetic scoring, diagnosis, repair, and text-to-UI generation, which exhibits partial aesthetic competence, but still lack the fine-grained understanding and judgment-action cohere...
Zhi-Jie Deng, Ling Li, Jun-Hao Ji et al.· 0 citations
The proposed AEKI framework translates formal art principles into structured, machine-executable instructions and indicates that domain knowledge can be incorporated into LVLM evaluation through a transparent rule-based layer, supporting applications in digital curation, computational aesthetics, and art education.
A post-training paradigm that integrates multimodal alignment (MA) and structural perception (SP) is proposed, MA enhances element interpretation by grounding metadata-defined elements to their visual counterparts, mitigating semantic drift, and SP models layer-aware inter-element spatial relationships to improve hiera...
Yiyang Huang, Zhao-Wen Wang, Simon Jenni et al.· 0 citations
We use cookies to run the site and, with your consent, for analytics and to show ads.
See our Cookie Policy.