Synthetic Gestures: An Evolutionary Sketching Machine

Abstract

There is growing interest in using pretrained machine learning models for non-photorealistic rendering and sketch synthesis. However, existing approaches typically rely on differentiable rendering, which can be constraining for artists working with algorithmic processes and mechanical plotters. This pictorial introduces a gradient-free, artist-centred method for synthesising sketches from text prompts, designed for use with a plotter. Drawing gestures are parameterised as implicit neural representations and optimised using an evolution strategy, with semantic guidance from CLIP, a vision-language model. The system runs locally on laptops and provides real-time visual feedback for artistic iteration. Its gradient-free design enables flexible, non-differentiable rendering pipelines implemented in the browser using familiar JavaScript graphics toolkits. An early version was exhibited in 2025 during a group exhibition in Tokyo, functioning both as an artwork and a research-through-art investigation into how machine learning systems might better align with artists’ practices.