AGR: A Rehearsal-to-Performance Workflow for Programming Audio Gestures in Max

Abstract

Musical Audio Gesture Recognition (AGR) is a Max-based workflow for turning rehearsal-time audio gestures into programmable performance actions. Rather than treating recognition as a fixed classifier inserted into a finished composition, AGR keeps gesture vocabulary design, dataset capture, feature representation, model training, and output routing editable inside the same patching environment used in rehearsal and performance. Built on top of FluCoMa, AGR provides an integrated capture--training--runtime pipeline that combines fixed-length padding without time warping, a local frequency-deviation representation derived from sinusoidal peaks, and deployment patches that expose recognized gestures as OSC, MIDI, or gesture-bound audio triggers. We frame AGR as a programmable artistic workflow: performers and composers define what counts as a gesture, test recognition behavior against rehearsal examples, and revise mappings or thresholds as part of the compositional process. We characterize the workflow across four instrument-specific datasets (clarinet, flute, trumpet, trombone), using recognition behavior and post-trigger processing time as practical checks on rehearsal use rather than as benchmark endpoints. We also discuss deployments ranging from single-instrument prototypes to concert works and a 12-model large-ensemble production. In these settings, AGR is part of the performance patch: gesture labels, training examples, thresholds, and routing decisions can be revised in rehearsal while the concert setup remains continuous.