Disrupted Harmony: How Visual–Audio Incongruence Influences Creativity in Collage Making

Abstract

A growing body of multimodal creativity support systems has explored the use of audio–visual integration to enhance creative engagement, typically by providing semantically congruent auditory feedback based on visual elements to support creative expression. However, little is known about how audio–visual incongruence may influence the creative process. Grounded in Koestler’s theory of bisociation, which posits that creative insight arises from holding two habitually incompatible matrices of thought in productive tension, we use traditional collage-making as a medium and design a multimodal setup that responds to users’ visual compositions with either semantically congruent or intentionally divergent sound cues. We conducted a within-subjects user study (N = 16) to examine how congruence versus incongruence shapes both creative experience and emotional responses. Our findings suggest that different forms of audio–visual alignment exert distinct influences on creative practice: congruence tends to support stability, immersion, and confirmatory decision-making, while incongruence can prompt reinterpretation, de-habituation, and exploratory thinking. These results offer exploratory empirical insights and inform the design of multimodal creativity support systems by highlighting audio–visual congruence and incongruence as tunable parameters for balancing coherence and creative disruption.