The Proactive Pipeline: Liquid Production Metadata for CG Pipelines

Abstract

As CG productions scale, teams depend on timely metadata to understand how work propagates across assets, shots, and files. Production data evolves continuously—new fields emerge mid-show, workflows change, and dependencies deepen—yet traditional relational systems struggle with costly schema changes and opaque transitive reasoning. This work addresses these challenges at the data-model level rather than through tools or interfaces. We present two complementary approaches: schema-flexible production metadata using JSONB within a validated relational database, and projection of relational state into a live dependency graph via Change Data Capture (CDC). Together, these enable near–real-time propagation, resilience to partial failure, and scalability within a Proactive Pipeline architecture. JSONB enables schema evolution without migrations or consumer breakage while supporting metadata exchange. In parallel, CDC streams relational state into a graph model, where dependency queries such as downstream impact and version reachability become natural traversals, enabling responsive, dependency-aware workflows.