Simone Alberto Peirone

Learning to Evolve Scenes: Reasoning about Human Activities with Scene Graphs

SG-Ego, a large scale annotation set extending Ego4D with spatio-temporal scene graphs, where relations triplets are consolidated over time into explicit time-evolving descriptions of the scene state is introduced, and GLEN, a graph-based model that operates over scene graph sequences to both align them with textual actions and model their temporal evolution is proposed.