A CNN-transformer multimodal architecture for weakly-supervised audio-visual violence detection
A CNN-Transformer multimodal architecture that extracts per-snippet features from frozen, pretrained CNN backbones, projects each modality through a lightweight temporal 1D-CNN, and fuses the two modalities across the full temporal extent of a video with a cross-modal Transformer encoder is proposed.