Wen-Jun Qiu

Let the Neurons Die: Exploiting ReLU-Induced Model Degradation

Rectified linear unit (ReLU) networks can suffer from dying neurons, where units with persistently negative pre-activations produce zero outputs, blocking gradients through their activations. To exploit this failure mode, we present three training-time availability attacks based on data ordering and poisoning. We begin...