Abdul A. S. Awwal

Entropy-Preserving Linear Unit (EPLU): A Robust Activation Function for Stable and Efficient Training of Deep Neural Networks

Experimental results demonstrate that EPLU consistently outperforms state-of-the-art activation functions, including ReLU, Swish, and GELU, achieving 1~2% improvements in classification accuracy, and detailed analysis of training dynamics reveals that EPLU maintains more stable gradient norms, leading to improved conve...