Chenyang Dai

A High-Performance and Low-Resource Reconfigurable Hardware Implementation for Softmax in Transformer

The efficient implementation of the softmax is critical for optimizing transformer hardware accelerators. Unlike its role as a static, one-time classifier in convolutional neural networks (CNNs), softmax in transformers is core to achieving dynamic contextual awareness, generating attention weights that enable the mode...