Atik Mahabub

FPGA-Based Softmax Approximation: Analytical Cost–Error Modeling and Pareto Optimization

The softmax function is a significant bottleneck in FPGA-based Transformer inference. This paper presents a comprehensive analytical framework for the design and optimization of base-2 softmax on FPGAs through systematic design-space exploration. We derive a high-accuracy LUT6 cost model that captures all major pipelin...