Open access
Jul 2026
Muon Reduces the Training Cost of Regulatory DNA Transformers
Analysis of Transformer models on ENCODE cis-regulatory sequences with Adam and Muon indicates that optimizer update structure and norm-control choices are practical levers for reducing the training resources required to reach matched perplexity targets in regulatory DNA pretraining.
Viraj Doshi, Malhar Bhide, Akshat Singh et al.
· bioRxiv · 0 citations