QTEA: Ternary LLMs with Sparse Residual Salient Weight and By-Column Optimization
This work presents QTEA, a sub-2-bit PTQ framework that quantizes weights into ternary values and uses salient weights as residual error compensators and develops a lookup-table based kernel that achieves 7.2% faster per-token generation over an FP16 baseline.
Yi-Pin Guo, Arun George, Jie Fu et al.
· 0 citations