Length Generalization for Transformers via Compression
A polynomial length generalization bound for transformers if the authors adopt compressed strings, via a novel connection to power words is shown, which yields a fine-grained analysis of the C-RASP conjecture that resolves contradicting experimental evidence against it.
G. Zetzsche, Hong-Jian Jiang, Andy Yang et al.
· 0 citations