Kento Kuwataka

Test time training enhances in-context learning of nonlinear functions

The theory reveals that TTT enables the single-layer transformers to adapt to both the feature vector and the link function, which vary across tasks, which creates a sharp contrast with ICL alone, which is theoretically difficult to adapt to shifts in the link function.