3 papers indexed here
We haven’t gathered this author’s papers yet. Follow them and we’ll fetch their work.
Not the right person? Other researchers publish under this name.
In this note, we show that the problem of checking, whether a sequence of bits is ordered, is not doable by 1-head 1-layer transformers but is doable by a 2-head 1-layer transformer. Unlike similar previous results, our results assume the model where transformers have an output MLP.
This work investigates prompt echoing small language models from different families and shows that echoing prompts are likely to have partial overlap with the training dataset but the phenomenon is primarily driven by the model's induction heads.
It is shown that the minimal number of layers a transformer needs to compute PARITY is two, and the open problem asking whether a one-layer transformer can compute PARITY is solved by showing that average sensitivity of a one-layer transformer grows slower than that of PARITY.
We use cookies to run the site and, with your consent, for analytics and to show ads. See our Cookie Policy.
