We have 4 of 17 papers
We haven’t gathered this author’s papers yet. Follow them and we’ll fetch their work.
Not the right person? Other researchers publish under this name.
Rotary Position Embedding (RoPE) encodes token positions by rotating each two-dimensional channel of the query and key vectors at a channel-specific frequency, making the attention logits invariant to a common shift of positions. However, this rotation is periodic, and it leads to position aliasing where relative posit...
A key consequence is that the optimal learning rate depends only on the number of unique layers $L$, not on the loop count $N, enabling direct hyperparameter transfer from small to large $N$ without retuning.
It is shown that existing hallucination detection methods, such as confidence-based filtering and inner-state probing, fundamentally fail in the presence of spurious correlations, and further elucidates why these statistical biases intrinsically undermine confidence-based detection techniques.
Results show that looping can improve Transformers even under budget matching, offering a practical recipe that turns depth reuse into measurable gains.
We use cookies to run the site and, with your consent, for analytics and to show ads. See our Cookie Policy.
