Zhi-Wei Liu

Where Position Matters: Profiling-Guided Layer-Wise RoPE Allocation in Transformers

This work proposes profiling-guided layer-wise RoPE allocation, assigning high/low coverage under a fixed budget using early coverage-benefit profiles, and normalized-depth mapping transfers profiles within model families, enabling small-model measurements to guide larger-model allocations before pretraining.