Ghosted Layers: Unconstrained Activation Alignment for Recovering Layer-Pruned LLMs
This paper derives a closed-form optimal linear operator from a small calibration set to reconstruct the activation discrepancy introduced by the pruned layers, and shows that this solution corresponds to the unconstrained optimum of the alignment objective, whereas LinearPatch is restricted to a fixed basis symmetric...