Adewale Akinfaderin

Super Weights in LLMs and the Failure of Selective Training

It is established that parameter importance does not imply parameter trainability in isolation, and that effective fine-tuning relies on structured decompositions over entire layers rather than targeting individually important weights.