Spectral variants of low-rank adaptation (LoRA) choose both a subspace and which factor to freeze. We separate these choices by freezing the input factor A or output factor B on the top or bottom singular directions of pretrained weights, with learning rates selected separately. At rank 2, the same-band advantage of fr...
Adnan Slimane Ali, Ayoub Belfatmi, David Ngwe Pouth· 0 citations
MGT-B provides a modest, reproducible local improvement on MATH-500 in the studied configuration, but the effect does not transfer to Omni-MATH and should not be interpreted as a general improvement in mathematical reasoning.
E. Ettifouri, Ayoub Belfatmi, Mahaman Sanoussi Yahaya Alassan et al.· arXiv.org· 1 citation
A training-free decoding controller that combines a degeneration-aware alarm score fusing token uncertainty with explicit verbatim repetition and a calibrated e-process-inspired sequential detector, and finds that non-termination, rather than looping, is the dominant failure mode on GSM8K.
E. Ettifouri, Ayoub Belfatmi, Mahaman Sanoussi Yahaya Alassan et al.· arXiv.org· 0 citations
We use cookies to run the site and, with your consent, for analytics and to show ads.
See our Cookie Policy.