This work develops an optimizer-agnostic, parameter-efficient adaptation method based on Low-Rank Adaptation (LoRA), keeping the pretrained weights stored on analog arrays fixed while training the LoRA weights to adapt to downstream tasks and hardware non-idealities.
Analog in-memory computing (AIMC) offers an alternative for model training by executing matrix operations directly where weights are stored. However, scaling AIMC to train modern deep models remains an open challenge due to severe hardware non-idealities, including physical weights with finite dynamic range and write g...
Zhao-Xian Wu, Tayfun Gokmen, Omobayode I. Fagbohungbe et al.· 1 citation
The Forward-Forward Algorithm (FFA) replaces backpropagation (BP) with layer-wise local contrastive objectives, eliminating the backward pass and the need to retain intermediate activations, yet suffers a persistent performance gap with BP that worsens with depth. This paper diagnoses two structural failure modes: an o...
This paper studies the convergence of stochastic gradient descent when the implemented updates are subject to a persistent and state-dependent bias, in which the desired update is scaled by response functions component-wise, and proposes a gradient-based algorithm, termed Residual Learning.
Zhaoxian Wu, Quan Xiao, Tayfun Gokmen et al.· 0 citations
We use cookies to run the site and, with your consent, for analytics and to show ads.
See our Cookie Policy.