Aug 2026· International Conference on Advanced Computational Intelligence· pp. 226-231· 0 citations· 25 references
Abstract
The Broad Learning System (BLS) enables efficient closed-form training by solving the output weights in a single ridge-regression step. However, this one-shot full-batch estimator often exhibits a pronounced generalization gap. This paper proposes Batch Weight-Averaged BLS (BWA-BLS), which partitions the training set into K non-overlapping batches, computes an independent ridge estimator for each batch, and averages the resulting output weight matrices. The proposed method preserves the non-iterative BLS pipeline while introducing a lightweight consensus mechanism that stabilizes output-layer estimation. Experiments on multiple benchmark datasets, with results averaged over multiple independent runs, show that the BWA-BLS outperforms the standard BLS and remains competitive with a Bagging-BLS baseline while substantially reducing the generalization gap. A K-perturbation study further indicates that moderate values of K provide the most favorable balance between predictive accuracy and computational cost. These results demonstrate that the BWA-BLS provides an effective and practical enhancement to the standard BLS for improved generalization.
Large language models (LLMs) are typically pretrained in high precision but increasingly deployed with low-precision post-training quantization (PTQ). Recent studies have shown that using weight averaging during pretraining can improve PTQ performance compared with learning-rate decay, suggesting that it might provide...
Han Wang, Tianqi Shen, Zong-Lin Liu et al.· 0 citations
Broad Learning System (BLS) supports analytical training and incremental expansion, but its growth needs guidance on which inputs new blocks should learn from. Weight errors pose a further challenge by displacing learned outputs across class boundaries. We propose HiER-BLS to couple hierarchy-guided representation grow...
Gong-Li Zhang, C. L. Philip Chen, Zhu-Lin Liu· 0 citations
A compute-efficient, two-step hyperparameter transfer framework that estimates optimal learning rates for training large MoE models by transferring them across scaling model widths, and subsequently extrapolating to trillion-token horizons is proposed.
Nayeon Kim, Hojin Lee, Yunju Bak et al.· 0 citations
Online Self-Weighted Fine-Tuning is proposed, a simple method that augments SFT with online, trajectory-level weighting and offers a favorable compute-performance trade-off as a practical approach for fine-tuning small-to-medium LLMs on binary-verifiable reasoning tasks with only 2 online rollouts.
Hai-Quan Wen, Yiwei He, Bei Peng et al.· 0 citations
This work asks how the optimal learning rate and batch size move with model scale, family, and data, and whether one selection rule transfers across them; what LoRA trades against full fine-tuning, and how its rank and alpha set what the adapter can learn; whether validation loss faithfully ranks downstream quality.
Charles O'Neill, M. Jayasekara, Harry B. Partridge· 1 citation
Wave-BLS, a robust broad learning framework that integrates the wave loss function, which is asymmetric, bounded, and smooth, enabling controlled penalization of large errors, is proposed, establishing Wave-BLS as a stable and robust alternative to existing broad learning models for learning under data uncertainty.
M. Akhtar, A. Varshney, A. Quadir et al.· 0 citations
We use cookies to run the site and, with your consent, for analytics and to show ads.
See our Cookie Policy.
