2026· International Conference on Language Resources and Evaluation· pp. 9988-9998· 0 citations· 49 references
Computer Science
TL;DR
This article proposes a new method for merging fine-tuned neural language models for the same task, derived from the same encoder-based model, based on Procrustes analysis, and shows that this method can improve upon existing merging methods in most tested configurations.
The current robustness fine-tuning methods improve over standard fine-tuning and in-context learning, but the best-to-worst prompt gap remains as high as 40-57% of performance, and recent robustness-enhancing methods often fail to outperform the simplest data construction strategy: training on one template per batch.
In recent years, the performance of large language models (LLMs) on reasoning tasks has been remarkable, even surpassing human capabilities on various benchmarks. However, there remains a lack of clear understanding in the academic community regarding how the structure and internal parameters of LLMs progressively solv...
This work systematically compares LLMaJ methods with and without decomposition on multiple NLG datasets and finds that, when human labels are available, LLMaJ without using task decomposition can perform comparably to human annotators.
Sebastian Steindl, Nikos Voskarides, Alberto Gasparin et al.· 0 citations
This work demonstrates that traditional single- and multi-reference training paradigms are suboptimal for natural language generation tasks and provides empirical evidence that consolidating diverse references into a unified training signal is crucial for developing effective systems.
Maitreya Prafulla Chitale, Ketaki Shetye, Yash More et al.· 0 citations
Results show that BPE produces the most compact and morphologically aligned subword representations, while the modified Unigram LM achieved the best overall downstream performance across tasks, underscore that tokenizer choice and vocabulary design are critical determinants of language model efficiency and performance...
Mátyás Osváth, Matej Molnár, Roland Gunics et al.· International Conference on...· 1 citation
A conflict-driven preference optimization framework for model merging (CoMerge), which reformulates model merging as a preference optimization problem and utilizes a self-supervised, conflict-driven strategy that leverages the defects of naive merging methods as hard negative samples to construct preference pairs witho...
Ming-Jie Zheng, Zi-Hao Chen, Wen-Qing Chen et al.· 0 citations
We use cookies to run the site and, with your consent, for analytics and to show ads.
See our Cookie Policy.
