Large language models (LLMs) have shown strong potential as training-free text encoders for long-context embeddings. Existing approaches primarily improve information flow under causal attention and typically construct embeddings by uniformly averaging all token representations. However, for long documents, such mean p...
Zi-Feng Cheng, Jie Zheng, Zhiwei Jiang et al.· 0 citations
A Dynamic Distribution-Aware Uncertainty Quantification framework (DDA-UQ) is proposed that shifts the paradigm from static mapping to a dynamic distribution-aware process and significantly outperforms state-of-the-art methods.
Ao Zhou, Zhiwei Jiang, Zi-Feng Cheng et al.· 0 citations
Step-Level On-Policy Distillation is proposed, which combines the long-horizon correction of supervised fine-tuning (SFT) with the on-policy advantage of OPD to provide step-level supervision over complete student-generated trajectories and substantially outperforms conventional SFT and OPD.
Changhui Sun, Lan-Bo Liu, Hang Lei et al.· 1 citation
This work proposes a structured suffix modeling method that incorporates the decoding results from the previous step into the suffix token representations at the current step, allowing them to carry evolving denoising information across generation steps.
Zi-Feng Cheng, Keda Li, Zhiwei Jiang et al.· 0 citations
Class-wise Covariance Regularization is proposed, which aligns the predicted covariance structure of class confidences with the semantic correlations encoded in pretrained text embed-dings with the geometric consistency of the class space throughout fine-tuning, resulting in more stable and interpretable confidence dis...
Ao Zhou, Zhi-Wei Jiang, Zi-Feng Cheng et al.· 0 citations
We use cookies to run the site and, with your consent, for analytics and to show ads.
See our Cookie Policy.
