Knowledge distillation can transfer reasoning from stronger teachers to frozen students through reusable prompts, but avoiding weight updates does not eliminate supervision. Without ground-truth answers, teacher solutions are unverified, and agreement with the teacher can reward shared mistakes. We introduce Knowledge-...
Ying-Chuan Zhang, Hao-Ran Lu, Wen-Xuan Zhong et al.· 0 citations
This review presents a unified Bayesian formulation of KD that formulates teacher predictions as prior information and establishes a rigorous connection to uncertainty quantification, providing a principled interpretation of how teacher information is incorporated into student learning.
Lu-Yang Fang, Hao-Ran Lu, Jia-Zhang Cai et al.· 0 citations
This framework organizes existing methods around five components: explicit state representation, transition structuring, validation and constraint enforcement, search and rollback, and uncertainty management, and yields a diagnostic hypothesis: interventions should be most effective when they target the error or uncert...
Jia-Zhang Cai, Tao Wang, Rui-Dong Zhang et al.· 0 citations
We use cookies to run the site and, with your consent, for analytics and to show ads.
See our Cookie Policy.