Reasoning-Induced Misalignment, where fine-tuning on reasoning data containing no harmful content, including mathematics, code, and problem-solving with chain-of-thought traces can induce harmful behaviors of LLM, posing a serious challenge to the safety of LLM reasoning. Cross-architecture, cross-scale, and cross-data...
Yipeng Zhao, Qi-Shun Yang, Shenzhe Zhu et al.· 0 citations
Hurtful verdict substitution is established, not universally harmful deployed plurality, and a testable but unverified hypothesis for negative process-reward-model weights is motivated for negative process-reward-model weights.
Chain-of-Thought (CoT) reasoning has been shown to improve the performance of large language models (LLMs), yet existing optimization methods largely rely on outcome-based feedback, leaving the logical validity of intermediate reasoning steps largely unverified. To address the gap whereby LLMs arrive at correct final a...
Jing-Yu Hu, Shu Yang, Wei-Ru Liu et al.· 1 citation
A user-centric framework for systematically auditing system prompts in AI systems, AISPA is introduced, a user-centric framework for systematically auditing system prompts in AI systems that examines specific parts of a system prompt and evaluates them along eight dimensions that matter to users.
Xiangning Lin, Shenzhe Zhu, Shu Yang et al.· arXiv.org· 0 citations
Experimental results across three models across both domain-specific and general benchmarks demonstrate that NGFT significantly outperforms existing mainstream fine-tuning methods in both efficiency and performance, while effectively mitigating catastrophic forgetting.
Ze-Yu Wu, Junchao Wu, Shu-Dong Liu et al.· 0 citations
These results show that evaluating only a final checkpoint can obscure both when counterfactual responsiveness changes and what the latent states contribute, and show that high task accuracy can coexist with low counterfactual responsiveness.
Heng-Yu Jin, Shu Yang, Di Wang· arXiv.org· 1 citation
Panel-consensus verification is introduced, which cross-checks a quote against independently collected blind votes, and preserves the positive but statistically inconclusive point estimate for genuine peer information under leave-one-out re-verification, to identify a low-cost attack surface and a concrete defense for...
SelfMem provides an environment with memory tools and feedback signals that allow the agent to explore, evaluate, and refine its own memory strategy, following the principle of teaching an agent to fish rather than giving it a fish.
Shu Yang, Junchao Wu, Derek F. Wong et al.· arXiv.org· 2 citations
LLM judge panels are a standard evaluation tool, but prior work reports highly correlated panel errors: nine judges provide roughly the effective information of two independent ones, and aggregation closes only a small fraction of the gap. A natural remedy--a signal from a different evidence source, e.g., executing a t...
Shu Yang· 0 citations
We use cookies to run the site and, with your consent, for analytics and to show ads.
See our Cookie Policy.
