Legal text generation and legal question-and-answer tasks impose stringent requirements on factual accuracy, evidence traceability and normative consistency. Conventional dynamic retrieval-augmented generation methods are difficult to directly adapt to legal task demands such as legal provision citation, terminology standardization and case evidence organization. Following the DRAGIN paradigm, this paper proposes LARIN (Lightweight Adapted Retrieval-Augmented Inference Network for Legal Issues), an adaptive retrieval-augmented reasoning framework tailored for legal scenarios. While maintaining the mainstream dynamic retrieval workflow, LARIN makes targeted optimizations in three key modules: retrieval triggering, query construction and evidence fusion. Specifically, LINDA identifies retrieval trigger points by comprehensively considering uncertainty, attention influence and semantic importance; JUDGE rewrites queries for legal terms and statutory expressions; MERF conducts relevance ranking, redundancy elimination and evidence fusion for legal provisions and case materials. In the CAIL2018 Chinese legal judgment prediction task, LARIN achieves a charge prediction exact-match accuracy of 0.3316, a micro-F1 score of 0.3837, and a precision of 0.3951, while the sentencing exact-match accuracy reaches 0.1575. Meanwhile, its average retrieval frequency is 1.81 and average token consumption stands at 373.15. Experimental results on CAIL2018 demonstrate that legal-oriented adaptation based on DRAGIN improves retrieval-augmented reasoning in this Chinese legal judgment prediction setting while maintaining low retrieval overhead. Cross-dataset robustness on additional legal QA or judgment benchmarks remains to be further validated.
Jailbreak defenses are essential for protecting large language models (LLMs), but they can also introduce secondary costs that weaken model utility. We present a systematic study of these defense trade-offs along three dimensions: performance impact, over-refusal on benign inputs, and inference cost. Rather than treating defenses as a single class, we organize them by operational strategy and examine how different strategies correlate with different side-effect profiles. Across state-of-the-art defense methods, widely used benchmark datasets, and representative open-source LLMs, we find that defenses rarely improve downstream capability, but instead vary in how they trade safety gains against usability and efficiency. In particular, rule-based defenses best preserve task performance, highly conservative self-reflective defenses often increase over-refusal, and multi-round defenses incur the largest runtime overhead. These results provide both a benchmark for evaluating defense side effects and practical guidance for selecting defenses under deployment constraints.
Tong Zhang, Zexin Li, Simin Chen et al.· 0 citations
Coding agents are ranked almost entirely by resolve rate: whether their final patch passes the target tests. Yet two agents can reach the same outcome through very different processes, and a single pass/fail label says nothing about why a run failed or why an accepted run spent extra steps, time, or tokens. This process evidence lives in the trajectory, which records a run's searches, reads, edits, tool calls, validation, and reversions. However, raw traces are heterogeneous and hard to compare across runs. We present TraceProbe, a trajectory-diagnostic framework that recovers what resolve rate hides. TraceProbe normalizes each raw run into a canonical nine-type action taxonomy with deterministic effect labels, then applies two rule-based modules: Insight names single-trajectory anti-patterns adapted from established debugging practice (e.g., search loops, verification skips), while Converge aligns pairs of runs and classifies where their behavior diverges under controlled references. Applying TraceProbe to 2,500 trajectories from five production settings on SWE-Bench Verified, we find that (i) file choice is too coarse to separate success from failure, whereas function selection and completion behavior localize it; (ii) Insight anti-patterns act mainly as corpus-level difficulty clues, with search loops the most stable; and (iii) even resolved runs differ in how quickly they reach relevant code and how much failed work they incur. Trajectory structure thus adds auditable diagnostic context to outcomes by localizing inspection targets, suggesting failure hypotheses, and prioritizing runs for review.
Rui Shu, Chun Yong Chong, Xin Zhou et al.· 1 citation
SWE-RPG is introduced, a repository-level benchmark that combines executable patch evaluation with validated ground-truth references (GTs) for Requirement Clarification and Implementation Planning, and suggests implicit-requirement recovery as a key candidate direction for improving coding agents.