Skip to content
Log in

Author

Zebing Wang

View on Semantic Scholar

2 papers indexed here

We haven’t gathered this author’s papers yet. Follow them and we’ll fetch their work.

Not the right person? Other researchers publish under this name.

#natural language process... Preprint Sep 2026

HarnessDev: Can LLMs Create and Evolve Their Own Agent Harness?

As agents move from research prototypes to deployed tools, their capability increasingly depends on model-external execution infrastructure, commonly termed the agent harness. Changing this harness while holding model weights fixed can substantially alter task performance. Current agent evaluations typically report downstream performance under a chosen harness, leaving a model's ability to develop the harness itself comparatively underexplored. We introduce HarnessDev, a benchmark that shifts the unit of evaluation from task outputs to runnable infrastructure. HarnessDev covers two stages. In Creation, the agent starts from a minimal seed and a small number of cases, then builds a complete execution system. In Evolution, it starts from its own created harness and iteratively revises it using downstream execution feedback, with the goal of improving benchmark performance. We then evaluate each constructed harness on capability (task success on held-out benchmarks) and efficiency (execution-token cost). The reported Creation results cover six creator LLMs, four domains, and five downstream benchmarks totaling 2,207 unique downstream instances, with hidden evaluation tasks withheld from development. We find that generated harnesses remain substantially behind mature human-engineered references on code and on search and research, while matching or exceeding the selected references on writing and machine-learning experimentation, with large variation in execution cost. Evolution produces some performance gains, but they are unstable and transfer only partially to held-out tasks. Experiments with a fixed runtime model further show that the gains depend strongly on the model executing the harness, indicating limited transfer across models.

Yu-Hao Wu, Jing-Yuan Zhang, Jia-Jun Shi et al. · 0 citations
Open access 2026

uir-cis at SemEval-2026 Task 12: Mitigating Prior-Induced Hallucinations in Retrieval-Augmented Reasoning via Precision-Oriented Decoding

This paper describes a system for the SemEval-2026 Task 12 on Abductive Event Reasoning (AER). We systematically address the pervasive “over-selection” hallucination pathology in Instruction-tuned Large Language Models (LLMs), where generative models erroneously align distractor options with semantic priors rather than retrieved evidence. The proposed architecture utilizes a 32-billion-parameter foundational model within a Retrieval-Augmented Generation (RAG) pipeline. To combat the hallucination bottleneck and adapt to the strict penalty for incorrect predictions, we propose a Precision-Oriented Decoding (POD) strategy, which tightly couples low-temperature Zero-shot Chain-of-Thought (CoT) sampling with scaled marginalization (majority voting). De-ployed efficiently on consumer-grade hardware via Low-Rank Adaptation (LoRA) fine-tuning, our system achieved a highly competitive average score of 0.802 on the official test set. Through a rigorous three-stage empirical evolution and analysis of the diagnostic failure of an asymmetric logical ensemble, the experimental results demonstrate that epistemic noise suppression is strictly superior to heuristic recall compensation in abductive reasoning tasks.

Chiyao Zhou, Zebing Wang, Kexin Deng et al. · 1 citation