This work evaluates three commit-time guard granularities (global epoch, read-set version, semantic commit predicate), multi-level verification, and model-side gates on three locally hosted quantized model families, and investigates how precisely runtime guards distinguish invalidating races.
Zi-Hao Zheng, Jia-Yu Long, Bai-Chuan Li et al.· 0 citations
The main conclusion is that the next useful step for LLM trading research is not only better agent design, but also clearer reporting standards for execution realism, reproducibility, and evaluation comparability.
Jun-Yi Yao, Zi-Hao Zheng, Bai-Chuan Li et al.· 0 citations
VP-CONTROL, a runtime-assurance design and deterministic benchmark for cost-aware commit gates, is presented, motivating explicit evidence lineage, cost-aware selection, and commit-time enforcement, while exposing the limits of approximate calibration and local-tool generalization.
Zi-Hao Zheng, Bai-Chuan Li, Jun-Yi Yao et al.· 1 citation
confidence estimation and selective prediction for financial named entity recognition (NER) on a three-tier stress test spanning SEC filings, financial news, and general-topic social media as an extreme out-of-domain condition, evaluating a BERT tagger and LoRA-tuned Qwen2.
Zi-Hao Zheng, Bai-Chuan Li, Jun-Yi Yao et al.· 2 citations
An episode-level evaluation protocol for healthcare NLP agents is introduced, supplying a reproducible intermediate evaluation layer between static benchmarks and prospective workflow studies, with an explicit cost-sensitive treatment of missed versus unnecessary escalation.
Jun-Yi Yao, Bai-Chuan Li, Zi-Hao Zheng et al.· 2 citations
The memory-clarification boundary is studied: whether interaction-derived information should be persisted, used only in the current context, re-verified, or clarified with the user, as well as across Claude and Qwen.
It is argued that public tutoring benchmarks can better support positive-impact evaluation by reporting solving-oriented and pedagogy-oriented scores separately and by making disclosure-sensitive, student-agency-preserving criteria more explicit.
This work describes coordinated perturbation as a budgeted subset-selection problem over pairwise observations and introduces an Adaptive Subset Selection Attack (ASSA) as a scalable search heuristic for probing high-impact perturbation sets.
JailMeter is proposed, an evidence-based evaluation framework designed to more faithfully measure jailbreak effectiveness and distill into a small language model, JailMeter\textsubscript{SLM}, which maintains comparable reliability with significantly reduced computational costs.
Qingjia Huang, Jingyu (Jack) Zhang, Jianguo Wu et al.· Annual Meeting of the Associ...· 2 citations· ⚡1
We use cookies to run the site and, with your consent, for analytics and to show ads.
See our Cookie Policy.