This work compares Jev-1.13.13.0 and two other decision models with LLMs on the RANIntent v1 benchmark, in closed-loop ns-3 simulation and on a real A1 and E2 path, and asks whether the extra delay of LLMs costs control deadlines, RIC capacity, or radio performance.
Abstract
Intent-based Open RAN needs an interpreter that turns intents into A1 policies within the loop of the RAN intelligent controller (RIC). Decision models such as Jev-1.13.0 return typed policy fields, whereas generative large language models (LLMs) produce the policy token by token. We ask whether the extra delay of LLMs costs control deadlines, RIC capacity, or radio performance. We compare Jev-1.13.0 and two other decision models with LLMs on the RANIntent v1 benchmark, in closed-loop ns-3 simulation and on a real A1 and E2 path. Median interpretation takes 0.286 to 2.35 s, against under 25 ms for A1 and E2 transfer. Jev-1.13.0 meets the 1 s near-real-time budget on 99.8% of calls, while two hosted LLMs meet it on 17.9% and 0%. In the radio network, ideal enforcement moves the affected-class service-level agreement (SLA) violation by 3.96 percentage points in the direction each intent requests, against no update at the base point. No hosted LLM showed a resolved increase over Jev-1.13.0 at that point. At the same point, per-second direct control gave no resolved SLA reduction over a numerical xApp. Slow interpreters miss the 1 s budget, and two interpreters saturate their queues at 2 intents/s, whereas no radio penalty of slow interpreters was resolved at the base point.
Jev's decision-oriented application programming interface (API) is integrated into edge service orchestration to reduce overhead while retaining service completion, and decision-model substitution for latency-bound admission on bounded contracts is supported.
De-Long Li, Xu Wang, Hao-Chen Gong et al.· arXiv.org· 12 citations· ⚡3
System X, a large language model (LLM) decision model that adds a text-reply option to the LLM-as-Jev readout, is introduced, and reply comparisons also differ in prompts and generation limits.
Large language models increasingly act as agents that call external tools, yet most tool-safety research examines only what follows a tool call. The prior decision of whether to call the tool remains largely unexamined and can influence downstream safety. Across the seven open-weight models, the unweighted model-averag...
Aditya Nawal, Manit Baser, M. Gurusamy· 0 citations
Decision models such as TypeSafe's Jev answer typed questions about a text (yes/no, multiple choice, or an ordinal score) with calibrated probabilities in a single forward pass and without generating text, which makes them an inexpensive candidate for LLM-as-a-judge workloads. We evaluate 14 System-1 decision models, s...
An agentic system issues several structurally different kinds of LLM calls. It routes intent, classifies actions, grounds language in a device registry, plans multi-agent pipelines and writes the Python code those pipelines run. The difficulty of these call sites varies by an order of magnitude, yet in practice a singl...
P. Kasnesis, Christos Chatzigeorgiou, Lazaros Toumanidis et al.· 0 citations
Jev is evaluated as a decision reader of truncated reasoning across seven LLM reasoners and five multiple-choice benchmarks, and it is suggested that System-1 decision models can complement LLM reasoning and reduce the need for costly reasoning completion.
Zhong-Yin Wang, Zhao-Qian Xue, Jun-Ting Mu et al.· 0 citations
We use cookies to run the site and, with your consent, for analytics and to show ads.
See our Cookie Policy.