Jul 2026· International Journal of Research in Engineering, Science and Management· 0 citations
TL;DR
LLM-Advisor is introduced, an open-source, adaptive framework designed for intelligent query categorization, dynamic model evaluation, and constraint-aware request routing across heterogeneous multi-LLM pools that achieves a 42% reduction in overall inference expenditure and a 35% decrease in average response latency.
Abstract
The rapid proliferation of Large Language Models (LLMs) with varying capability profiles, context window limits, execution latencies, and financial costs presents a significant operational challenge for enterprise AI deployments. Monolithic deployment strategies wherein all requests are directed to a single high-capability frontier model result in substantial compute over-provisioning and excessive operational costs for routine queries. Conversely, relying solely on lightweight models degrades output accuracy on complex multi-step reasoning tasks. To resolve this trade-off, this paper introduces LLM-Advisor, an open-source, adaptive framework designed for intelligent query categorization, dynamic model evaluation, and constraint-aware request routing across heterogeneous multi-LLM pools. LLM-Advisor analyzes incoming prompt features, structural complexity, domain requirements, and user-defined constraints (e.g., maximum cost per request, latency thresholds) to route tasks to the optimal candidate model. We evaluate LLM-Advisor using a benchmark suite of 1,000 queries across code generation, general reasoning, and contextual retrieval tasks using both proprietary and open-weight models (including GPT-4o, Claude 3.5 Sonnet, Llama 3, and Mistral). Experimental results demonstrate that LLM-Advisor achieves a 42% reduction in overall inference expenditure and a 35% decrease in average response latency while retaining 94.6% task accuracy compared to static GPT-4o baseline routing. These findings highlight LLM-Advisor as an efficient, highly scalable middleware solution for production-grade AI system deployments.
A framework is intended as a design reference for practitioners building inference gateways and as a structuring basis for future empirical work on multi-objective LLM request scheduling, and an evaluation protocol comprising workload definitions, baselines, and metrics is specified.
Rajalakshmi Srinivasaraghavan· International journal of com...· 0 citations
Cascade, an LLM serving system that estimates and continuously updates this per-request latency budget from request characteristics, KV-cache state, and current system load, and uses a single per-request budget to jointly coordinate request scheduling and KV-cache management across the memory hierarchy.
Muhammad Adnan, R. Mahapatra, Prashant J. Nair et al.· 0 citations
VDAR-Router is proposed, a difficulty-aware retrieval-based routing framework for training-free LLM routing that consistently achieves better cost-performance trade-offs than existing baselines and helps retrieve more relevant examples and supports more reliable routing decisions.
Yu-Chien Tang, Jun-Chen Hung, Wen-Chih Peng et al.· 0 citations
CRM+RCCR, an architecture-agnostic cost-aware objective that encodes cost preference into continuous relevance targets through per-pair independent scoring, eliminating multi-positive dilution while regularizing queries with similar routing preferences to be closer in the routing space.
Query optimization remains a critical challenge in relational database management systems (RDBMSs). Traditional Cost-Based Optimization (CBOs) depend on static cardinality estimation and rigid heuristics that break down under skewed data distributions, complex join structures, and heterogeneous workloads. While learned query optimization methods offer improved cost estimation, they remain vulnerable to schema drift and unseen query templates and lack the interpretability required by database administrators. This paper presents LLM-QOpt++, a novel hybrid, confidence-aware query optimization framework that unifies traditional CBO estimation, machine learning–based cost prediction, and large language model (LLM) reasoning within a single adaptive pipeline. Key components include operator-level cost decomposition; an LLM-based query reasoning advisor that detects structured query language (SQL) anti-patterns and generates context-aware rewrite suggestions; a confidenceaware decision mechanism for adaptive strategy selection; a failed-plan learning memory (FPLM) that suppresses historically inefficient execution plans; and an explainability-guided layer leveraging SHapley Additive exPlanations (SHAP) attribution, attention analysis, and natural language reasoning. A dynamic hybrid fusion mechanism blends CBO, ML, and LLM signals according to per-query confidence estimates. Evaluated on TPC-H (SF100) and TPCDS (SF300), LLM-QOpt++ achieves a 38.4% reduction in mean absolute cost estimation error over the best learned baseline (95% CI: [35.1%, 41.9%]; p < 0.001), a 2.37_ geometric mean speedup over PostgreSQL’s native optimizer (95% CI: [2.21_, 2.53_]; p < 0.001), and an F1-score of 0.89 in strategy classification. By integrating statistical, learned, and LLM-based reasoning under a confidence-aware architecture, LLM-QOpt++ delivers query optimization that is simultaneously more accurate and more interpretable than existing baselines.
Empirical results on RouterBench and SWE-Bench demonstrate that WR-Offline surpasses existing baselines in performance under a fixed budget and adheres more closely to budget constraints, and WR-Online achieves comparable performance to the baselines, while using substantially less exploration data.
