Accelerated LLM: A Fuzzy-Logic-Augmented Router Architecture for Efficient Multi-Domain Query Processing via Specialised Small Language Models

Abstract

Large language models (LLMs) incur prohibitive computational costs when deployed as monolithic systems for multi-domain query processing. This paper proposes Accelerated LLM, a modular architecture that replaces a single general-purpose LLM with an ensemble of task-specialised small language models (SLMs) governed by a neural query router and a Mamdani fuzzy inference system. The router embeds each user query using a frozen sentence encoder and classifies it across four task domains—summarisation, translation, question answering, and text generation—routing confident queries directly to the corresponding SLM. Ambiguous queries are escalated to a three-input fuzzy logic system operating on Query Length, inter-Domain Overlap Score, and Classifier Confidence, enabling principled handling of imprecise inputs. A reinforcement-learning feedback loop, validated through a controlled pilot deployment, continuously refines the routing policy. The complete pipeline, including the sentence encoder, totals approximately 2.14 billion parameters—a 98.8% reduction relative to GPT-3.5 (175 B). The integration of fuzzy logic into the routing stage raises classification accuracy from 91.5% to 94.3% and reduces the hallucination rate to 9.8% (minor) and 6.4% (major). Evaluated on healthcare-augmented benchmarks against ChatGPT-3.5, Claude, Mistral 70B, and two contemporary compact models (GPT-4o-mini and Llama 3.1-8B-Instruct), Accelerated LLM achieves competitive or superior task-specific performance at a fraction of the parameter count. A small-scale pilot evaluation in the legal domain indicates that the routing and fuzzy logic components retain partial effectiveness beyond the primary healthcare setting, though full multi-domain validation remains future work.