Comparative Analysis of Embedding, LLM, and RAG-Based Approaches for OOD Intent Detection

Abstract

Out-of-distribution (OOD) intent detection remains a critical challenge for deploying conversational agents in realworld scenarios. This paper presents a comparative analysis of three distinct paradigms for OOD detection: embedding-based semantic similarity, zero-shot Large Language Model (LLM) reasoning, and Retrieval-Augmented Generation (RAG). Evaluated on the banking-specific Ban king77 d a t aset, these met hods assessed across both in-distribution (ID) classification accuracy and OOD detection metrics. Our experimental results demonstrate that relying solely on vector distances or ungrounded zeroshot LLM logic yields high false-positive rates. In contrast, the RAG-based framework significantly out performs the baselines, achieving an AUROC of 0.933 and an ID accuracy of 0.926. The findings highlight that anchoring L LMreasoning with retrieved ID exemplars provides the most effective and reliable strategy for discriminating OOD queries.