Efficient LLM Reasoning via Logit-Based Introspection

Abstract

Recent Reasoning Large Language Models have demonstrated impressive long-form reasoning capabilities, achieving expert-level performance on complex tasks such as competition-level mathematics and scientific question answering. However, they often suffer from the overthinking problem, leading to excessive token usage, higher latency, and an increased risk of hallucinations over long contexts. Existing solutions often require costly post-training of the underlying LLM or incur extra decoding overhead from probing intermediate answers. In this paper, we propose LLM Introspection, a practical inference-time reasoning control method that leverages keyword-token logits to enable early stopping during the reasoning process. By introspecting the model’s internal confidence state through a lightweight GRU-based module operating on the token-logit stream in the reasoning phase, our method dynamically terminates reasoning once sufficient confidence is reached, effectively mitigating overthinking. Experiments on Qwen3 and DeepSeek-R1-Distill across MATH-500, GSM8K, AIME, and GPQA Diamond show consistent token savings with comparable accuracy in most cases. On Qwen3-4B, token reductions range from 4.3% to 41%, with generally improved Accuracy–Efficiency Scores. We further integrate our method into a vLLM-based proxy, providing a practical, plug-and-play deployment path.