Adaptive Claim Level Hallucination Detection and Mitigation for BlackBox LLMs on Amazon Bedrock

Abstract

Large language models (LLMs) deployed through managed cloud APIs are increasingly relied upon in settings where incorrect output carries real cost enterprise search, healthcare triage, financial advisory, and legal question answering among them. Hallucination mitigation in these black-box environments is hard for two reasons: strong detection methods typically depend on model internals that hosted APIs do not expose, and current mitigation pipelines apply retrieval and verification uniformly, adding latency and cost even when a response is already well-grounded. This article describes Claim Guard, a cloud-native pipeline that breaks each LLM response into atomic factual claims, retrieves targeted evidence for each claim, estimates a calibrated per-claim risk score from three signal sources retrieval support, verifier disagreement, and Amazon Bedrock Guardrail feedback and selects the cheapest mitigation action whose expected cost does not exceed the risk. Actions range from accepting the original answer to RAG regeneration, chain-of-verification, selective claim rewriting, and abstention. Built on Amazon Bedrock, Kendra, Lambda, S3, DynamoDB, and Bedrock Guardrails, ClaimGuard treats factuality, latency, and per-query cost as joint objectives. A dynamic evaluation protocol pairs standard benchmarks TruthfulQA, FEVER, HotpotQA, HaluEval, RAGTruth with adversarial evidence perturbation and temporal drift stress tests. Across all benchmarks ClaimGuard achieves a mean hallucination rate of 11.9%, against 37.1% for unmitigated generation and 18.0% for always on RAG, at a mean end to end latency of 1.02 s well below the 2.87 s of always on RAG.