Deterministic Validation for Reliable LLM-Based Causal Knowledge Extraction

Abstract

Large Language Models produce stochastic outputs that undermine reproducibility in knowledge extraction. We present a deterministic post-processing framework with 14 explicit validation predicates that transforms unreliable LLM output into consistent, validated causal triplets. Evaluated on four benchmarks spanning 2,177 documents, the framework achieves 88% precision on DocRED (validated by 5-agent LLM-based inter-annotator agreement), 100% semantic F1 on causal-specific samples, and 100% byte-level determinism across 150 repeated extractions. Multi-model validation on three architectures (Qwen8B, Gemma-2B, Llama-3B) confirms that determinism is a property of the validation architecture, not the underlying model-all achieve perfect consistency despite extraction rates varying by $9 \times$. Stochastic sampling experiments (temperature 0.8) confirm the framework contains no hidden randomness. Against dependency-based Open IE, the framework produces complete causal triplets where Open IE yields 60% incomplete extractions. The key contribution: reliability emerges from deterministic validation architecture rather than model improvements. All code and results are publicly available.