The Reliability Wall : Why Scaling Cannot Buy Certifiable Compositional Reasoning, and Why It Is Permanent

Abstract

Large language models are increasingly asked to perform tasks made of several dependent steps (multi-digit arithmetic, multi-step planning, rule-following, program synthesis) where the final answer is correct only if every one of those steps is correct; a single mistake anywhere invalidates the whole output. This paper shows that, on such tasks, three problems the field usually treats as separate and separately fixable (accuracy falling as tasks get longer, confidence scores becoming meaningless exactly when they are most needed, and the cost of obtaining a correct answer exploding) are in fact one and the same event, governed by a single quantity. That event is not a temporary shortcoming that more training will remove: it is a structural consequence of how these models compute, and it sets a hard, predictable limit on how far they can be trusted on deep, multi-step work. Formally, end-to-end accuracy declines geometrically with the number of dependent steps, because every added step multiplies the chance of a fully correct answer by the same fixed factor below one, so useful performance necessarily ends at a finite critical depth, beyond which accuracy, the trustworthiness of any confidence score, and the cost per correct answer all collapse together. The practical implication is that certifiable compositional reasoning is not a target that scaling today’s architectures will eventually reach: it requires moving to computation that is genuinely serial and exact, not a larger version of the same parallel one.