Skip to content

Complex Problem Solving in Large Language Models: A Statistical Control Survey and Diagnostic Framework

Sep 2026 · arXiv.org · Vol abs/2609.20973 · 0 citations
Mathematics Computer Science

TL;DR

This framework organizes existing methods around five components: explicit state representation, transition structuring, validation and constraint enforcement, search and rollback, and uncertainty management, and yields a diagnostic hypothesis: interventions should be most effective when they target the error or uncertainty component implicated by an observed failure.

Abstract

Complex problem solving (CPS) with large language models (LLMs) is often framed as a matter of stronger reasoning or longer generation. Yet early-step error amplification, prompt brittleness, and failures to revise incorrect commitments are difficult to explain by missing knowledge or expressive capacity alone. This survey interprets CPS as a sequential estimation-and-decision problem over a latent solution state. A controller maintains a belief about an unobserved solution trajectory, updates it as noisy intermediate evidence arrives, and decides whether to commit, verify, branch, roll back, or abstain to minimize expected loss. Reasoning supplies candidate transitions and interpretations, whereas process control shapes and evaluates those proposals and regulates subsequent transitions and observations. Within this framework, we organize existing methods around five components: explicit state representation, transition structuring, validation and constraint enforcement, search and rollback, and uncertainty management. We also interpret evaluation metrics according to the statistical quantities they estimate. The framework further yields a diagnostic hypothesis: interventions should be most effective when they target the error or uncertainty component implicated by an observed failure. We distinguish systematic, stochastic, and irreducible error together with epistemic and aleatoric uncertainty, and call this alignment problem-control fit and its failure control mismatch. For example, additional sampling may reduce sampling variability while leaving a shared systematic error unchanged. This perspective clarifies what current methods estimate and control, what remains uncontrolled, and why reliable validation, targeted recovery, calibrated uncertainty, and matched-budget evaluation are central open problems.

View source

Similar papers

Preprint Aug 2026

Test-Time Scaling in Reasoning LLMs: Inference Regimes, Evaluation, and Reproducibility

This empirical study covers broad knowledge, symbolic reasoning, and competition mathematics, and it introduces an evaluation profile whose coordinates and simple functionals recover or bound common repeated-sampling metrics, and require compute accounting and uncertainty estimates that match the protocol.

Mohsen Hariri, Weicong Chen, Nahal Shahini et al. · 5 citations · ⚡1
#artificial intelligence Preprint Sep 2026

Belief-State Engine: Augmenting LLMs for Principled Planning Under Partial Observability

It is proved that the LLM paired with the BSE is a sound Markov policy on the belief MDP induced by the underlying POMDP, and inherits the Bellman optimality guarantees of classical POMDP theory, provided the LLM is never exposed to the raw history.

Arnab Chattopadhayay, Debdipta Halder · 0 citations

Reasoning Pathologies in Large Language Models: A Diagnostic Perspective

A diagnostic framework is introduced that abstracts chain-of-thought reasoning as a trajectory through discrete latent reasoning states, yielding a positional taxonomy of six reasoning pathologies, each paired with a transition-grounded detection predicate and a matched inference-time controller that positions it as a...

Unknown authors · 0 citations
#machine learning Preprint Sep 2026

Sampling via Decision-Flow: Training-Free Extraction of Improved Latent Reasoning Paths in Large Language Models

Decision-Flow Sampling (DF-Sample), a training-free, data-free inference-time framework that constructs a hierarchical reasoning tree, scores terminal nodes for quality, and back-propagates utilities to inform each intermediate branching decision, indicating substantial latent reasoning potential in pretrained base mod...

Zhen-Dong Mi, Shao-Yi Huang · 0 citations

Related blog posts

We use cookies to run the site and, with your consent, for analytics and to show ads. See our Cookie Policy.