Skip to content
Book Open access

Preserving LLM Output Distributions for Probabilistic Query Evaluation

Nov 2026 · Proceedings of the 35th ACM International Conference on Information and Knowledge Management · 0 citations · 13 references

Abstract

Language models can be used to augment databases; typically, when this is done, their top prediction is used while the uncertainty in this prediction is ignored. We propose a framework that preserves output distributions from language model predictions and uses them to evaluate SQL queries probabilistically. First, a language model is used to infer new features from existing ones. Second, ProvSQL, a PostgreSQL extension implementing a probabilistic database through data provenance, encodes the predictions as mutually exclusive events in a provenance circuit and evaluates probabilistic queries. We extend ProvSQL to support text-value comparisons and join operations on language model predictions. We demonstrate the framework on 10\,612 Amazon musical products and 230\,156 user reviews, locally augmented with the Llama 3.1 model. We show that probabilistic querying reveals contradictions between model predictions and existing labels, recovers data records otherwise missed, and propagates joint uncertainty across multi-table queries.

Read PDF

We use cookies to run the site and, with your consent, for analytics and to show ads. See our Cookie Policy.