Skip to content
Open access

Reasoning in Language Models

TL;DR

The work done in this thesis indicates that LLMs do exhibit properties and behaviors one would expect of a system capable of reasoning, but always with caveats: models solve tasks requiring reasoning only some of the time, and the relations between concepts and propositions appear to be reflected in their internal representations, but imperfectly.

Abstract

Whether language models can reason is an important factor in their potential as a technology, both in terms of how well they can tackle novel problems and how well they can justify their decisions or actions by giving reasons for them. The question of whether they can is also contentious. Some argue that LLMs already show signs of general intelligence; others argue that their apparent reasoning rests predominantly on having seen similar problems during training. This thesis approaches the question from two angles. First, we develop methods that can be used to identify to what degree aspects of reasoning occur in LLMs. Second, we explore how LLM reasoning can be used to help detect and characterize toxic language. To look for reasoning in language models we use different strategies. First, we expect reasoning to be reflected in the model’s behavior, specifically its abilities. Thus, we evaluate LLMs on a newly constructed benchmark whose correct answers, in a zero-shot setting, are only reachable by ruling out interpretations that are logically incompatible with the premises provided. Second, we expect the capacity to reason to be reflected in the model’s internals. We want to identify if hidden states accurately reflect various reasoning-relevant aspects of the inputs. To gauge this, we develop and apply new interpretability methods and approaches. One approach makes controlled substitutions in model inputs to identify if certain properties are used. Two others use probes to find out if concepts and propositions are represented in a way that respects the (logical) relations between them. To enable further experimentation in this direction, we also improve an existing probing method. The second part of the thesis turns to toxic language, which communicates hateful, derogatory, or offensive ideas. We argue for a toxic reasoning approach, which focuses on characterizing the implications of the content. We propose a schema of relevant traits, which describes the toxicity that is expressed, and is compatible with existing taxonomies. We also construct the multilingual ToxiREX dataset of contextual Reddit comments annotated with this schema, and use it to evaluate language models for toxic reasoning. The work done in this thesis indicates that LLMs do exhibit properties and behaviors one would expect of a system capable of reasoning, but always with caveats: models solve tasks requiring reasoning only some of the time, and the relations between concepts and propositions appear to be reflected in their internal representations, but imperfectly. These results reflect the incomplete status of this line of research. For toxic language, previous approaches do not robustly solve the problem. For a problem that is constantly evolving, the toxic reasoning approach provides a promising direction, and the schema and dataset provide the infrastructure needed to further demonstrate its benefits.

Read PDF

Similar papers

The Age of Reasoning

T. Graepel, David S. Berman, Michael A. Warren et al. · 0 citations
#natural language process... Conference Open access Sep 2026

From Token Probabilities to Semantic Constraints: Towards Declarative Probabilistic Evaluation of Language Models

A declarative probabilistic framework for pre-training evaluation that makes the semantic structure of model behavior explicit and provides new formal tools for relating evaluation to learning, and links evaluation and learning through a shared semantics.

Kyle Richardson, C. Anderson, Pranav Balakrishnan et al. · 1 citation

Chronologic: Measuring Language Models' Ability to Represent the Past

Language models are appealing tools for research on the past. But to trust the evidence a model provides, researchers need to know whether its responses fit the period represented. Validation is challenging, because this is not a task living people ordinarily perform, and because many questions have multiple correct an...

T. Underwood, Zi-Liang Qiu, Sarah Griebel et al. · 1 citation
#machine learning Preprint Oct 2026

Can LLMs Anticipate Their Cruxes?

To use language models to help us make decisions we would like to understand the rationale behind the language model's outputs. In principle, the language model provides its rationale as part of its text output. However, the sheer volume of text makes this hard to use in practice. In this paper, we are interested in ex...

David Reber, James Reber, Ari Holtzman et al. · 0 citations
Conference Open access Sep 2026

Reasoning in Large Language Models: Methods, Challenges, and Future Directions

This paper reviews several methods aimed at improving reasoning in large language models, including prompt-based approaches, including Chain-of-Thought, self-consistency, and Auto- CoT, which try to guide models to generate intermediate reasoning steps.

Zhuang-Chen Xu · 0 citations

We use cookies to run the site and, with your consent, for analytics and to show ads. See our Cookie Policy.