A Preliminary Inference on How Large Language Models Form Knowledge and Answer Questions

Abstract

The previous paper examined what knowledge is in a large language model and how knowledge forms structures. This gave rise to a further question: after this knowledge has formed, how does a large language model use it to answer questions? This paper continues the reasoning along this question. First, starting from pre-training and post-training, it is conjectured that pre-training mainly forms knowledge and its structures, while post-training further adjusts how knowledge is used and how output behaves. Then the paper asks what happens when a large language model faces a question: how is the input understood, how are retrieval conditions formed, how are relevant Knowledge Clusters activated, and how is knowledge combined with the question and the context to form an answer. Furthermore, with respect to the invocation of context, this process is provisionally divided into three parts: information acquisition, information retention, and information retrieval. Thus, this paper attempts to move step by step from "how knowledge forms" to "how knowledge is invoked," and further to "how the context participates in computation," ultimately forming a reasoning chain of working mechanisms from knowledge structure to answer output.