Towards an Evaluation Framework for Generated Formal Knowledge
This work proposes a set of metrics designed to evaluate the ability of LRMs to generate formal knowledge in order to solve domain-specific problems, and curates – based on a collection of LRM-generated Prolog programs – common issues in generated KBs, and derive metrics that can be used to assess such issues.