Skip to content

Towards an Evaluation Framework for Generated Formal Knowledge

· 0 citations · 24 references

TL;DR

This work proposes a set of metrics designed to evaluate the ability of LRMs to generate formal knowledge in order to solve domain-specific problems, and curates – based on a collection of LRM-generated Prolog programs – common issues in generated KBs, and derive metrics that can be used to assess such issues.

View source

Similar papers

Open access Aug 2026

Constrained Entity Selection Under Partial Knowledge for LLM-Based Knowledge Graph QA

Experiments show that precision improves by filtering invalid candidates, while recall is preserved due to retaining candidates whose constraints are not explicitly violated, and a three-valued constraint semantics that avoids incorrect rejections under open-world assumptions.

Emanuel Kitzelmann · 0 citations
Conference Open access Aug 2026

K-GRASP: Tacit Knowledge Externalisation with Language Models and Knowledge Graphs

This work proposes an architecture called K-GRASP (Knowledge Graph-based Retrieval-Augmented Structured Prompting), which combines the representational power of knowledge graphs (KG) with the probabilistic reasoning capabilities of Large Language Models (LLMs) to address the externalisation of tacit knowledge.

Rafael Luna, Gabriel S. Luna, C. E. Barbosa et al. · 0 citations
Conference Open access Aug 2026

From Tacit Knowledge to Structured Documents: A Framework for Knowledge Elicitation

The Assistant-Scribe-Knowledge Checker (ASK) framework for generating structured specification documents through guided interviews is evaluated in a consulting-firm setting where consultants are required to produce project “return-of-experience” documents to capture reusable knowledge.

Sylvain Roudiere, Bianca Lento · 0 citations

A Knowledge Graph–Augmented Large Language Model for Automated Quality Compliance Checking in Construction

This research proposes a large language model–based approach combined with structured algorithms that process domain-specific competency questions and unstructured construction documents to build ontologies and extract structured knowledge graphs to support the automated generation of quality inspection checklists and...

Yi-Heng Wang, Juan Wang, W. Fang · 0 citations
Conference Open access Aug 2026

Deriving Uncertain Knowledge in Knowledge Graphs Under Theoretical Mapping of Heuristics

This paper proposes a deterministic, explainable framework for validating uncertain entities during Knowledge Graph (KG) expansion. The contribution is conceptual and formal: we present the framework and demonstrate its internal mathematical coherence, while empirical evaluation is explicitly deferred to future work. S...

Simon Von Oppenkowski, Benedikt Lerch, Klemens Schnattinger · 0 citations
Conference Open access Sep 2026

Information-Needs-Guided Virtual Knowledge Graph Enrichment via Large Language Models

This work forms the task of Information-Needs-Guided VKG Enrichment (IN-VKGE), and proposes an iterative framework that leverages large language models to assess whether information needs can be supported using SPARQL execution feedback and generate ontology and mapping enrichment proposals.

Lin Ren, Guohui Xiao, Guilin Qi et al. · 0 citations

We use cookies to run the site and, with your consent, for analytics and to show ads. See our Cookie Policy.