Skip to content
Open access

Structural Metric Evaluation of LLM Generated Ontologies

Nov 2026 · Proceedings of the AAAI Symposium Series · 0 citations

Abstract

Ontologies support reasoning, information retrieval, data integration, and FAIR data practices by representing domain knowledge as structured entities and relationships. They are a fundamental component of neuro-symbolic AI systems, providing risk mitigation for AI systems through explicitly-encoded knowledge. Large language models offer a promising way to generate ontology-like structures from unstructured text, but evaluating these generated ontologies remains difficult. Semantic validation is essential, but it is often costly, domain-specific, and difficult to scale across large numbers of generated artifacts. This paper investigates whether structural metrics can support evaluation for LLM-generated ontologies. We compute a suite of metrics across OntoPortal ontologies, LLMs4OL gold-standard ontologies, ontologies generated from the LLMs4OL dataset, and LLM-generated ontologies from Wikipedia articles. We examine whether generated ontologies are structurally distinguishable from curated ontology populations. Our results suggest that structural metrics can identify systematic differences between generated and curated ontologies, particularly in hierarchy, connectivity, branching, and relation usage. These metrics do not replace semantic validation, but they can support scalable triage, comparison, and diagnosis of generated ontologies, providing a first step toward understanding how ontology-generation methods can be improved.

Read PDF

We use cookies to run the site and, with your consent, for analytics and to show ads. See our Cookie Policy.