Skip to content
Conference Open access

Preface for LLMs4OL 2026: The 3rd Large Language Models for Ontology Learning Challenge at the 25th ISWC

Nov 2026 · Open Conference Proceedings · 3 citations

Abstract

This volume collects the contributions to the 3rd LLMs4OL Challenge, Large Language Models for Ontology Learning, held at the 25th International Semantic Web Conference (ISWC 2026). The LLMs4OL initiative began with a narrow question: can large language models perform the individual operations that ontology learning tasks such as term typing, taxonomy discovery, and relation extraction? Two editions of the challenge answered that question in controlled benchmark settings and established task formulations, evaluation protocols, and a growing body of methods. With this third edition, we asked a harder question. Real ontology engineering rarely begins with a clean list of terms, and it rarely ends with a single prediction. It involves reading unstructured sources, extending what already exists, and carrying structure into domains nobody has modelled yet. The 2026 challenge was designed around those three situations. The Flagship Task asked participants to construct a primitive ontology, including terms, types, taxonomic relations, and non-taxonomic relations, directly from raw text. The Reuse Task supplied an existing partial ontology and asked for the statements that extend it. The Taxonomy Task provided ontologies from source domains and a list of types from an unseen target domain, and asked for the hierarchy connecting them. Together, the three tracks cover extraction, integration, and generalization. The datasets are maintained in a dedicated GitHub repository (https://github.com/sciknoworg/LLMs4OL-Challenge). Thirteen teams contributed scored submissions: eleven to Flagship, eight to Reuse, and five to at least one Taxonomy target (see https://sites.google.com/view/llms4ol2026). Their systems span retrieval-augmented prompting, fine-tuning and continual pretraining of open-weight models, embedding-based candidate retrieval, conventional NLP pipelines, symbolic post-processing, and multi-model debate. The results are informative in both directions. Constraining the label space mattered more than model scale; adapted small models held their own against frontier systems, and retrieval grounding transferred more reliably than post-hoc correction. At the same time, non-taxonomic relation extraction was largely left aside under the current scoring; cross-domain taxonomy induction proved unstable across targets, and vocabulary induction emerged as the principal bottleneck for end-to-end construction. We consider these three findings key challenges for the community. This edition also introduced a second phase. Participating systems are being integrated into OntoLearner, our modular Python framework for ontology learning, so the methods described in these pages persist as executable, comparable components rather than reported numbers alone. The ambition is to collect learners, not only to rank them. We extend our gratitude to all participants for their invaluable contributions; their solutions and dedication have greatly enriched this challenge. We also sincerely thank the conference organizers and committee for their efforts in hosting this event. We deeply appreciate the reviewers for their evaluations and feedback. Their reviews have been instrumental in enhancing the quality of the submissions. We want to acknowledge specifically: Andrei Aioanei (TIB Leibniz Information Centre for Science and Technology, Germany) Upal Bhattacharya (Utrecht University, Netherlands) Miquel Canal Esteve (University of Alicante, Spain) Chuhan Deng (Tsinghua University, China) Ryn Gray (Sandia National Laboratories, United States) Jiaqi Jin (ZTE, China) Sandra Kara (Université Paris-Saclay, France) Erban Ku (Volt-IQ S.R.L., Italy) Pasquale Lisena (EURECOM, France) Lvting Liu (University of Science and Technology of China, China) Shivam Mishra (Indian Institute of Information Technology, India) Andy Symonds (Thoughtworks, United Kingdom) Guojian Zhao (Southern University of Science and Technology, China) Xinyi Zhao (Laboro.AI Inc., Japan) Lastly, we would like to express our gratitude to ”Xenia Felice van Edig” and ”Marco Gronewold” from the TIB Open Publishing for their support in bringing these proceedings online. We also acknowledge that the 3rd LLMs4OL Challenge @ ISWC 2026 was supported by the NFDI4DataScience initiative (DFG, German Research Foundation, Grant ID: 460234259).

Read PDF

We use cookies to run the site and, with your consent, for analytics and to show ads. See our Cookie Policy.