Built with Tutify.io

    AIStudyLabs

    Ontologies and LLMs: Grounding AI Answers and Agents

    Ontologies and LLMs: Grounding AI Answers and Agents

    Large language models know a lot about the world in general and very little about your organization in particular. Ask one which customers are on the enterprise plan, or whether a product is covered by a regulation, and it will produce a fluent answer whether or not it has the facts. Ontologies and LLMs are a natural pairing for exactly this reason. The model brings language skills; the ontology brings a fixed, reviewed definition of what exists in your domain and how things relate.

    The practical patterns are: using an ontology to constrain what an LLM extracts, linking mentions to canonical entities, validating model output before it is stored, using LLMs to speed up ontology building, and giving AI agents a governed vocabulary for actions. This article is part of a series that starts with ontology in data and AI, which explains what an ontology is and why shared meaning matters.

    Why ontologies and LLMs need each other

    LLMs fail in a few predictable ways when they work with enterprise data:

    An ontology addresses each of these. It defines the allowed classes, the relationships between them, the identifiers for real entities, and the rules that must hold. The LLM does not need to memorize any of it. It needs to be given the relevant part at the right moment and checked against it afterwards.

    The research community has framed this relationship in both directions. A roadmap by Pan and colleagues, published in IEEE TKDE in 2024, describes knowledge-graph-enhanced LLMs, LLM-augmented knowledge graphs, and systems where the two work together. The patterns below fit into those three groups.

    Pattern 1: Ontology-guided extraction

    The most common job for LLMs in data pipelines is turning unstructured text into structured records: contracts into obligations, support emails into ticket fields, papers into findings. Without guidance, the output schema changes from document to document.

    The fix is to give the model the ontology's classes and properties as the target schema. The SPIRES method, published in Bioinformatics in 2024 and implemented in the open-source OntoGPT package, did exactly this. A user defines a schema, the system prompts the model recursively to fill each part, and then it grounds every extracted value to an identifier from an existing ontology. The authors report accuracy comparable to the middle of the range of existing relation extraction methods, without any task-specific training data. That is not state of the art, but it is achieved with no labelled examples, which is often the deciding factor.

    In practice, ontology-guided extraction means three things in the prompt:

    1. The list of allowed entity types, with one-line definitions taken from the ontology's labels and comments.
    2. The list of allowed relationships, with their domain and range.
    3. An instruction to return only those types and relationships, in a fixed JSON structure.

    Structured output features in current model APIs help enforce the JSON shape. They do not enforce meaning. The model can still label a product as a person in perfectly valid JSON, which is why validation comes next.

    Pattern 2: Entity linking

    Entity linking maps a mention in text to a canonical identifier. "Big Blue", "IBM" and "International Business Machines" should all resolve to one IRI in your knowledge graph. Without entity linking, every new document adds duplicates, and graph queries quietly undercount.

    The ontology helps in two ways. Its individuals provide the target list of identifiers, along with preferred and alternative labels. Its classes narrow the search: if the mention is the subject of worksFor, it should resolve to a person, so you only compare it against people.

    A workable pipeline looks like this:

    1. The LLM extracts mentions and proposes a type for each.
    2. A retrieval step finds candidate entities of that type by label similarity or embeddings.
    3. The LLM, or a simple scoring rule, picks the best candidate or declares a new entity.
    4. New entities go to a review queue rather than straight into the graph.

    Entity linking errors compound: every missed match adds a duplicate, and every wrong match merges two real entities. Spend time on the candidate retrieval and the review queue, not just the prompt.

    Pattern 3: Validating LLM output against the ontology

    Generated facts should never go straight into a production knowledge graph. Check them first, in two layers: a cheap filter for unknown types and relationships, then SHACL shapes for the rules. The example below takes what an LLM returned for a short paragraph and runs both checks. It uses RDFLib and pySHACL, the same libraries as the tutorial on building an ontology in Python with RDFLib.

    import json
from rdflib import Graph, Namespace, Literal, RDF
from pyshacl import validate

EX = Namespace("https://data.example.com/ontology#")
ID = Namespace("https://data.example.com/id/")

ALLOWED_TYPES = {"Company", "Person", "Product"}
ALLOWED_RELATIONS = {"worksFor", "makes"}

# What an LLM returned for: "Dana Ortiz leads design at Acme, which makes the Falcon drone.
# Acme was acquired by Globex."
llm_output = json.loads("""
{
  "entities": [
    {"id": "dana-ortiz", "type": "Person",  "name": "Dana Ortiz"},
    {"id": "acme",       "type": "Company", "name": "Acme"},
    {"id": "falcon",     "type": "Product", "name": "Falcon"},
    {"id": "globex",     "type": "Organisation", "name": "Globex"}
  ],
  "relations": [
    {"subject": "dana-ortiz", "predicate": "worksFor",   "object": "acme"},
    {"subject": "acme",       "predicate": "makes",      "object": "falcon"},
    {"subject": "globex",     "predicate": "acquired",   "object": "acme"},
    {"subject": "falcon",     "predicate": "worksFor",   "object": "acme"}
  ]
}
""")

g = Graph()
rejected = []
for e in llm_output["entities"]:
    if e["type"] not in ALLOWED_TYPES:
        rejected.append(f"entity {e['id']}: unknown type {e['type']}")
        continue
    g.add((ID[e["id"]], RDF.type, EX[e["type"]]))
    g.add((ID[e["id"]], EX.name, Literal(e["name"])))

for r in llm_output["relations"]:
    if r["predicate"] not in ALLOWED_RELATIONS:
        rejected.append(f"relation {r['subject']} {r['predicate']} {r['object']}: unknown relation")
        continue
    g.add((ID[r["subject"]], EX[r["predicate"]], ID[r["object"]]))

print("\n".join(rejected))

shapes = Graph().parse(format="turtle", data="""
@prefix sh: <http://www.w3.org/ns/shacl#> .
@prefix ex: <https://data.example.com/ontology#> .

ex:WorksForShape a sh:NodeShape ;
    sh:targetSubjectsOf ex:worksFor ;
    sh:class ex:Person .

ex:MakesShape a sh:NodeShape ;
    sh:targetObjectsOf ex:makes ;
    sh:class ex:Product .
""")

conforms, _, text = validate(g, shacl_graph=shapes)
print("Conforms:", conforms)
for line in text.splitlines():
    if line.strip().startswith(("Focus Node", "Message")):
        print(line.strip())

    Output (Python 3.13, RDFLib 7.6.0, pySHACL 0.40.1):

    entity globex: unknown type Organisation
relation globex acquired acme: unknown relation
Conforms: False
Focus Node: <https://data.example.com/id/falcon>
Message: Value does not have class ex:Person

    The output shows three different failures. The model invented a type, Organisation, that the ontology spells Company. It invented a relationship, acquired, that the ontology does not define. And it produced a well-formed but wrong fact: a product that "works for" a company. The first two are caught by the allow-list; the third only by the SHACL shape, which says that anything with worksFor must be a Person.

    What you do with failures depends on the pipeline. Unknown types often point at a missing synonym in the ontology (here, Organisation should probably be an alternative label for Company). Unknown relationships can go to a modelling backlog. SHACL violations can be sent back to the model with the error message for one retry, then to a person. Validation of LLM-generated triples with SHACL is an active research area; xpSHACL, for example, uses an LLM to explain SHACL violations in plain language.

    Two gaps are worth closing before production. The loop above still adds a relation whose subject or object was rejected, so globex makes falcon would slip in with an untyped subject; drop any relation that points at a rejected entity. And each shape checks only one end of its property, so dana-ortiz worksFor falcon would pass; production shapes should check both the domain and the range.

    The SHACL standard itself is explained in ontology languages: RDF, RDFS, OWL and SHACL.

    Pattern 4: LLMs helping build ontologies

    The relationship also runs the other way. Building an ontology by hand is slow, and LLMs are good at proposing terms, definitions and hierarchies from documents.

    The LLMs4OL study, presented at ISWC 2023, tested nine families of LLMs on three ontology learning tasks: assigning types to terms, discovering taxonomic hierarchies, and extracting non-taxonomic relations. Its test data came from WordNet, GeoNames and the medical vocabulary UMLS. A follow-up challenge at ISWC 2024 extended the evaluation with few-shot and zero-shot phases on held-out parts of ontologies.

    The practical lesson from LLMs4OL is that LLMs are useful drafting assistants and poor final authorities. Use them to:

    Then have a domain expert review every proposed change, exactly as you would review a pull request. The common mistake is letting the model write directly to the ontology. Errors at the schema level spread to every fact typed by it, and they are much harder to find than a single wrong fact.

    Pattern 5: Ontologies for AI agents

    Agents take actions, not just answer questions. That raises the stakes. An agent that misunderstands what a "cancellation" is might refund the wrong order. Ontologies give AI agents two things: a shared vocabulary for reading the world, and a defined set of actions with rules about when they apply.

    Palantir's Foundry documentation is a well-documented public example of this design. It describes its Ontology as an operational layer for the organization with two parts: a semantic layer of objects, properties and links, and a kinetic layer of actions and functions that change data under organizational controls. Agents built in its AIP tooling use tools that query objects, call functions and execute ontology edits, either automatically or after a person confirms.

    You don't need that platform to apply the idea. The general pattern for AI agents is:

    1. Expose the ontology as tools. Instead of free-form SQL, give the agent typed operations such as "find customers by plan" or "list open tickets for customer X", each defined against ontology classes.
    2. Define actions in ontology terms. "Issue refund" takes an Order and an amount, has preconditions, and records who triggered it.
    3. Check before acting. Validate the agent's intended change against SHACL shapes and business rules, just as with extraction.
    4. Log everything in the same vocabulary. When every action is recorded as a change to ontology objects, audits become queries.

    A typed tool might be list_open_tickets(customer: Customer) -> list[Ticket], where Customer and Ticket are ontology classes and each returned ticket carries its ontology IRI. Whatever mechanism exposes tools to the model, describing their inputs and outputs this way keeps every agent and every person working from the same definitions.

    Grounding answers with knowledge graph retrieval

    The patterns above deal with getting data in and acting on it. Answering questions is the other half. When an LLM answers from a knowledge graph typed by an ontology, it can cite specific entities and relationships rather than paraphrasing loosely related documents. If the graph is built over integrated source systems, as in ontology-based data integration, the answers reflect current operational data.

    Building that graph from documents with an LLM, and using the ontology to keep its entity types consistent, is the subject of ontology-driven GraphRAG.

    Costs and limits of ontologies for LLMs

    The approach has real costs:

    Do you need a full OWL ontology for LLM work?

    Usually not at first. For extraction and validation, start with a list of classes with definitions, a list of relationships with domain and range, and a few SHACL shapes. That is an RDFS-level model, and it can live in a single Turtle file. Full OWL axioms start to pay off when you want a reasoner to classify entities automatically or to reconcile terms from several sources. A sensible rule when combining ontologies and LLMs is to start with RDFS plus SHACL and add OWL axioms only when a reasoner earns its place.

    An ontology is worth the effort when wrong answers are expensive, when many applications must agree on definitions, or when agents take actions with consequences. For a casual internal chatbot over a few documents, plain retrieval may be enough. The decision of how much structure you need at all is covered in ontology vs taxonomy vs knowledge graph.

    Getting started with ontologies and LLMs

    A small first project with ontologies and LLMs shows the value quickly:

    1. Pick one extraction task where inconsistent output already causes problems.
    2. Write a minimal ontology with five to ten classes and a handful of relationships.
    3. Put the allowed types and relationships in the prompt.
    4. Add the allow-list filter and two or three SHACL shapes, as in the example above.
    5. Measure how many outputs fail each check, and fix the ontology or the prompt based on what you see.

    The failure counts are the most useful artifact. They tell you whether the problem is the model, the prompt or the ontology. Track them per model version, too: when you upgrade the LLM, the same checks tell you at once whether extraction quality moved, which is far cheaper than discovering drift in the knowledge graph months later. The same checks apply to AI agents, where the count to watch is how many proposed actions fail validation before a person ever sees them.

    Key takeaways

    Related articles in this series