Built with Tutify.io

    AIStudyLabs

    Ontology-Driven GraphRAG: Better Retrieval with Knowledge Graphs

    Ontology-Driven GraphRAG: Better Retrieval with Knowledge Graphs

    Standard retrieval-augmented generation finds text chunks that look like the question and hands them to a language model. That works well for questions whose answer sits in one or two passages. It struggles with questions about a whole collection, such as "what are the main risks raised across these 500 audit reports?" GraphRAG addresses this by building a knowledge graph from the documents first, then retrieving from the graph.

    There is a catch. When an LLM builds the graph unsupervised, it chooses its own entity types and relationships, and the result is often inconsistent. Ontology-driven GraphRAG fixes that by giving the extraction step a defined model of what can exist in the graph. By the end you should be able to decide whether GraphRAG fits your corpus, and how to give its extraction step a schema. This article is part of a series that begins with ontology in data and AI.

    How GraphRAG works

    Microsoft Research introduced GraphRAG in a blog post in February 2024 and described it in the paper From Local to Global, first posted in April 2024 and revised in February 2025. The indexing pipeline has four main steps:

    1. Split the documents into text units. The paper used chunks of 600 tokens with 100 tokens of overlap.
    2. Extract entities, relationships and claims from each chunk with an LLM.
    3. Cluster the resulting graph into hierarchical communities of closely related entities, using the Leiden algorithm, a community detection method that groups densely connected nodes.
    4. Summarize each community with the LLM, bottom-up, so there are summaries at several levels from the broad root level down to small groups.

    At query time there are several search modes. Global search answers whole-corpus questions by asking each community summary for a partial answer, scoring those answers for helpfulness, and combining the best. Local search starts from the entities mentioned in a question and pulls in their neighbourhood in the graph. Later versions added DRIFT search, which combines local search with community context, and a basic mode that is plain vector search, as described in the GraphRAG documentation.

    What the paper found

    The paper evaluated GraphRAG on two datasets of roughly 1 million and 1.7 million tokens: podcast transcripts and news articles. Using an LLM as the judge, GraphRAG's global search beat plain vector RAG on comprehensiveness in 72 to 83 percent of comparisons on the podcast data and 72 to 80 percent on news. It also won on diversity of answers. Answers built from the root-level community summaries used more than 97 percent fewer context tokens than summarizing the source text directly.

    Those results come with a caveat: an LLM judge is not ground truth. The paper also ran claim-based checks, and they agreed with the judge's verdicts 78 percent of the time for comprehensiveness and about 70 percent for diversity. Treat the numbers as evidence that the approach helps on whole-corpus questions, not as a guaranteed improvement on your data.

    Local and global questions: a worked example

    Picture an insurer with 3,000 claim adjuster notes from the past year. Two kinds of question come up.

    The first kind is local: "What did the adjuster conclude about claim 88213, and which repair shop was involved?" Plain vector search handles this well. The claim number appears in a handful of chunks, retrieval finds them, and the model reads the answer straight out of the text.

    The second kind is global: "What are the most common causes of disputed water-damage claims this year, and which regions see them most?" No single chunk contains that answer. Vector search returns the ten passages that look most like the question, which might all come from one unusually wordy adjuster. The model then generalizes from a sample of ten, and the answer sounds confident and is mostly wrong.

    A graph-based approach answers the global question differently. During indexing, each note contributes nodes for the claim, the cause, the region, the repair shop and the outcome. Community detection groups related claims, causes and regions, and each group gets a summary. At query time, global search asks every community summary at the chosen level for a partial answer and combines the most helpful ones, so the answer draws on the whole year rather than ten passages.

    This is also where a fixed schema earns its keep. If one note records the cause as "burst pipe", another as "pipe failure" and a third as "plumbing leak", and the model creates three separate cause nodes, the summaries split the same pattern three ways and the most common cause looks less common than it is. Defining Cause as a type with a reference list of values, and resolving mentions to that list, fixes that. It also makes an exact answer possible: with consistent Cause nodes, a structured query over the graph can count disputed claims per cause and region, something LLM-written summaries can't do reliably on their own.

    Where free-form entity types go wrong

    Left to choose freely, an LLM extracting entities from thousands of chunks will produce a graph that looks impressive and queries badly. The common problems:

    Each problem degrades retrieval. Community detection groups the wrong things together, local search misses neighbours connected by an unexpected relationship name, and any structured query over the graph undercounts. The fix for the first four is to define the allowed entity types and relationships up front, which is exactly what an ontology does. The fifth needs entity resolution, covered below.

    What makes GraphRAG ontology-driven

    Ontology-driven GraphRAG means the extraction step works from a defined schema rather than inventing one. The ontology supplies:

    If you already have an OWL or RDFS ontology, you can generate the extraction schema from it, so the GraphRAG pipeline and the rest of your data platform share one definition. The snippet below reads a small ontology with RDFLib and turns it into the lists most knowledge graph builders accept.

    import re
from rdflib import Graph
from rdflib.namespace import RDF, RDFS, OWL

ontology = Graph().parse(format="turtle", data="""
@prefix ex:   <https://data.example.com/ontology#> .
@prefix owl:  <http://www.w3.org/2002/07/owl#> .
@prefix rdfs: <http://www.w3.org/2000/01/rdf-schema#> .

ex:Company a owl:Class ; rdfs:comment "A legal business entity." .
ex:Person  a owl:Class ; rdfs:comment "An individual human." .
ex:Product a owl:Class ; rdfs:comment "A good or service a company sells." .

ex:worksFor a owl:ObjectProperty ; rdfs:domain ex:Person  ; rdfs:range ex:Company .
ex:makes    a owl:ObjectProperty ; rdfs:domain ex:Company ; rdfs:range ex:Product .
ex:acquired a owl:ObjectProperty ; rdfs:domain ex:Company ; rdfs:range ex:Company .
""")

def local(iri):
    return str(iri).split("#")[-1]

def rel_name(iri):
    # worksFor -> WORKS_FOR, the usual relationship style in property graphs
    return re.sub(r"(?<!^)(?=[A-Z])", "_", local(iri)).upper()

node_types = sorted(
    ({"label": local(c), "description": str(ontology.value(c, RDFS.comment))}
     for c in ontology.subjects(RDF.type, OWL.Class)),
    key=lambda n: n["label"])

relationship_types = sorted(rel_name(p) for p in ontology.subjects(RDF.type, OWL.ObjectProperty))

patterns = sorted(
    (local(ontology.value(p, RDFS.domain)), rel_name(p), local(ontology.value(p, RDFS.range)))
    for p in ontology.subjects(RDF.type, OWL.ObjectProperty))

for n in node_types:
    print("node:", n["label"], "-", n["description"])
print("relationships:", relationship_types)
for p in patterns:
    print("pattern:", p)

    Output (Python 3.13, RDFLib 7.6.0):

    node: Company - A legal business entity.
node: Person - An individual human.
node: Product - A good or service a company sells.
relationships: ['ACQUIRED', 'MAKES', 'WORKS_FOR']
pattern: ('Company', 'ACQUIRED', 'Company')
pattern: ('Company', 'MAKES', 'Product')
pattern: ('Person', 'WORKS_FOR', 'Company')

    The properties use the same ex:worksFor and ex:makes IRIs as the validation example in the LLM article, and rel_name() converts them to the UPPER_SNAKE style property graphs usually expect. The domain and range of each property become the allowed patterns, and the rdfs:comment of each class becomes the description the LLM sees in its prompt. Good descriptions matter more than people expect. "A legal business entity" helps the model decide that a government agency or a product brand is not a Company. One limitation: ontology.value() assumes each property has exactly one domain and one range; a property with several, or none, needs a loop and a fallback. The modelling choices behind this file are covered in how to build an ontology in Python with RDFLib.

    Feeding the schema to a GraphRAG pipeline

    Different tools accept the schema in different ways:

    The Neo4j documentation describes the schema as a guiding structure for the language model. That is the right expectation. A schema in the prompt reduces drift; it does not eliminate it. Turn on enforcement or pruning where the tool offers it, and validate afterwards.

    Validating the extracted knowledge graph

    After extraction, check the knowledge graph against the ontology before it is used for retrieval. For RDF graphs, SHACL shapes do this directly. For property graphs, the same rules can be written as queries: find every WORKS_FOR edge whose source is not a Person, every Product with no MAKES edge pointing at it, and so on. The validation pattern for LLM output, with a runnable example, is in ontologies and LLMs.

    Entity resolution: the step that decides quality

    Even with fixed entity types, the same real-world thing appears under different names across documents. GraphRAG quality depends heavily on merging those nodes correctly. Neo4j's package includes fuzzy-matching and spaCy-based resolvers, according to its knowledge graph builder guide; other teams use embeddings plus an LLM to confirm matches.

    The ontology helps here too. Entities can only be merged within the same type, which removes a large class of false matches. Canonical identifiers from a reference list, such as company registry numbers or product SKUs from your own systems, give the resolver something definitive to match against. If your structured data is already integrated through a shared ontology, as described in ontology-based data integration, the document-derived graph can link directly to those existing entities instead of creating new ones.

    Is GraphRAG worth the cost?

    Indexing is the expensive part. The GraphRAG repository warns that indexing can be an expensive operation and advises starting small. Every chunk needs at least one LLM call for extraction, and every community needs a summary. On a large corpus with a strong model, that adds up quickly. The repository now also describes the project as being in maintenance mode, which is worth knowing before you build on it.

    Lighter alternatives exist:

    | Approach | Idea | Reported trade-off | |---|---|---| | Vector RAG | Retrieve similar chunks | Cheapest; weak on whole-corpus questions | | GraphRAG | LLM-built graph plus community summaries | Strong on global questions; costly indexing | | LazyGraphRAG | Defers LLM work to query time | Microsoft reports indexing at 0.1% of GraphRAG's cost | | LightRAG | Graph plus vector retrieval at two levels, incremental updates | Authors estimate under 100 retrieval tokens per query against about 610,000 for GraphRAG on one dataset | | Ontology-driven GraphRAG | Any graph approach with a fixed schema | Extra modelling effort; OG-RAG authors report about 55% higher recall of accurate facts |

    An ontology does not make indexing cheaper per chunk. It can make it cheaper overall, because a focused schema extracts fewer low-value entities, and because a consistent graph is reusable for structured queries and other applications, not just one RAG pipeline.

    Use GraphRAG when your users ask whole-corpus questions, when answers need several connected facts, or when you want the graph itself as a durable asset. Stay with vector RAG when questions are mostly local lookups and the corpus changes constantly. Add the ontology when the graph will be shared, queried directly, or combined with structured data.

    Common mistakes

    Building your first ontology-driven GraphRAG pipeline

    1. Write ten competency questions that plain RAG answers badly.
    2. Model the entity types and relationships those questions need, reusing terms from schema.org or your domain ontology. The guide to RDF, RDFS, OWL and SHACL covers the syntax.
    3. Generate the extraction schema from the ontology, as in the snippet above.
    4. Index a sample of a few hundred documents and measure cost.
    5. Validate the extracted graph and fix the schema descriptions where the model goes wrong.
    6. Compare answers to the competency questions against your existing vector RAG.

    If the graph still feels messy, check whether you need an ontology at all or just a cleaner taxonomy of topics; ontology vs taxonomy vs knowledge graph walks through that decision.

    Key takeaways

    Related articles in this series