Ontology, Knowledge Graph, RAG, and GraphRAG: The Practical Differences

In Part 1, we summarized when an ontology and knowledge graph are needed for an AI agent.

The point was simple.

Ontology/KG is not needed because it is an AI Agent, but it is needed because semantic inconsistency, authority, rules, and data connection problems increase in organizational unit automation.

In this second part, we will go into a bit more technical details.

Ontology, knowledge graph, RAG, GraphRAG, SHACL, and SPARQL often appear together, but their roles are different. Failure to distinguish between these roles creates two problems.

First, knowledge graphs are excessively introduced to problems that can be solved with RAG.

Second, on the contrary, we are trying to solve problems that require verification of authority, rules, and relationships with a simple RAG.

So the goal of this article is clear.

We will organize what each technology solves and where it is placed within the AI ​​Agent architecture from a practical perspective.

Analysis base date: 2026-05-01 Key references: W3C OWL/SHACL/SPARQL, Microsoft GraphRAG, OpenAI/Anthropic agent design data Analysis scope: Not the entire theoretical semantics, Practical architecture perspective required for organizational AI automation

---

Key takeaways

• Ontology is a conceptual blueprint of the world of work. Define classes, relationships, constraints, and semantics.
• Knowledge Graph is a network of actual business objects. Connect customers, contracts, documents, tickets, contacts, and policies.
• RAG ​​is a method of finding relevant pieces of documentation and providing them to the LLM. It is strong in document-based question and answering.
• GraphRAG ​​is a method of constructing search and summary context using graph structures. Strong on questions that deal with relationships and overall structures.
• SHACL is a rule layer that verifies whether the graph satisfies certain conditions, and SPARQL is a language for querying RDF graphs.

---

1. Summary of Part 1: Why do you need a semantic hierarchy?

When an AI Agent assists an individual or team with work, the following combination is usually sufficient.

Copy1# Example structure2prompt3+ Example4+ RAG5+ Call tool6+ Checklist

For example, tasks such as summarizing meeting minutes, drafting code reviews, document retrieval, or ticket creation work quite well with this structure.

But things change when scaled up to organizational units.

Copy1# Example structure2Different systems store the same customer differently.3The same word ‘done’ is interpreted differently by each team.4Even for the same action, whether or not it can be executed varies depending on the user role.5Even for the same report, you must check the supporting documents and approval status.

What you need here isn't just longer context. A semantic layer is needed that structures organizational concepts, relationships, and rules.

The technologies we will cover in Part 2 are the elements that make up this semantic layer.

Remembering this table will make future concepts much clearer.

---

2. What is ontology?

An ontology is a conceptual model that describes an organization's world of work.

More practically speaking, it is a structure that answers the following questions:

Copy1# Example structure2What business objects do we have in our organization?3What properties do those objects have?4How are the objects related to each other?5What relationships must exist?6Which relationships should not exist simultaneously?7Are certain actions possible only under what conditions?

For example, the ontology of a customer onboarding task might start like this:

Copy1# Example structure2Class:3- Customer4- Contract5- SecurityReview6- OnboardingChecklist7- Person8- Team9- Policy10- Permission11- Evidence12- Action13 14Relation:15- Customer hasContract Contract16- Customer assignedTo Person17- Person belongsTo Team18- Contract hasStatus ContractStatus19- Action requires Permission20- Decision basedOn Evidence21- Policy constrains Action

W3C OWL allows this knowledge to be expressed in structures such asclass,property,individual, andrestriction. OWL 2 Primer describes objects as individuals, categories as classes, and relationships as properties. Properties are also described as relationships between objects and data value relationships.

In practice, there is no need to look at ontology too grandly.

Ontology is a blueprint that specifies concepts and relationships that AI should not be confused about in an organization.

With ontology, AI Agent can interpret words such as “customer,” “contract,” “approval,” “completion,” and “basis” into organizational standards.

Of course, ontology does not solve all problems. Ontologies are not real data. This is a blueprint. The knowledge graph is responsible for determining where Customer A, Contract B, and Document C are and how they are connected.

---

3. What is a knowledge graph?

The knowledge graph is a data structure that connects actual business objects and relationships.

Let’s say ontology is defined like this.

Copy1# Example structure2Customer hasContract Contract3Contract approvedBy Person4Decision basedOn Evidence5Action requires Permission

The knowledge graph connects actual data like this.

Copy1# Example structure2Customer:AcmeCorp3→ hasContract → Contract:ACME-2026-0014→ assignedTo → Person:KimCSM5→ belongsToIndustry → Industry:Finance6 7Contract:ACME-2026-0018→ hasStatus → Signed9→ approvedBy → Person:LegalLead10→ basedOn → Document:SecurityReviewReport11 12Action:StartOnboarding13→ requiresPermission → Permission:OnboardingStart14→ constrainedBy → Policy:FinanceCustomerPolicy

The strength of graphs is that they allow you to follow relationships.

For example, for an AI Agent to determine “Can I start onboarding this customer?”, it would follow the following relationship:

Copy1# Example structure2Customer to contract to contract status3Customer to security review to review status4Customer to industry to applied policy5Requester to Team/Role to Permissions6Action to Required Permissions to Policy Constraints7Decision to supporting documentation to audit log

Simple document search can find related text, but this relationship-based state verification requires a separate structure.

In the end, the knowledge graph is not a technology for decorating search results, but an operational data structure for connecting and verifying organizational business objects.

---

4. Difference between ontology and knowledge graph

Ontology and knowledge graph are often used together, but they are not the same thing.

The easiest analogy is this:

Copy1# Example structure2Ontology = Blueprint3Knowledge graph = Network actually built according to the blueprint

A little more technically, it can be explained by dividing it into T-Box and A-Box.

In practice, understanding it this way is sufficient.

For example, “a contract must have an approver” is the domain of ontology or validation rules. “The approver of contract B is Kim OO” is the area of ​​the knowledge graph.

Only when these two come together can AI Agent use organizational knowledge reliably.

---

5. What does RAG do well and what does it do poorly?

RAG is short for Retrieval-Augmented Generation. Simply put, it searches for document fragments related to the user's question and then provides the content to LLM to generate an answer.

RAG is very useful in organizational AI automation.

Copy1# Example structure2user questions3↓4Vector Search / Keyword Search5↓6Search related document chunks7↓8Providing context to the LLM9↓10Generate Answer

Here’s what RAG does well:

But RAG also has limitations.

A document chunk is basically a piece of text. If relationships among customers, contracts, contact persons, approvals, authorities, and policies are not explicitly outlined, the LLM must infer them from the text.

So while RAGs are great for document-based question and answering, they don't solve all the problems of automating organizational units.

Practical Judgment If the question is “Find and summarize relevant documents,” RAG comes first. If the question is “Check the relationships and states of multiple objects and determine whether they are executable,” a graph and validation rules become necessary.

---

6. What problem does GraphRAG solve?

GraphRAG is an approach that combines RAG with graph structures.

The Microsoft GraphRAG documentation describes GraphRAG as a structured and hierarchical RAG approach in contrast to simple text chunk-based semantic search. This method extracts a knowledge graph from raw text, creates a community layer and summary, and then uses it for RAG work.

To simplify, the flow goes like this.

Copy1# Example structure2document corpus3↓4Entity/Relationship/Assertion Extraction5↓6Create a knowledge graph7↓8Community detection and summary9↓10Search for graph context that fits your question11↓12Generate LLM Answers

The questions that GraphRAG has an advantage over are not the typical “find the right paragraph” questions, but ones that require looking at the structure of the entire dataset.

For example:

Copy1# Example structure2Link contracts, contacts, policies, and trouble tickets related to this customer issue.3Please summarize the main themes of customer complaints last quarter and the related product groups.4Please let me know in which teams and documents this project's risks appear repeatedly.5Summarize the security exception approval flow scattered across multiple documents.

The comparison between RAG and GraphRAG is as follows:

However, GraphRAG is not always better than regular RAG.

Creating a graph requires entity extraction, relationship extraction, deduplication, community summarization, and quality verification. If your data is small and your questions are simple, a regular RAG may be faster and more economical.

caution GraphRAG is not “upwards compatible with RAG.” Effectiveness increases when data is relational, questions cross multiple documents and objects, and tracking evidence is important.

---

7. Why is SHACL important for organization rule verification?

The biggest risk in organizational AI automation isn't just that the AI ​​gives the wrong answer.

The greater risk is executing actual system actions based on plausible judgment.

For example, something like this might happen:

Copy1# Example structure2Unauthorized Contract Changes3Sending unauthorized customer notices4Onboarding begins before security screening5Issuance of reports without supporting documents6Complete completion of tickets with incomplete status

These problems are difficult to prevent with just a prompt. Before execution, data and actions must be mechanically verified to ensure that they satisfy organizational rules.

This is where SHACL comes into play.

W3C SHACL is a language that verifies whether an RDF graph satisfies certain conditions. In SHACL, verification conditions are expressed as shapes graph, and it is checked whether the actual data graph satisfies the conditions.

For example, you might consider the following rule:

Copy1# Example structure2Contract change actions must have an approver.3The approver must have the authority to change the contract.4Customer onboarding actions require passing a security screening.5The decision to issue a report must have at least one supporting document.

Expressing this in SHACL style, it has the following form: This is a concept example rather than actual operational code.

Copy1# example structure2:OnboardingActionShape3  a sh:NodeShape ;4  sh:targetClass :OnboardingAction ;5  sh:property [6    sh:path :requiresSecurityReview ;7    sh:minCount 1 ;8  ] ;9  sh:property [10    sh:path :basedOnEvidence ;11    sh:minCount 1 ;12  ] .

In practical terms, the value of SHACL is:

An LLM can help with judgment, but final validation of organizational rules should not be left to the LLM alone.

Practical Judgment If the AI ​​Agent makes changes to the actual system, a mechanical verification layer is needed separate from the prompt guardrail. SHACL is a representative standard that can play a role in an RDF graph-based environment.

---

8. SPARQL can be a graph query tool for AI Agent

SPARQL is a language for querying RDF graphs.

The W3C SPARQL 1.1 document explains that SPARQL is a query language for RDF graphs, and can query not only data stored as RDF, but also a variety of data sources that look like RDF through middleware.

From an AI Agent perspective, SPARQL can be thought of as “SQL for graph databases.” Of course, the syntax and data model are different, but the roles are similar.

For example, let's say the Agent needs to answer these questions:

Copy1# Example structure2Check Customer A’s contract status, person in charge, security review status, and applicable policies.

Conceptually, a SPARQL query looks like this.

Copy1# example structure2SELECT ?contract ?contractStatus ?csm ?reviewStatus ?policy3WHERE {4  :CustomerA :hasContract ?contract .5  ?contract :hasStatus ?contractStatus .6  :CustomerA :assignedTo ?csm .7  :CustomerA :hasSecurityReview ?review .8  ?review :hasStatus ?reviewStatus .9  :CustomerA :constrainedBy ?policy .10}

If you hand over the results of this query to LLM, LLM can generate answers much more reliably.

Copy1# Example structure2Customer A's contract is in the Signed state.3The CSM in charge is KimCSM.4The security screening status is Passed.5The applicable policy is FinanceCustomerPolicy.6Therefore, the conditions for starting onboarding appear to have been met.7However, verification of the user's OnboardingStart permission is required before actual execution.

In AI Agent architecture, SPARQL can be registered as a tool.

Copy1# Example structure2Tool: queryKnowledgeGraph3Input: customerId, actionType4Output: Relevant subgraphs, states, supporting documents, policies

The important thing is not to put the entire graph into the prompt.

Practical Tips Rather than showing the entire graph to the LLM, you should query and provide only the subgraphs needed for the question. The graph is not the context itself, but rather an index and query tool that cuts into the context.

---

9. How to connect to the AI ​​Agent execution flow

Now let's put each element into one execution flow.

Copy1# Example structure2user request3↓4Understand intent5↓6Interpreting business concepts with ontology7↓8Search related objects with knowledge graph9↓10Search for evidence context with RAG/GraphRAG11↓12Condition validation with SHACL/policy engine13↓14run tool15↓16Record execution results and rationale in a graph

The role of each step is as follows:

Here, LLM does not judge everything directly. The role of an LLM is close to:

Copy1# Example structure2Request Interpretation3Determine what information you need4Establish a tool call plan5Interpretation of verification results6Explain to the user

On the other hand, things that need to be handled mechanically are left to a separate layer.

Copy1# Example structure2Check permissions3Policy verification4Required field validation5State transition verification6Audit log records

This separation is important.

The key to operating an AI Agent is not to leave all judgment to LLM, but to make LLM run on organizational knowledge and verification rules.

---

10. Practical architecture patterns

Several patterns are possible for actual adoption in an organization.

Pattern A. RAG-centric + minimal glossary

This is the simplest structure.

Copy1# Example structure2document storage3↓4Vector DB5↓6RAG7↓8LLM Answers

Suitable situations include:

• Team internal document search
• Policy document summary
• Organize meeting minutes
• Code/document-based Q&A
• Tasks with limited execution actions

At this stage, a common glossary may be sufficient instead of an ontology.

Pattern B. RAG + Business Object Mapping

Add business object mapping to document search.

Copy1# Example structure2document chunk3+ Customer ID4+ Contract ID5+ Project ID6+ Ticket ID7↓8Hybrid Search

Suitable situations include:

• Search for specific customer-related documents
• Organize knowledge by project
• Link tickets and documents
• Analysis of customer support issues

At this stage, even if it is not a complete knowledge graph, simply attaching entity IDs consistently will be effective.

Pattern C. Ontology + KG + RAG

It is a structure close to organizational automation.

Copy1# Example structure2Ontology3↓4knowledge graph5↔ Existing DB/API/document storage6↓7RAG/GraphRAG8↓9Run Agent tool

Suitable situations include:

• Multiple teams use the same customer/contract/policy
• Requires permission and status verification
• Link multiple system data
• Evidence tracking and audit logs required

Pattern D. KG + Policy Engine + Agent Operating System

This is the most advanced structure.

Copy1# Example structure2User/Agent Request3↓4policy engine5↓6Knowledge graph inquiry7↓8Execute validation rules9↓10Authorization or Human-in-the-loop11↓12run tool13↓14Audit log records

This pattern becomes necessary in high-risk environments such as finance, healthcare, security, and automation within large corporations.

---

11. Risk and adoption judgment

Ontologies and knowledge graphs are powerful, but they have operational risks.

construction risk

• Conceptual models can become overly large.
• Modeling only developers without domain experts may miss out on practical business implications.
• Trying to connect too many systems at once can result in a PoC that never ends.

Data quality risk

• The same customer may be stored with multiple IDs.
• There may be errors in the entities and relationships extracted by LLM.
• Older documents can be linked as if they were current policies.

operational risk

• An incorrect graph may become the basis for the Agent's judgment.
• Incidents can escalate if tool execution is linked without permission verification.
• There may be discrepancies between graph updates and the actual system state.

Security Caution If the AI ​​Agent changes the actual business system, graph query and document search alone are not enough. Permissions, authorizations, policies, audit logs, and human-in-the-loop design must be viewed together.

Judgment of adoption can be made based on the following criteria.

---

12. Checklist, Q&A, reference materials

Practical Checklist

Copy1# Example structure2Architecture Design Checklist3 4[ ] Is this a document retrieval problem or a relationship verification problem?5[ ] We separated questions that required RAG from those that required KG.6[ ] Core business objects are defined.7[ ] There is a way to map the same object in multiple systems.8[ ] The classes and relationships in the ontology are defined to a minimum extent.9[ ] There is a set method for handing over graph search results to LLM.10[ ] Rules to be verified by SHACL or policy engine are organized.11[ ] There is a permission verification procedure before executing the tool.12[ ] There is a log structure to record execution results and rationale.13[ ] There is a process for human review of graph quality.

Q1. Are ontology and database schema the same?

It's not the same. Database schema is closer to a storage structure, and ontology is closer to business concepts and semantic relationships. Of course the two can be connected.

Q2. If I create a knowledge graph, don’t I need RAG?

no. Knowledge graph is strong on objects and relationships, while RAG is strong on document context. In organizational automation, the two are often used together.

Q3. Does using GraphRAG necessarily improve performance?

no. If your question is more of a simple document retrieval and your data is small, a plain RAG may be more efficient. GraphRAG becomes valuable for questions that deal with relationships and overall structures.

Q4. Is it necessary to use SHACL?

If you use RDF-based graphs, SHACL is a powerful option. However, not all organizations must use SHACL. It can also fulfill some roles as a policy engine, type system, and business rules engine.

Q5. Can I have an LLM write SPARQL themselves?

It's possible, but risky. In an operational environment, it is recommended to limit allowed query templates, parameter validation, and access rights. Leaving LLM to run arbitrary queries can lead to data leaks and performance issues.

---

References and Uncertainty

References

• W3C, OWL 2 Web Ontology Language Primer
• W3C, OWL Web Ontology Language Overview
• W3C, Shapes Constraint Language SHACL
• W3C, SPARQL 1.1 Query Language
• Microsoft, GraphRAG Documentation
• Microsoft Research, Project GraphRAG
• OpenAI, A practical guide to building agents
• Anthropic, Building Effective AI Agents

confirmed facts

• OWL is an ontology language that expresses classes, properties, individuals, relationships, etc.
• SHACL is a W3C standard for conditionally validating RDF graphs.
• SPARQL is a W3C standard for querying RDF graphs.
• Microsoft GraphRAG describes a structured approach to creating knowledge graphs and community summaries from raw text and leveraging them in RAGs.

Author's interpretation

• In organizational AI Agent architecture, RAG and KG are not competitive, but complementary.
• Although LLM helps with business judgment, it is safe to separate authority and rule verification into separate layers.

uncertainty

• The effectiveness of GraphRAG largely depends on data structure, question type, extraction quality, and summary quality.
• In actual operation, OWL/SHACL/SPARQL may be used altogether, only some of them, or replaced with other policy engines.

---

finish

In summary, the important thing in organizational AI automation is not that LLM reads longer documents.

What is important is which concepts should be interpreted within which relationships and rules.

Ontology is the blueprint of these concepts and relationships. The knowledge graph is a network of actual business objects. RAG takes the document context, and GraphRAG uses relationships and structures to construct the context. SHACL verifies the condition, and SPARQL queries the required subgraph.

If you summarize it in one line, it is like this.

Organizational AI Agent is not complete with just one LLM. The semantic layer and validation layer must be designed together so that the LLM can read, validate, and act on organizational knowledge.

The next three parts will cover how to introduce this structure into a real organization. Rather than creating a company-wide KG, we will organize a step-by-step strategy that starts with a minimal ontology from one core task.

---

summary card

• One-line summary: RAG finds documents, KG links relationships, and SHACL verifies execution conditions.
• Recommended for: Developers, data engineers, and platform teams designing AI Agent architecture
• Not recommended for: Teams that only need a simple document summary.
• The most important setting: Provide LLM with only the subgraphs it needs, not the entire graph.
• Biggest risk: misunderstanding GraphRAG as upward compatible with regular RAG.
• What to do right now: Categorize current business questions into “document retrieval type” and “relationship verification type”