Introduction to RAG and Knowledge Graphs - Understanding How Internal Search Works

What Is RAG?

RAG (Retrieval-Augmented Generation) is an AI technology that combines "retrieval" and "generation." It addresses a fundamental limitation of large language models (LLMs): while LLMs generate answers based on their pre-trained data, they cannot answer questions about internal company information or the latest data that was not part of their training set.

RAG solves this by first retrieving relevant information from a database, then passing that information to the LLM so it can generate a contextually informed answer.

How RAG Works

1. The user submits a question
2. The search engine retrieves relevant documents (Retrieval)
3. The retrieved documents are passed to the LLM as context
4. The LLM generates an answer informed by that context (Generation)

This mechanism enables accurate responses to company-specific questions that an LLM alone could not answer.

The Basics of Vector Search

The retrieval component of RAG relies on vector search. Traditional keyword search finds documents based on exact or partial word matches. Vector search, by contrast, searches based on the meaning of the text.

Specifically, it converts text into vectors (arrays of numbers) and calculates the distance between the question vector and document vectors to identify semantically similar documents.

For example:

• A keyword search for "remote work policy" finds "Remote_Work_Policy.pdf"
• A vector search for "rules for working from home" also finds "Remote_Work_Policy.pdf"

The latter illustrates the strength of vector search: it finds the right document even when the wording differs, as long as the meaning is close.

What Is a Knowledge Graph?

A knowledge graph is a database that represents relationships between pieces of information in a network structure.

While conventional databases store data in table format, a knowledge graph explicitly records relationships such as "A is related to B" or "C is part of D."

The Problem Knowledge Graphs Solve

Internal documents are interconnected. Product specifications relate to manuals, manuals relate to FAQs, and FAQs relate to inquiry histories. However, vector search alone cannot fully exploit these relationships.

By incorporating a knowledge graph, search results can include related contextual information from connected documents, enabling more comprehensive and context-aware answers.

GraphRAG: The Fusion of RAG and Knowledge Graphs

GraphRAG is a technology that combines RAG's vector search with the relationship data stored in a knowledge graph. This fusion improves both search precision and the comprehensiveness of answers compared to traditional RAG.

Comparing Traditional RAG and GraphRAG

Feature	Traditional RAG	GraphRAG
Search method	Vector search only	Vector search + graph traversal
Inter-document relationships	Not considered	Automatically traced
Answer comprehensiveness	Only directly relevant documents	Includes indirectly related information
Handling complex queries	Limited	Strong

TIMEWELL's ZEROCK implements GraphRAG technology for enterprise use. It automatically maps relationships between internal documents and can handle complex questions such as "Which technical specifications in our sales materials are related to last month's negotiation with Client A?"

Making Enterprise Search Work in Practice

To effectively operate an internal search system powered by RAG or GraphRAG, several factors deserve attention.

Data Preparation

The quality of the data the AI searches directly determines the quality of its answers. Pre-processing steps such as archiving outdated documents, adding metadata (titles, creation dates, departments), and consolidating duplicate documents are essential.

Chunk Design

The granularity used to split long documents into searchable units (chunks) also affects accuracy. If chunks are too small, context is lost; if they are too large, irrelevant information creeps in.

Building a Feedback Loop

Collecting user feedback on whether answers were helpful or not, and using that feedback to improve search accuracy, is another important practice.

Summary

• RAG combines retrieval and generation to answer questions that an LLM alone cannot handle
• Vector search finds documents based on semantic similarity rather than keyword matching
• Knowledge graphs manage relationships between pieces of information in a network structure
• GraphRAG fuses RAG with knowledge graphs to deliver more precise and comprehensive search results

In the next article, we cover the security challenges that enterprises must address when deploying AI.