GraphRAG & Knowledge Graphs

When Vector Search Fundamentally Fails

GraphRAG vs Naive RAG

Standard RAG is retrieval over isolated chunks. The question "What are the major themes across all of our quarterly earnings reports?" cannot be answered by Top-K retrieval. No single chunk contains a global answer. This is the problem Microsoft's GraphRAG was designed to solve.

Building a Knowledge Graph from Text

GraphRAG uses an LLM to parse documents and extract entities (Companies, People, Events) and their relationships (Acquired, Reported, Partnered) to construct a directed property Knowledge Graph. Nodes are entities. Edges are typed relationships with attributes (date, sentiment, etc.).

For global queries, GraphRAG doesn't retrieve chunks — it traverses graph communities (clusters of tightly connected nodes), generates summaries for each community, and synthesizes a final answer across summaries. This enables cross-document reasoning that defeats standard vector search.

Local vs. Global Query Routing

Smart GraphRAG systems route queries by type. Local queries ("What did Tim Cook say about AI in Q4 2024?") use standard vector retrieval since they target specific facts. Global queries ("What risks are mentioned across all board meetings?") use graph traversal and community summarization. Misrouting global queries to vector search is a common failure mode.

Neo4j as the Graph Backend

Production knowledge graphs live in dedicated graph databases like Neo4j. The Cypher query language allows direct relationship traversal that SQL and vector databases cannot express. LangChain has a native GraphCypherQAChain that converts natural language queries into Cypher queries via an LLM, executes them, and returns structured results.