What is GraphRAG and when to use it

GraphRAG in brief

GraphRAG models relationships between document chunks as a graph so the retrieval-generation process can preserve connected context. It complements pure vector search with relationship and hierarchy signals that are easy to miss.

When to use it

• Domains where relationships matter (wikis, policy docs, org charts, dependency graphs)
• When “A influences B” style relationships are central to the answer
• When connectivity across documents affects quality more than any single document

Design considerations

1. Chunk strategy: Manage sections plus linking nodes (titles/headers) as nodes together.
2. Graph construction: Create edges from links, heading hierarchy, and shared entities.
3. Hybrid retrieval: Narrow candidates with vector search, then expand with graph neighbors.

Example flow

1. Embed the question and retrieve top K documents with vector search.
2. Expand graph neighbors of those documents to increase candidate context.
3. Deduplicate and summarize/normalize before sending to the LLM.

Caveats

• Expanding the graph too far increases noise.
• Unclear relation definitions can confuse the model.
• Build cost is high, so adopt selectively for MVPs.

Quick checklist

• Is vector search alone insufficient for accuracy?
• Is relationship information central to the question?
• Is the impact worth the build and ops cost?

Ultimately, GraphRAG excels when relationship-based context is critical.