What is GraphRAG? Cross-document relationships and a Neo4j example

Vector retrieval finds passages with meaning close to a question. But a contract may never mention an acquisition, while the acquisition notice never mentions company A. A graph can store A→contract→B and C→acquisition→B, connecting both pieces through B to gather the evidence.

Neo4j GraphRAG Python connects retrievers and LLMs. Options include VectorCypherRetriever, which expands vector results with connected entities, and Text2CypherRetriever, which translates questions into Cypher. Building a small graph manually first makes the information to extract easier to understand.

Start on Mac, Windows, or Linux with Docker running. The command downloads Neo4j Community 2026.09.0 and binds both ports to your own computer. Replace replace_this_password with your chosen password before running it. The graphrag-demo-data volume stores the exercise data.

docker run -d --name graphrag-demo \
  -p 127.0.0.1:7474:7474 -p 127.0.0.1:7687:7687 \
  -e NEO4J_AUTH=neo4j/replace_this_password \
  -v graphrag-demo-data:/data \
  neo4j:2026.09.0

After startup, open http://127.0.0.1:7474.

Log in as neo4j with your chosen password. Run the first query in the query editor to create Demo A, B, and C, plus contract and acquisition relationships. Each relationship also carries fictional source filenames, page numbers, and an acquisition date.

MERGE (a:DemoCompany {name: 'Demo A'})
MERGE (b:DemoCompany {name: 'Demo B'})
MERGE (c:DemoCompany {name: 'Demo C'})
MERGE (a)-[contract:CONTRACTED_WITH]->(b)
SET contract.source = 'demo-contract.pdf', contract.page = 1
MERGE (c)-[purchase:ACQUIRED]->(b)
SET purchase.date = '2026-10-01',
    purchase.source = 'demo-notice.pdf', purchase.page = 2
RETURN a, b, c;

The second query follows A's supplier and then the company that acquired that supplier. Expected results are Demo B, Demo C, 2026-10-01, and both source filenames. Returning evidence with the answer lets you check a mistaken relationship against the source.

MATCH (:DemoCompany {name: 'Demo A'})-[contract:CONTRACTED_WITH]->(b)
MATCH (c:DemoCompany)-[purchase:ACQUIRED]->(b)
RETURN b.name AS supplier, c.name AS acquirer,
       purchase.date AS acquisition_date,
       contract.source AS contract_source,
       purchase.source AS notice_source;

Real PDFs require text extraction and entity/relation extraction before writing the graph. Resolve alternate company names to the same entity, and preserve relationship direction, dates, and sources. Start with two or three documents and review the extracted connections: one incorrect acquisition link can affect later answers.

Use Neo4j GraphRAG's OllamaLLM and OllamaEmbeddings to connect local models. The chat model writes answers; the embedding model supports vector retrieval. Python running inside Docker needs a separate Ollama endpoint, since container localhost points to the container rather than the host.

Compare on the same corpus and questions, separating single-fact lookups from cross-document relationship questions. Record whether the right evidence is retrieved and how long answers take. Expand the graph when relationship queries benefit; otherwise keep vector retrieval. If the browser does not open, check startup logs and port conflicts with docker logs graphrag-demo.