Skip to content
MiraiTech
AI Lab

Experiment · RAG

PDF to knowledge graph

A pipeline that extracts entities, relationships and claims from PDFs with a language model, stores them in Neo4j and draws them with D3. Every node links back to its source passage. We have used it on papers, contracts and technical specs.

Type
RAG
Status
Live
Models
Claude Sonnet, GPT-4o

Results

  • Entity extraction F1 of 0.89
  • Interactive graph view
  • Every node linked to its source

Entity extraction F1 was 0.89 across 40 documents, and relationship extraction F1 was 0.82. Processing a 20-page document took 45 seconds on average. The graph view renders up to 500 nodes at 60fps. In user testing, exploratory analysis was 2.3x faster than with keyword search.

What we tested

A pipeline that extracts entities, relationships and claims from PDFs, builds a queryable knowledge graph in Neo4j and renders it as an interactive D3.js visualization. Instead of reading a 50-page research paper from start to finish, you can explore its concepts, methods and findings as a connected graph.

Method

We tested the pipeline on 40 documents: 15 ML/AI research papers, 10 legal contracts, 10 technical specifications and 5 annual reports. LlamaParse extracts structured text from the PDFs. Claude Sonnet then extracts entities (people, organizations, concepts, methods, metrics) and relationships (uses, cites, compares, contradicts, extends) in two passes. Neo4j stores the graph with indexes on entity and relationship types. We measured entity extraction F1 against human-annotated ground truth.

Findings

  1. 01

    Two passes raised F1 from 0.79 to 0.89

    A single LLM pass for entities and relationships together scored 0.79 F1. Extracting entities first, then relationships between the known entities, raised F1 to 0.89, mainly because fewer relationships were missed.

  2. 02

    The graph shows links across documents

    With several papers loaded, the graph shows connections the authors never made explicit: shared methods, contradictory findings and citation chains that trace a line of research.

  3. 03

    Browsing the graph beat keyword search

    In user testing (n=10), participants answered open-ended questions 2.3x faster with the graph than with keyword search in the original PDFs.

  4. 04

    Every node links to its source

    Each node and edge links back to the page and paragraph it came from in the source PDF. Clicking a relationship opens the passage, so users can check what was extracted.

How it is built

LlamaParse turns each uploaded PDF into structured text blocks with page and position metadata. Claude Sonnet extracts entities from each block, then receives the entities back with the text to extract relationships. The resulting triples (entity, relationship, entity) are written to Neo4j with source citations. The D3.js frontend queries Neo4j through a Cypher API and draws entities as nodes and relationships as edges, colored by type, with interactive filtering.

LlamaParse handles layout-aware PDF parsing. Claude Sonnet extracts entities and relationships using structured prompts. Neo4j stores the knowledge graph and provides full-text search and Cypher queries. D3.js renders the interactive force-directed graph, and Next.js serves the frontend and runs graph queries on the server.

  • Neo4j
  • LlamaParse
  • D3.js
  • Next.js
  • FastAPI

Problems we hit

Entity coreference resolution

The same entity appears under different names ("GPT-4", "GPT4", "OpenAI's latest model"). A step that combines fuzzy matching with embedding similarity merges the variants into one canonical node with a list of aliases.

Large graphs were hard to read

Graphs with 300+ nodes became too dense to read. Hierarchical clustering now groups related entities into collapsible super-nodes, and semantic zoom reveals more detail as users focus on a region.