Skip to content

Olink Knowledge Graph Platform

A production system for building and querying biomedical knowledge graphs from scientific literature. Extracts protein-disease relationships from papers using LLMs, stores them in graph databases, and exposes a RAG-powered API for intelligent queries.

Architecture Overview

graph TB
    subgraph Sources["Data Sources"]
        PM[PubMed]
        PMC[PMC Full-Text]
        BR[bioRxiv]
        PDF[Local PDFs]
        CSV[CSV/Ontologies]
    end

    subgraph Ingestion["Ingestion Pipeline (bioingest)"]
        FT[Fetch & Extract Text]
        CH[Token Chunker]
        LLM[LLM Entity Extraction]
        SC[Schema Validation]
        ER[Entity Resolution]
    end

    subgraph Storage["Storage Layer"]
        NP[Neptune / Neo4j<br/>Graph Database]
        AU[Aurora pgvector<br/>Embeddings]
    end

    subgraph Serving["Query Serving (graphrag-api)"]
        API[REST API<br/>FastAPI + Granian]
        AG[Query Agents<br/>Two-Phase / Dynamic]
        KT[KG Tools<br/>Cypher + Vector Search]
    end

    subgraph Clients["Clients"]
        FE[React Frontend]
        PL[00panel Dashboard]
        CL[curl / SDK]
    end

    Sources --> FT
    FT --> CH --> LLM --> SC --> ER
    ER --> NP
    CH --> AU

    NP --> KT
    AU --> KT
    KT --> AG --> API

    API --> FE
    API --> PL
    API --> CL

    style Sources fill:#e8f4fd,stroke:#2196F3
    style Ingestion fill:#fff3e0,stroke:#FF9800
    style Storage fill:#e8f5e9,stroke:#4CAF50
    style Serving fill:#f3e5f5,stroke:#9C27B0
    style Clients fill:#fce4ec,stroke:#E91E63

How It Works

Step What happens Where
1. Fetch Papers downloaded from PubMed/PMC/bioRxiv or PDFs loaded bioingest
2. Chunk Text split into 512-token segments with overlap bioingest
3. Extract LLM (Bedrock) identifies proteins, diseases, relationships bioingest
4. Validate Schema enforces entity types and filters garbage bioingest
5. Resolve Duplicates merged via UniProt ID, MONDO ID, fuzzy matching bioingest
6. Store Nodes/edges → Neptune, embeddings → Aurora pgvector bioingest
7. Query User asks question → agent searches graph + vectors → LLM synthesizes answer graphrag-api

Repositories

Repo Role Key Entry Points
graphrag-api Query serving, REST API, agents api/app.py, src/agents/
bioingest Data fetch + KG creation uv run bioingest ingest
graphrag_eval Benchmarking + evaluation uv run python -m benchmark
graphrag_react React frontend npm run dev

Quick Start

aws sso login
cd graphrag-api && uv sync
uv run python ingest_main.py --source pdf --pdf-files paper.pdf --service bedrock
uv run granian --interface asgi api.app:app --host 0.0.0.0 --port 8000

See Getting Started for the full setup guide.

Quick Navigation

Section Description
Getting Started Installation, configuration, first KG build
Build Your First KG 15-minute hands-on walkthrough
CLI Reference All commands and flags
Architecture System design, data model
Schema Expansion PTMs, variants, data model discussion
Ingestion Pipeline Sources, extraction, consolidation
Query System Agents, modes, search depth
API Reference REST endpoints, SSE streaming
Neptune Production ingestion runbook
Operations Maintenance, incremental updates

Production Stats

Backend Nodes Relationships Embeddings
Neptune (beta) 70K 263K
Aurora pgvector 216K

Tech Stack

  • API: Python 3.12 / FastAPI / Granian (Rust ASGI)
  • Graph: Neptune (serverless) / Neo4j
  • Vectors: Aurora pgvector (768-dim, cosine)
  • LLM: AWS Bedrock (Llama 3.3, Nova Pro, Claude)
  • Frontend: React + Sigma.js
  • Infra: AWS CDK / ECS Fargate / EventBridge
  • CI: GitHub Actions (two-layer Docker: base cached, app fast)