Skip to content

Platform Overview

Architecture

graph TB
    subgraph Sources["Data Sources"]
        PM[PubMed]
        PMC[PMC Full-Text]
        BR[bioRxiv]
        PDF[Local PDFs]
        CSV[CSV/Ontologies]
    end

    subgraph Ingestion["Ingestion Pipeline (bioingest)"]
        FT[Fetch & Extract Text]
        CH[Token Chunker]
        LLM[LLM Entity Extraction]
        SC[Schema Validation]
        ER[Entity Resolution]
    end

    subgraph Storage["Storage Layer"]
        NP[Neptune / Neo4j<br/>Graph Database]
        AU[Aurora pgvector<br/>Embeddings]
    end

    subgraph Serving["Query Serving (graphrag-api)"]
        API[REST API<br/>FastAPI + Granian]
        AG[Query Agents<br/>Two-Phase / Dynamic]
        KT[KG Tools<br/>Cypher + Vector Search]
    end

    subgraph Clients["Clients"]
        FE[React Frontend]
        PL[Panel Dashboard]
        CL[curl / SDK]
    end

    Sources --> FT
    FT --> CH --> LLM --> SC --> ER
    ER --> NP
    CH --> AU

    NP --> KT
    AU --> KT
    KT --> AG --> API

    API --> FE
    API --> PL
    API --> CL

    style Sources fill:#e8f4fd,stroke:#2196F3
    style Ingestion fill:#fff3e0,stroke:#FF9800
    style Storage fill:#e8f5e9,stroke:#4CAF50
    style Serving fill:#f3e5f5,stroke:#9C27B0
    style Clients fill:#fce4ec,stroke:#E91E63

How It Works

Step What happens Where
1. Fetch Papers downloaded from PubMed/PMC/bioRxiv or PDFs loaded bioingest
2. Chunk Text split into 512-token segments with overlap bioingest
3. Extract LLM (Bedrock) identifies proteins, diseases, relationships bioingest
4. Validate Schema enforces entity types and filters garbage bioingest
5. Resolve Duplicates merged via UniProt ID, MONDO ID, fuzzy matching bioingest
6. Store Nodes/edges → Neptune, embeddings → Aurora pgvector bioingest
7. Query User asks question → agent searches graph + vectors → LLM synthesizes answer graphrag-api

Repositories

Repo Role Links
graphrag-api Query serving, REST API, agents Wiki
graphrag_eval Benchmarking + evaluation Wiki
bioingest Data fetch + KG creation Wiki
graphrag_react React frontend Prod

Tech Stack

  • API: Python 3.12 / FastAPI / Granian (Rust ASGI)
  • Graph: Neptune (serverless) / Neo4j
  • Vectors: Aurora pgvector (768-dim, cosine)
  • LLM: AWS Bedrock (Llama 3.3, Nova Pro, Claude)
  • Frontend: React + Sigma.js
  • Infra: AWS CDK / ECS Fargate / EventBridge
  • CI: GitHub Actions (two-layer Docker: base cached, app fast)

Production Stats

Backend Nodes Relationships Embeddings
Neptune (beta) 70K 263K —
Aurora pgvector — — 216K