EpiGraph

Building Generalists for Evidence-Intensive Epilepsy Reasoning in the Wild

A knowledge-graph-powered benchmark and code release for evaluating whether AI systems can reason across epilepsy literature, EEG findings, genes, treatments, and clinical outcomes.

EpiGraph Interactive Project Page · Paper: arXiv:2605.09505

48,166 Papers · 24,324 Entities · 32,009 Triplets · 5 Evidence-Intensive Epilepsy Reasoning Tasks

How to Cite · News · Why EpiGraph · Key Features · Quick Start · Tasks · Metrics

---

How To Cite

If you use EpiGraph, EpiKG, EpiBench, the Graph-RAG pipeline, or this code release, please cite the arXiv version:

@article{dai2026epigraph,
  title={EpiGraph: Building Generalists for Evidence-Intensive Epilepsy Reasoning in the Wild},
  author={Dai, Yuyang and Chen, Zheng and Pradeepkumar, Jathurshan and Matsubara, Yasuko and Sun, Jimeng and Sakurai, Yasushi and Dong, Yushun},
  journal={arXiv preprint arXiv:2605.09505},
  eprint={2605.09505},
  archivePrefix={arXiv},
  url={https://arxiv.org/abs/2605.09505},
  year={2026}
}

---

News

• 2026-05-13 - EpiGraph is available on arXiv as arXiv:2605.09505.
• 2026-05-13 - The project page now includes a responsive interactive KG explorer with search presets, node inspection, edge inspection, and mobile-friendly layouts.
• 2026-05-10 - The code release includes five paper-aligned EpiBench task runners, Graph-RAG retrieval, metrics, and a private-data-aware adapter for the Harvard EEG task.

---

Why EpiGraph

Modern medical AI is moving from short-form question answering toward evidence-intensive clinical reasoning: connecting literature, mechanisms, phenotypes, EEG patterns, genetic biomarkers, treatment choices, safety constraints, and patient outcomes.

Epilepsy is a demanding testbed for this shift. Correct answers often depend on multi-hop evidence: a syndrome may be linked to a gene, the gene to a seizure phenotype, the phenotype to EEG signatures, and the treatment decision to contraindications or guideline evidence. EpiGraph makes these links explicit through an epilepsy knowledge graph and evaluates whether generalist models can use that evidence in realistic reasoning tasks.

This repository provides the paper-aligned code release for:

Component	What it gives you
EpiKG	A lightweight builder for an epilepsy knowledge graph from literature and clinical resources
Graph-RAG	Retrieval over graph neighborhoods with PPR ranking and serialized reasoning paths
EpiBench	Five benchmark tasks spanning QA, EEG reports, precision medicine, treatment recommendation, and research planning
Metrics	Task-specific evaluation utilities aligned with the paper
Project page	A GitHub Pages-ready site with an interactive KG explorer and benchmark overview

---

Key Features

• Large-scale epilepsy evidence graph: EpiKG connects syndromes, phenotypes, genes, treatments, outcomes, and literature-backed evidence into a graph designed for multi-hop clinical reasoning.
• Generalist-model benchmark: EpiBench asks whether broad AI systems can handle epilepsy reasoning in the wild, not just answer short isolated medical questions.
• Graph-RAG out of the box: Retrieval combines personalized PageRank neighborhoods with serialized evidence paths so models can ground answers in graph structure.
• Five clinically grounded tasks: Evaluate clinical QA, EEG impression generation, biomarker precision medicine, treatment recommendation, and deep research planning.
• Private-data-aware release: Task 2 keeps the Harvard EEG data local while preserving the schema, build logic, and evaluation interface.
• Interactive project page: The included GitHub Pages site gives readers a searchable KG demo, task cards, visual overviews, and download links.

Interactive knowledge graph Explore a compact EpiGraph subgraph directly in the browser. Search nodes, inspect evidence paths, and view relation metadata used by Graph-RAG.	Plug-and-play evaluation Run the same task scripts with your own model, retriever, prompts, or local data exports. EpiBench is designed for fast model testing and fair ablation.
Five clinically grounded tasks Evaluate models on epilepsy diagnosis, EEG impression generation, biomarker-driven medication selection, treatment recommendation, and deep research planning.	Private-data-aware release The Harvard EEG task is supported through a local schema adapter, so the evaluation logic is reproducible without redistributing restricted data.

---

Visual Tour

EpiKG organizes epilepsy evidence into connected clinical layers, linking syndromes, phenotypes, genes, treatments, and outcomes through evidence-grounded triplets.

EpiBench turns the graph and clinical inputs into five model-facing tasks, making it easy to compare standard prompting, retrieval, and Graph-RAG settings.

---

At A Glance

Signal	Scale in the paper
Literature corpus	48,166 papers
Knowledge graph entities	24,324 entities
Knowledge graph triplets	32,009 triplets
Benchmark tasks	5 tasks
Core setting	Evidence-intensive epilepsy reasoning

---

Project Page

This repo includes a static GitHub Pages site in docs/. It contains:

Page feature	Included
Responsive hero section	PC, laptop, tablet, and mobile friendly
Interactive KG explorer	Search, presets, clickable nodes, clickable edges, evidence inspector
EpiBench overview	Five task cards with metrics
Quick-start commands	Copy-ready evaluation command
Downloads	README, manifest, T2 schema, demo graph, license

To publish the page on GitHub:

Settings -> Pages -> Deploy from a branch
Branch: main
Folder: /docs

GitHub will then serve the page from the repository's Pages URL.

---

Quick Start

git clone https://github.com/<your-org>/<your-repo>.git
cd <your-repo>
python -m venv .venv
source .venv/bin/activate
pip install -r requirements.txt
export OPENROUTER_API_KEY="your_key_here"

Run a Graph-RAG evaluation on Task 1:

python tasks/t1_clinical_decision_accuracy.py \
  --dataset data/epibench/t1/mcq.json \
  --triplets data/epikg/triplets.json \
  --model openai/gpt-4o \
  --mode graph_rag \
  --out runs/t1_mcq_graph_rag.json

For local models, replace the ChatClient implementation in epigraph/common.py with your local inference wrapper or point it to an OpenAI-compatible local endpoint.

---

Build A Lightweight EpiKG Preview

The full paper graph is built from 48,166 papers plus clinical resources. This release includes a reproducible preview builder for local PMC XML files:

python -m epigraph.build_kg \
  --pmc_dir /path/to/pmc_xml \
  --out_dir data/epikg

Expected outputs:

data/epikg/triplets.json
data/epikg/paper_metadata.json

Triplets follow the paper-aligned schema:

{
  "head": "SCN1A",
  "relation": "caused_by_gene",
  "tail": "Dravet syndrome",
  "head_layer": "gene",
  "tail_layer": "syndrome",
  "paper_count": 12,
  "paper_ids": ["pmc_..."]
}

---

EpiBench Tasks

Task	Name	What it measures	Main metrics
T1	Clinical Decision Accuracy	Epilepsy-specific MCQ and open-ended clinical QA	Top-1 accuracy, BLEU-1, ROUGE-L, Token-F1
T2	Clinical Report Generation	EEG description and patient context to neurologist-style impression	ROUGE-L, Token-F1, report alignment
T3	Biomarker Precision Medicine	Gene variant and phenotype to antiseizure medication selection	Top-1 accuracy, drug safety score
T4	Treatment Recommendation	Guideline-consistent therapy choice under patient-specific constraints	Top-1 accuracy, drug safety, KG evidence coverage
T5	Deep Research Planning	Literature-grounded research question and feasible study-plan generation	ROUGE-L, Token-F1, LLM-as-judge dimensions

T1 Clinical Decision Accuracy

python tasks/t1_clinical_decision_accuracy.py \
  --dataset data/epibench/t1/mcq.json \
  --triplets data/epikg/triplets.json \
  --model openai/gpt-4o \
  --mode graph_rag \
  --out runs/t1_mcq_graph_rag.json

T2 Clinical Report Generation

The Harvard EEG data used by the paper cannot be redistributed. This release provides a local adapter and evaluator. Prepare a private JSONL export with the following fields:

{"patient_history":"...","eeg_description":"...","bandpower":{"delta":0.31},"spike_rate":2.4,"impression":"..."}

Then build and evaluate:

python tasks/t2_clinical_report_generation.py build \
  --raw_jsonl data/private/harvard_eeg/local_export.jsonl \
  --out data/epibench/t2/harvard_preview.json

python tasks/t2_clinical_report_generation.py eval \
  --dataset data/epibench/t2/harvard_preview.json \
  --triplets data/epikg/triplets.json \
  --model medgemma-4b-it \
  --mode graph_rag

T3 Biomarker-Driven Precision Medicine

python tasks/t3_biomarker_precision_medicine.py build \
  --out data/epibench/t3/bpm_mcq.json

python tasks/t3_biomarker_precision_medicine.py eval \
  --dataset data/epibench/t3/bpm_mcq.json \
  --triplets data/epikg/triplets.json \
  --model openai/gpt-4o \
  --mode graph_rag

T4 Treatment Recommendation

python tasks/t4_treatment_recommendation.py build \
  --out data/epibench/t4/medqa_epilepsy.json \
  --max_items 200

python tasks/t4_treatment_recommendation.py eval \
  --dataset data/epibench/t4/medqa_epilepsy.json \
  --triplets data/epikg/triplets.json \
  --model openai/gpt-4o \
  --mode graph_rag

T5 Deep Research Planning

python tasks/t5_deep_research_planning.py build \
  --lay_summaries data/epibench/t5/lay_summaries.json \
  --out data/epibench/t5/research_planning.json

python tasks/t5_deep_research_planning.py eval \
  --dataset data/epibench/t5/research_planning.json \
  --triplets data/epikg/triplets.json \
  --model openai/gpt-4o \
  --mode graph_rag

---

Repository Layout

EpiGraph_code_release/
  configs/default.json
  docs/
    index.html
    styles.css
    app.js
    data/demo_graph.json
  epigraph/
    build_kg.py
    common.py
    metrics.py
    retrieval.py
  tasks/
    t1_clinical_decision_accuracy.py
    t2_clinical_report_generation.py
    t3_biomarker_precision_medicine.py
    t4_treatment_recommendation.py
    t5_deep_research_planning.py
  CODE_MANIFEST.md
  LICENSE
  README.md
  requirements.txt

---

License

This project is released under the Apache License 2.0.

---

Repository Metrics

---

EpiGraph turns epilepsy evidence into graph structure, then tests whether generalist AI systems can reason with it.