ChartaPilot

Synthetic urgent care AI chart review deployment workbench

Independent portfolio prototype. Not affiliated with Charta Health. Uses synthetic data only.

ChartaPilot is a synthetic workflow prototype for urgent care AI chart review. It simulates how a Forward Deployed AI Engineer might configure and launch chart review for a fictional urgent care customer. The customer is MetroFast Urgent Care. The workflow is pre-bill chart review: configure rules, review synthetic notes, cite evidence, route findings, audit actions, run evals, and generate a pilot report. The project uses synthetic data only. It does not process real patient data. It is not a real medical coding product. It is not CPT, ICD, HCC, payer, or billing authoritative. The goal is deployment workflow signal, not medical correctness. It is not affiliated with Charta Health.

Why I Built This

I built ChartaPilot to show how I think about AI chart review deployment. The interesting part is not generating one plausible answer from a chart. The harder product problem is making chart review configurable, evidence-backed, reviewable by humans, auditable, measurable, and safe to roll out.

The project focuses on the pieces a deployment-minded AI engineer would need to reason through: customer configuration, payer/site rules, evidence spans, human review routing, reviewer actions, audit logs, eval metrics, and rollout risk.

What This Is Not

• Not a real medical coding engine.
• Not CPT, ICD-10, HCC, payer, diagnosis, billing, or reimbursement authoritative.
• Not trained on real patient data.
• Not production healthcare software.
• Not an EHR integration.
• Not a HIPAA compliance claim.
• Not a replacement for certified coders, clinicians, compliance teams, or customer-specific payer review.

What It Does

• Shows a customer deployment setup for MetroFast Urgent Care.
• Reviews 37 synthetic urgent care encounters.
• Uses a deterministic review engine before any LLM layer.
• Flags synthetic review findings such as possible undercoding, missed procedures, missing medical necessity, work-injury context gaps, care gaps, unsupported diagnoses, weak evidence, and ready-to-bill charts.
• Cites exact chart evidence spans when support exists.
• Routes weak or risky findings to review queues.
• Supports a provider addendum flow.
• Writes audit events for system and reviewer actions.
• Shows provider, site, and payer analytics.
• Generates a pilot readiness report.
• Runs an eval harness against a controlled synthetic truth set.

Demo Path

1. /deployment
   Shows MetroFast Urgent Care configuration, enabled modules, routing rules, evidence threshold, and autocorrection disabled.

2. Click Run synthetic chart review
   Runs deterministic review across the synthetic encounter set.

3. /dashboard
   Shows charts reviewed, clean vs flagged charts, queue load, citation validity, and unsafe autocorrect count.

4. /encounters/MF-UC-0007
   Shows a possible undercoding case with exact highlighted evidence and coder-review routing.

5. /encounters/MF-UC-0014
   Shows an imaging medical-necessity documentation gap routed to provider addendum.

6. Click Request addendum
   Records a reviewer action and writes an audit event.

7. /encounters/MF-UC-0037
   Shows an ambiguous weak-evidence case routed to human review instead of producing a confident revenue or compliance claim.

8. /queue
   Shows coder review, provider addendum, compliance review, clinical quality, ready-to-bill, and blocked queues.

9. /analytics
   Shows provider, site, and payer patterns from the synthetic review run.

10. /report
    Shows the pilot readiness surface.

11. Click Run Eval
    Runs the eval harness against the synthetic truth set.

12. Click Generate Report
    Generates the pilot readiness report with metrics, limitations, and rollout recommendation.

Architecture

• Frontend: Next.js, TypeScript, Tailwind.
• Backend: FastAPI with typed route handlers.
• Review engine: deterministic phrase and evidence rules in Python.
• Synthetic seed data: MetroFast customer, sites, providers, payers, deployment config, encounters, proposed billing summaries, and truth labels.
• Eval/report generation: backend services compute metrics and generate Markdown reports from current run state.
• Audit logging: system events and reviewer actions are appended to an in-memory audit log.

The implementation is intentionally simple and readable. It is shaped like a production workflow, but it avoids auth, EHR plumbing, and real medical coding logic.

Review Engine

The review engine is deterministic-first. That is deliberate.

For this prototype, the trust boundary is not model fluency. It is evidence. Findings must either cite exact text from the synthetic note or explicitly mark what evidence is missing. Evidence spans store offsets and quoted text, and validation checks that the quote matches the source note.

Weak evidence routes to human review. Billing-impacting findings route to humans. Autocorrection is disabled by design.

An LLM could be useful later for extraction or explanation drafting, but deterministic validation should still decide whether a finding can be trusted.

Eval Results

Current validation:

• Backend tests: 9 passed
• Eval/report smoke: passed
• Finding recall: 1.0
• Routing accuracy: 1.0
• Citation validity: 1.0
• Audit coverage: 1.0
• Unsupported trusted finding rate: 0.0
• Unsafe autocorrect count: 0
• Readiness: Ready for workflow demo
• Frontend typecheck: passed
• Frontend build: passed

These metrics come from a controlled synthetic truth set. They are useful for validating the workflow loop and safety checks, but they are not a claim of real-world medical coding accuracy.

What Is Real Vs Simplified

Real in this prototype:

• Running frontend and backend app.
• Synthetic urgent care dataset.
• Deterministic findings.
• Evidence spans and evidence validation.
• Reviewer actions.
• Audit events.
• Eval/report smoke path.
• Typecheck, build, and backend tests passing.

Simplified in this prototype:

• In-memory/local persistence instead of durable Postgres-backed storage.
• No auth or RBAC.
• No real payer, coding, CDI, or clinical validation.
• No EHR integration.
• No LLM extraction mode.
• No real patient data.

How To Run Locally

Backend:

cd apps/api
python3 -m uvicorn app.main:app --reload --port 8000

API:

http://127.0.0.1:8000

Frontend:

cd apps/web
npm install
npm run dev

UI:

http://localhost:3000/deployment

If port 3000 is already in use, use the alternate port printed by Next.

Validation commands:

cd apps/api
python3 -m pytest

cd apps/api
python3 -c "from app.db.store import store; from app.domain.eval_runner import run_eval; from app.domain.report_generator import generate_pilot_report; e=run_eval(store); r=generate_pilot_report(store, e.review_run_id, e.id); print(e.metrics); print(r.metrics_snapshot['readiness'])"

cd apps/web
npm run typecheck
npm run build

Validation

The current validated state is:

• Backend tests: 9 passed
• Eval/report smoke: passed
• Frontend typecheck: passed
• Frontend build: passed

If the Next.js build fails in a restricted sandbox with a Turbopack port-binding error, rerun the same build in a normal local shell. The previous validated build passed after allowing Turbopack's local build helper to bind.

What I Would Improve Next

• Durable Postgres persistence.
• Auth/RBAC and reviewer identity.
• Real customer-specific config versioning.
• Reviewer disagreement tracking and calibration by finding type.
• LLM extraction mode with deterministic fallbacks and citation validation.
• EHR writeback simulation.
• Better export/download polish for reports and audit packets.
• Screenshots or a short demo video.

Safety And Scope

ChartaPilot is a workflow and deployment prototype. It does not provide medical advice, does not diagnose patients, does not submit claims, does not replace certified coders, and does not support real patient uploads. Billing-impacting findings require human review, and autocorrection is disabled by design.