The reliability layer for LLM traffic. One endpoint speaks every major SDK; fault-aware circuit breaking and in-flight failover keep your app serving when your providers aren't.
📖 Docs: getbusbar.com
⚡ Install: curl -fsSL https://getbusbar.com/install.sh | sh
🤖 Agent-readable: getbusbar.com/llms.txt
Busbar sits between your application and your LLM providers. Point any SDK — OpenAI, Anthropic, Gemini, Bedrock, Cohere — at one URL, and it routes, translates, and keeps serving through provider failures. It's a different class of tool than a proxy with a long model list.
You define a model name and its backends. Busbar accepts any input protocol — OpenAI, Anthropic, Gemini, Bedrock, Cohere, Responses — and routes and translates accordingly. One model name, reachable by every client; you choose what runs behind it.
- Speaks every protocol losslessly, both ways — not flattened to OpenAI shape, so Anthropic thinking blocks, Gemini safety settings, and Bedrock tool use survive the hop. Use whatever SDK your code already speaks, reach every model, swap providers with a config edit.
- Fails over inside the request — before your client sees a byte, even mid-stream, across protocol families. Not a 500 your user feels, not a 3am page.
- A circuit breaker on every provider connection — classifies each error (provider outage, your bad request, context-length, hard auth/billing failure) and treats each differently instead of retrying into a wall.
- A programmable request path — routing is the first hook: your policy steers each request by cost, latency, live concurrency, budget, or rate headroom — native, or your own logic via webhook (any language) or a sandboxed Rhai script. A broken or slow hook never blocks a request, and the same fail-safe machinery is built to carry PII-steering, audit, and guardrails.
- Encrypted, zero-trust transport, built in — Busbar terminates TLS itself (cert + key in config, no reverse proxy). Turn on mTLS and only clients holding a certificate signed by your CA can connect at all — rejected at the handshake, before any HTTP or token check. Zero-trust without a service mesh.
A single static Rust binary — no Python sidecar, no interpreter, no GC in the request path. Linux, macOS, Windows (Intel and ARM). Your keys, your network, your data path.
Status: 1.0.0-rc.5 — feature-complete, API-stable, hardened across a multi-round security and correctness audit. AGPL-3.0.
Your code already speaks OpenAI (or Anthropic, or Gemini). Swap the base URL:
- client = OpenAI(api_key=OPENAI_KEY)
+ client = OpenAI(api_key=BUSBAR_TOKEN, base_url="http://busbar:8080")
# `model` now names a single model OR a pool you define in config
# (e.g. "fast" = 80% Claude / 20% GPT-4o, Gemini on failover)
client.chat.completions.create(model="fast", messages=[...])That request left as OpenAI, may have been served by Anthropic, and came back as OpenAI — translated losslessly both ways. If Anthropic returned a 429 mid-flight, Busbar rerouted to the next pool member before your client saw a single byte. The model name is a config value, not a code dependency.
- Six wire protocols, lossless both ways — any client protocol reaches any pool → Protocols
- Fault-attributed circuit breaking + streaming-safe in-flight failover → Reliability
- Weighted pools — smooth weighted round-robin, session affinity, per-lane concurrency → Reliability
- Hooks — a programmable request path — routing is hook #1:
weighted/cheapest/fastest/least-busy/usagenatively, plus operator-owned webhook and Rhai script policies. A policy sees per-member cost, latency, live concurrency, budget, and rate headroom — your logic, in any language, fail-safe (a timeout or error falls back, never blocks the request) → Routing - Native TLS + optional mTLS — Busbar terminates TLS itself (cert + key in config, no reverse proxy needed). Turn on mutual TLS to require a client certificate signed by your CA; clients without one are rejected at the handshake, before any HTTP or bearer-token check. TLS = encrypted and server-verified out of the box; mTLS = only your cert-holding clients can connect at all — zero-trust without a service mesh → Security
- Governance — virtual keys, budgets, RPM/TPM limits, spend tracking → Governance
- Vetted provider catalog — plus any provider on the six protocols in a few lines of YAML → Providers
- Security-hardened — SSRF guards, constant-time auth, SHA-256 key storage, secrets never logged → SECURITY.md
- Observability — Prometheus
/metrics(per-key spend / budget / tokens, per-lane breaker state), OTLP traces, per-request audit webhook → Configuration
Same arena as LiteLLM / OpenRouter — the difference is that Busbar is built reliability-first. → Why Busbar
curl -fsSL https://getbusbar.com/install.sh | sh # busbar + providers.yaml into ./A minimal config.yaml (keys come from env vars, named here — never written into config):
providers:
anthropic: { api_key_env: ANTHROPIC_KEY } # the NAME of the env var, not the key
models:
claude: { provider: anthropic, max_concurrent: 10 }
pools:
fast: { members: [ { target: claude, weight: 1 } ] }export ANTHROPIC_KEY=sk-ant-...
BUSBAR_PROVIDERS=./providers.yaml BUSBAR_CONFIG=./config.yaml ./busbar
curl -s localhost:8080/v1/chat/completions -H 'content-type: application/json' \
-d '{"model":"fast","messages":[{"role":"user","content":"Hello!"}]}'Full walkthrough → Getting Started
Full documentation at getbusbar.com (agent-readable: llms.txt). Contributor docs — architecture, internals, ADRs — in docs/.
Single Rust binary, MSRV 1.87. Contributions welcome (CONTRIBUTING.md). Licensed AGPL-3.0-or-later (LICENSE) — because Busbar runs as a network service, the AGPL §13 network-use clause applies.