LiteFS-aware Ecto middleware for automatic write forwarding in distributed SQLite clusters.
EctoLiteFS detects which node is the current LiteFS primary and automatically forwards write operations to it, allowing replicas to handle reads locally while writes are transparently routed to the primary.
Built on EctoMiddleware: EctoLiteFS includes EctoMiddleware as a dependency, so installing
ecto_litefsgives you everything you need!
- Automatic write forwarding - Writes on replicas are transparently forwarded to primary
- Local reads - Replicas handle reads locally for low latency
- Multiple detection methods - Filesystem polling + HTTP event stream + database tracking
- Zero-config in dev/test - Gracefully degrades when LiteFS is not present
- Minimal setup - Just add middleware and a supervisor to your app
- Works with multiple repos - Can track any number of Ecto repos in the same app
- Telemetry integration - Monitor write forwarding performance and failures
Add ecto_litefs to your list of dependencies in mix.exs:
def deps do
[
{:ecto_litefs, "~> 1.0"}
]
endAdd EctoLiteFS.Supervisor to your application, after your Repo:
defmodule MyApp.Application do
def start(_type, _args) do
children = [
MyApp.Repo,
{EctoLiteFS.Supervisor,
repo: MyApp.Repo,
primary_file: "/litefs/.primary",
poll_interval: 30_000,
event_stream_url: "http://localhost:20202/events"
}
]
Supervisor.start_link(children, strategy: :one_for_one)
end
enddefmodule MyApp.Repo do
use Ecto.Repo, otp_app: :my_app
use EctoMiddleware.Repo # Included with ecto_litefs!
@impl EctoMiddleware.Repo
def middleware(_action, _resource) do
[EctoLiteFS.Middleware] # Add anywhere in your middleware stack
end
end# On primary node - executes locally
MyApp.Repo.insert!(%User{name: "Alice"})
# On replica node - automatically forwarded to primary
MyApp.Repo.insert!(%User{name: "Bob"})
# Reads always execute locally (low latency!)
MyApp.Repo.all(User)That's it! Writes are automatically forwarded when running on a replica.
Future Plans: Support for forwarding transactions and bulk operations is planned for future releases.
All options are configured when starting the supervisor:
:repo- Required. The Ecto Repo module to track.:primary_file- Path to LiteFS.primaryfile. Default:"/litefs/.primary":poll_interval- Filesystem poll interval in ms. Default:30_000:event_stream_url- LiteFS HTTP events endpoint. Default:"http://localhost:20202/events":table_name- Database table for primary tracking. Default:"_ecto_litefs_primary":cache_ttl- Cache TTL in ms. Default:5_000:erpc_timeout- Timeout for RPC calls to primary. Default:15_000
For most use cases, you only need to specify :repo:
{EctoLiteFS.Supervisor, repo: MyApp.Repo}All other options use sensible defaults that work with standard LiteFS configurations.
EctoLiteFS uses multiple detection methods to determine primary status:
- Filesystem polling - Checks for the presence of LiteFS's
.primaryfile - Event streaming - Subscribes to LiteFS's HTTP event stream for real-time updates
- Database tracking - Stores primary node information in a replicated table
When a write operation is detected on a replica node, it's automatically forwarded
to the primary node via :erpc.call/4.
By default, both filesystem polling and event streaming are enabled for robust detection, but either of these can be disabled if desired.
┌─────────────────────┐ ┌─────────────────────┐
│ Primary Node │ │ Replica Node │
│ │ │ │
│ ┌──────────────┐ │ │ ┌──────────────┐ │
│ │ Repo.insert │ │ :erpc │ │ Repo.insert │───┼──┐
│ └──────┬───────┘ │ ◄────────┼──│ (forwarded)│ │ │
│ │ │ │ └──────────────┘ │ │
│ ┌────▼────┐ │ │ │ │
│ │ SQLite │◄─────┼──────────┼─► Reads happen │ │
│ │ (write) │ │ replicate│ locally │ │
│ └─────────┘ │ │ │ │
└─────────────────────┘ └─────────────────────┘ │
│
Middleware detects write ─────────────┘
and forwards to primary
EctoLiteFS gracefully handles environments where LiteFS is not present:
- Production (with LiteFS): Forwards writes to primary, reads from replica
- Development/Test (no LiteFS): Executes all operations locally
This means you can add the middleware to your Repo without any conditional logic - it will "just work" in all environments!
EctoLiteFS emits telemetry events for observability:
# Monitor slow write forwards
:telemetry.attach(
"log-slow-forwards",
[:ecto_litefs, :forward, :stop],
fn _event, %{duration: duration}, %{repo: repo, action: action}, _config ->
if duration > 5_000_000 do # 5ms
Logger.warning("Slow forward: #{action} took #{duration}ns")
end
end,
nil
)
# Track forwarding failures
:telemetry.attach(
"track-forward-errors",
[:ecto_litefs, :forward, :exception],
fn _event, _measurements, %{repo: repo, reason: reason}, _config ->
Logger.error("Forward failed: #{inspect(reason)}")
end,
nil
)Available events:
[:ecto_litefs, :forward, :start]- Write forwarding initiated[:ecto_litefs, :forward, :stop]- Forwarding completed successfully[:ecto_litefs, :forward, :exception]- Forwarding failed
mix testThe E2E test suite validates the full LiteFS cluster behavior including write forwarding and automatic failover. It requires Docker with privileged mode (for FUSE filesystem).
cd e2e
./run_tests.shThe E2E tests spin up a Docker Compose cluster with:
- Consul - Leader election coordinator
- Primary node - LiteFS primary with Elixir app
- Replica node - LiteFS replica with Elixir app
Test scenarios:
- Cluster status verification
- Write to primary, replicate to replica
- Write forwarding from replica to primary
- Primary failover - replica promoted, data preserved
- litefs - The original LiteFS library for Elixir by @sheertj.
Uses a repo wrapper approach where you rename your repo to
MyApp.Repo.Localand create a newMyApp.Repothat proxies writes, and relies on filesystem polling only. EctoLiteFS differs by using middleware instead (so your existing repo stays unchanged), and adds HTTP event streaming for faster primary detection.
MIT License - see LICENSE for details.