A composable, async-first pipeline for candidate processing:
sourcing → hydration → filtering → scoring → selection → side effects
Designed to be modular, testable, and concurrency-friendly.
This project explores a clean architecture for building asynchronous pipelines where each stage has a single responsibility and predictable behavior.
The design intentionally favors clarity over cleverness, and correctness over premature optimization.
Key ideas behind the design:
- Clear ownership and immutability across stages
- Async concurrency only where it provides real value (sources, side effects)
- Deterministic, sequential transforms for ranking logic
- Strong trait boundaries for easy testing and experimentation
🚧 Early scaffolding
Core traits, structure, and execution model are in place.
APIs may evolve as the pipeline matures and more real-world constraints are explored.
Query— request-level contextCandidate— entity being ranked
Source— produces candidates (fan-out, async)Hydrator— enriches candidatesFilter— removes candidatesScorer— mutates candidate scoresSelector— chooses final resultsSideEffect— observes results (read-only)
- Sources run in parallel to reduce latency
- Hydrators, filters, scorers run sequentially for deterministic behavior
- Side effects run in parallel but are strictly read-only
- No shared mutable state inside the pipeline core
Side effects are intentionally prevented from:
- Reordering candidates
- Dropping candidates
- Mutating candidate data
- Rust (edition 2024)
- Tokio
- async-trait
No additional async utilities or futures crates are used.
Each pipeline stage is designed to be tested in isolation.
This enables:
- Clear failure attribution
- Faster feedback loops
- Confidence in composition without relying solely on end-to-end tests
Async tests use #[tokio::test].
cargo run
Notes
This project is primarily an architectural exploration.
It aims to model how real production ranking or matching systems are structured,
while keeping the codebase small, explicit, and readable.