Skip to content

CHANGELOG.md

Latest commit

 

History

History
94 lines (63 loc) · 4.47 KB

CHANGELOG.md

File metadata and controls

94 lines (63 loc) · 4.47 KB
Raw

Changelog

All notable changes to RustQueue are documented here.

Format loosely follows Keep a Changelog.

[Unreleased]

In progress

  • Telegram job handler (second executor type)
  • Scheduled jobs via API (run_at field)
  • Typed JobStatus enum (separate branch: feature/job-status-enum)
  • Cursor-based pagination on GET /jobs
  • Graceful shutdown on Ctrl+C
  • Cloudflare Email Service exploration as alternative executor

[0.3.0] - 2026-06-26

Added

  • LISTEN/NOTIFY trigger on job insert -- worker wakes instantly instead of polling every 2 seconds
  • Second trigger on UPDATE for retry-back-to-pending transitions -- retried jobs also wake the worker promptly
  • Fallback 2-second timeout on the worker loop as a safety net for missed notifications
  • processing_started_at column -- dedicated timestamp for stale-job detection, replacing the noisy updated_at signal
  • Stale-job reaper now clears processing_started_at on reclaim

Changed

  • Worker loop rebuilt around PgListener instead of tokio::time::sleep
  • SMTP transport built once at startup via HandlerRegistry and shared via Arc -- no longer rebuilt per job execution
  • run_worker now takes Arc<HandlerRegistry> instead of Config directly
  • HandlerRegistry introduced as a central map of job type strings to pre-built handlers

Fixed

  • Race condition: stale-job reaper could incorrectly reclaim jobs that were legitimately mid-retry-backoff (not actually stuck). Fixed by introducing processing_started_at -- the reaper now only looks at this dedicated column, not the general-purpose updated_at.

[0.2.0] - 2026-06-22

Added

  • Concurrent job processing via tokio::spawn per job, capped at 10 simultaneous jobs via Semaphore
  • Exponential backoff retry logic: failed jobs retry at 1s, 5s, 25s... up to max_attempts (default 3)
  • Stale-job reaper: background task reclaims jobs stuck in processing for more than 5 minutes
  • attempts column now correctly incremented on every execution attempt (success or failure)
  • processing_started_at column added to jobs table

Changed

  • mark_done and mark_failed now also update attempts and clear processing_started_at
  • Worker loop drains all pending jobs in a tight inner loop before sleeping

Fixed

  • attempts was previously only incremented on failure, not on success -- a successful first attempt would show attempts: 0 incorrectly

[0.1.0] - 2026-06-21

Added

  • Initial working implementation
  • POST /jobs -- submit a job
  • GET /jobs -- list jobs with optional ?limit=N and ?status= filters (default limit: 20, max: 100)
  • GET /jobs/:id -- get a specific job
  • GET /events -- SSE stream of live job status updates
  • PostgreSQL-backed job persistence via sqlx
  • SELECT FOR UPDATE SKIP LOCKED job claiming -- race-condition-safe, multiple workers can run without double-processing
  • Email job handler via AWS SES (lettre crate, STARTTLS on port 587)
  • broadcast::channel powering SSE fanout to multiple connected clients
  • Arc<AppState> shared between Axum handlers and worker loop
  • Basic retry on failure (later evolved into full exponential backoff in 0.2.0)
  • Migrations tracked via sqlx-cli

Architecture decisions of note

Why SELECT FOR UPDATE SKIP LOCKED instead of application-level locking? Database-level locking is simpler, more reliable, and works correctly even with multiple worker processes or instances. Application-level locks (Redis, in-memory mutexes) add a dependency and a failure mode; Postgres already has the data, so it should own the lock too.

Why a dedicated processing_started_at column instead of using updated_at? updated_at gets touched by retry scheduling, status transitions, and anything else that modifies a job row. Using it to detect "stuck in processing" led to a real race condition where legitimately-retrying jobs (waiting on backoff delay) were incorrectly flagged as stale and reclaimed. processing_started_at is set ONLY when a job enters processing and cleared on every exit -- its meaning is unambiguous.

Why HandlerRegistry instead of building handlers in get_handler per job? Building a new SMTP transport per job meant a full TLS handshake and authentication round-trip on every single email. Pre-building handlers at startup and sharing them via Arc eliminates this overhead entirely. The registry pattern also makes adding new job types a one-line registration, not a scattered conditional.