feat(queue): POST /queues/:name/purge to drain stuck PENDING jobs (refs #13)#15
Merged
Merged
Conversation
Operators currently have no way to clear bad messages from a queue short of dropping the entire queue. After updating a flow's code to fix a runtime bug, messages enqueued under the old broken code keep retrying on a 5-minute visibility cycle until max_attempts hits the DLQ — that's ~15 minutes of latency before fresh enqueues even get picked up. Purge deletes only PENDING rows (status='PENDING'). PROCESSING rows are intentionally left alone so we don't yank a row out from under an in-flight worker; those age out naturally via visibility timeout if their handler dies. PG backend only — Redis/Kafka/RabbitMQ users have their broker UIs for this. Returns the deleted row count so an operator can confirm. Refs #13. This is a mitigation, not a root-cause fix: see comment on the issue for analysis of the reporter's "stale cached code" hypothesis (loadFlowConfig already SELECTs fresh on every delivery; the more likely root cause is messages stuck in retry loop).
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Adds an operator endpoint to clear stuck PENDING jobs from a queue without dropping the queue itself.
PG backend only — Redis/Kafka/RabbitMQ users have native purge in their broker UIs.
Why
After fixing a flow's code, messages enqueued under the old broken code keep retrying on the visibility timeout cycle (5 min × max_attempts = 15 min) until they hit the DLQ. New enqueues sit behind them. Today the only mitigation is "drop the queue and recreate it" — destructive and loses queue config (concurrency, backoff, etc).
Scope
status='PENDING'rows onlystatus='PROCESSING'(in-flight) — those age out via visibility timeout if the handler diesRefs
Refs #13 — this is a mitigation, not a root-cause fix. Posted analysis as a comment on that issue: the reporter's "stale cached flow code" hypothesis doesn't match the code (
loadFlowConfigdoes a fresh SELECT on every delivery); the most likely root cause is messages stuck in the retry loop. Purge is the operator escape hatch.Test plan
go build ./...)go vetclean