feat: max_in_flight — bounded stream handoff#33
Merged
Conversation
- Add max_in_flight: int = 1 to Step and DagNode - Validate max_in_flight >= 1 and integer at build time - Serialize max_in_flight in DAG JSON (always present) - Sync: BoundedIterator deque-backed wrapper for max_in_flight > 1 (max_in_flight=1 preserves exact current behavior) - Async: use max(max_in_flight, 100) for queue sizing - Update all 18 corpus snapshots with max_in_flight: 1
35b37ac to
944c416
Compare
- Add test_dag_builder_max_in_flight.py (8 build-time validation tests) - Add test_runner_max_in_flight.py for sync and async (4 tests each) - Fix BoundedIterator: propagate exceptions immediately, don't buffer - Fix test parity by using identical test function names in sync and async
944c416 to
27c0fdd
Compare
- Add 10 unit tests for BoundedIterator edge cases - Fix exception handling: buffer items before raising, only raise pending exception when buffer is empty - maxsize validation, empty source, partial iteration covered
…zing - Sync fan-out: apply BoundedIterator to source BEFORE itertools.tee so producer advancement is bounded at the source level - Async queue sizing: use node.max_in_flight directly when > 1, keep 100 for default=1 (backward compatible) - Add tests: sync fan-out bounded, async bounded ahead verification, parity between sync and async test names
f1ff2d2 to
5d8059d
Compare
Codex changes: - AsyncQueueBranch: implement __aiter__/__anext__ for direct async iteration - Async executor: _attach_argument_cleanup, _close_stream_arguments - Async executor: queue sizing uses max(2, max_in_flight+1) for EOF - Async executor: _resolve_queue handles AsyncQueueBranch type - Sync executor: observer threads via SyncFanout branches - Sync executor: _observer_threads with cleanup on pipeline finish - Sync executor: _notify_observers in publish stream paths - 12 new tests: ahead_distance_bounded, producer_blocks, terminal_lazy_drains, flattening_stream_internal_items (sync+async) - 3 new observer tests: does_not_force_eager, does_not_consume_slots, bound_is_unchanged (sync+async)
…pool corpus output type
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Adds
max_in_flight— controls how many items a producing stream can emit ahead of the downstream consumer. Default 1 preserves exact current lockstep behavior.API
Implementation
DAG model:
Step.max_in_flightandDagNode.max_in_flightdefault to 1max_in_flightSync runner:
BoundedIterator(deque-backed) wraps progressive stream outputs whenmax_in_flight > 1itertools.tee— bounds producer at sourceAsync runner:
max(100, node.max_in_flight)— backward compat at defaultmax_in_flightTests: 20 new (10 build-time + 5 sync + 5 async).
Truerejected).DagNode.max_in_flightrather than directStepvalues.max_in_flight=1).Known v1 limitations:
putacross multiple consumer queues.max_in_flight > 100honored exactly.itertools.teeinternal buffer not bounded per-branch. Source bounded, but slow branches can accumulate tee backlog.Corpus: All snapshots (including the new
max_in_flight_threadpool.py) updated and fully validated.445 tests passing.