Add multi-worker concurrency (page-level and session-level parallelism)#22
Merged
Conversation
…lism Introduce two layers of concurrency, available identically across CLI, web, and the Python API: - Page-level: TestConfig.workers (CLI --workers, web 'workers' body field) tests multiple pages of one run in parallel via a thread-safe Frontier, each worker owning its own browser/context. Defaults to 1 (unchanged sequential behaviour); capped at 16. Auth is performed once and replicated to every worker via Playwright storage_state. - Session-level: BatchRunner runs multiple independent sessions through a bounded thread pool. CLI exposes --batch-file/--pool-size; the web server now uses it instead of unbounded thread-per-job (QA_AGENT_JOB_POOL_SIZE). Also stop mutating the caller's TestConfig (deep-copy in QAAgent), route worker-thread stdout via a worker_thread_init hook so web SSE keeps working, and re-export the full public API from qa_agent. Adds unit tests for the Frontier/PageIndexer primitives, BatchRunner pool, parallel agent runs, plus integration smoke tests for the real sync-Playwright-in-threads path.
…-a4ry0 # Conflicts: # qa_agent/cli.py
CI runs mypy 2.1.0 (stricter than local 1.x): annotate _launch_browser's playwright param so it returns Browser not Any, cast user-supplied cookies to satisfy add_cookies' SetCookieParam typing, and str() the recording video path.
- Add "Page Workers" number input (1–16) to the Browser Settings section
of the run form; collectFormData and applyConfig now include workers so
it flows through the existing POST /api/run body → _build_config path.
- Add "Server Settings" collapsible card below the run form with a Pool
Size input (1–8); index.js fetches the current value from the new
GET /api/server-config endpoint on page load and the Apply button
PATCHes the new value.
- New GET /api/server-config endpoint returns {pool_size, pool_size_max,
workers_max}; new PATCH /api/server-config recreates the BatchRunner
singleton with the requested pool size (clamped 1–8).
- 16 new tests in TestApiServerConfig covering GET fields, PATCH
update/clamp/validation, and _build_config workers handling.
https://claude.ai/code/session_0159uLk7nyt7cbv4PX1bTe5o
Gives Claude Code instances a quick orientation: editable-install requirement for packaging tests, test/lint/build commands, and a summary of the request-flow architecture and extension points.
Add HostRateLimiter, a thread-safe min-interval-per-host throttle for page.goto() navigations, enabled by default at 3 req/s (config.rate_limit, --rate-limit, 0 disables). One shared limiter covers all workers within a QAAgent run; BatchRunner can hold a single shared limiter passed to every QAAgent it constructs so concurrent batch jobs targeting the same host share a budget. The web server applies the same default (QA_AGENT_RATE_LIMIT). Addresses "too many connections" errors when --workers/--pool-size fan out many concurrent browsers against a single dev/staging target.
Log/report instead of silently swallowing worker_thread_init exceptions in both the page-worker and batch-pool paths, and document the single-interpreter assumption behind the web server's global pool/rate limiter. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
_QueueWriter emitted a generic 'log' SSE event for every line AND a 'progress'/'finding' event for Testing:/[SEVERITY] lines, causing run.js to render those lines twice in the live log. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Adds two layers of concurrency, available identically across CLI, web, and the Python API. The codebase previously tested pages strictly sequentially within a run, and the web server spawned one unbounded thread per job.
Layer 1 — page-level parallelism (
qa_agent/concurrency.py,agent.py)Frontier(shared BFS queue with correctmax_pages/max_depthbudgeting under concurrency + deadlock-free termination) andPageIndexer(worker-safe screenshot indices).QAAgent._run_concurrent(): N worker threads, each owning its own Playwright stack (required — the sync API is thread-bound).workers == 1keeps the original sequential path unchanged.storage_state.TestConfig.workers(default1, capped at 16) → CLI--workers, webworkersfield inPOST /api/run.Layer 2 — session pool (
qa_agent/batch.py)BatchRunner(boundedThreadPoolExecutor) +BatchJob. The web server now uses it instead of unbounded thread-per-job (QA_AGENT_JOB_POOL_SIZE, default 4). CLI exposes--batch-file/--pool-size; library users getfrom qa_agent import BatchRunner.Cross-cutting
QAAgentdeep-copies the config so concurrent agents don't clobber each other's output dirs (previously__init__mutated the caller's config).worker_thread_inithook routes worker-thread stdout to the web SSE job queue (thread-local stdout would otherwise miss sub-threads);_QueueWritermade thread-safe.qa_agent(matches the README's documented imports).Test plan
tests/test_concurrency.py(Frontier budget/termination/indexer under threads),tests/test_batch.py(bounded concurrency, ordering, per-job exception isolation),TestQAAgentParallelintests/test_agent.py(aggregation, config isolation, stop-event, exploremax_pages).TestSmokeParallelintegration tests exercising the real sync-Playwright-in-threads path.Notes
qa_agent/testers/custom.py:190(untouched code, flagged only by newer mypy 2.x vs. the project's pinnedmypy>=1.0) — not introduced by this change.https://claude.ai/code/session_0159uLk7nyt7cbv4PX1bTe5o
Generated by Claude Code