Remove fixture project agent runtime#71
Conversation
|
Warning Review limit reached
Next review available in: 42 minutes Enable usage-based reviews in Billing to review now. Otherwise, wait until the next included review is available. How can I continue?After more reviews become available, a review can be triggered using the To avoid repeated limits, reduce automatic review volume by pausing incremental auto-reviews earlier, using label-based review opt-in, excluding WIP or generated PR titles, or requesting reviews manually when the PR is ready. If your team needs uninterrupted high-volume reviews, an organization admin can enable usage-based reviews. How do review limits work?CodeRabbit enforces per-developer PR review limits for each organization. Most developers receive the normal plan review availability. For paid Pro and Pro+ PR reviews, CodeRabbit uses adaptive limits for sustained high-volume activity. When a developer's recent PR review activity reaches the 95th percentile or higher among CodeRabbit users, additional reviews become available more gradually as earlier reviews age out of the rolling window. Please refer docs for additional details. Review details⚙️ Run configurationConfiguration used: defaults Review profile: CHILL Plan: Pro Plus Run ID: 📒 Files selected for processing (4)
📝 WalkthroughWalkthroughThe change removes the production ChangesOpenAI Agents SDK-only runtime
Estimated code review effort: 3 (Moderate) | ~25 minutes Possibly related PRs
Suggested reviewers: 🚥 Pre-merge checks | ✅ 5✅ Passed checks (5 passed)
✨ Finishing Touches🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
There was a problem hiding this comment.
🧹 Nitpick comments (1)
backend/tests/test_projects.py (1)
643-647: 📐 Maintainability & Code Quality | 🔵 Trivial | ⚡ Quick winRepeated monkeypatch boilerplate across ~9 tests.
The same
monkeypatch.setattr(project_service_module, "get_project_guide_agent_runtime", lambda: DeterministicTestProjectGuideAgentRuntime())block is duplicated in every test that needs the deterministic runtime. A shared fixture would reduce duplication and centralize future changes to the fake runtime wiring.Note: tests that also need `monkeypatch` for other purposes (e.g. `OPENAI_API_KEY` at line 1591) would keep the `monkeypatch` fixture alongside this new one.♻️ Proposed fixture extraction
+@pytest.fixture +def deterministic_project_agent_runtime(monkeypatch: pytest.MonkeyPatch) -> None: + monkeypatch.setattr( + project_service_module, + "get_project_guide_agent_runtime", + lambda: DeterministicTestProjectGuideAgentRuntime(), + ) + + async def test_create_guide_autostart_runs_celery_pipeline_to_draft_policy( project_client: AsyncClient, - monkeypatch: pytest.MonkeyPatch, + deterministic_project_agent_runtime: None, ) -> None: monkeypatch.setenv("WORKSTREAM_PROJECT_SETUP_PIPELINE_AUTOSTART", "true") monkeypatch.setenv("WORKSTREAM_CELERY_TASK_ALWAYS_EAGER", "true") get_settings.cache_clear() - monkeypatch.setattr( - project_service_module, - "get_project_guide_agent_runtime", - lambda: DeterministicTestProjectGuideAgentRuntime(), - )Also applies to: 692-696, 1587-1596, 1806-1812, 2237-2245, 2263-2271, 2305-2313, 2462-2470, 2515-2523
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@backend/tests/test_projects.py` around lines 643 - 647, The deterministic runtime monkeypatch is duplicated across multiple project service tests, so extract it into a shared fixture to centralize the wiring. Create a reusable fixture around project_service_module.get_project_guide_agent_runtime that returns DeterministicTestProjectGuideAgentRuntime(), then replace the repeated monkeypatch.setattr blocks in the affected tests with that fixture while keeping any other monkeypatch usage (like environment setup) alongside it.
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Nitpick comments:
In `@backend/tests/test_projects.py`:
- Around line 643-647: The deterministic runtime monkeypatch is duplicated
across multiple project service tests, so extract it into a shared fixture to
centralize the wiring. Create a reusable fixture around
project_service_module.get_project_guide_agent_runtime that returns
DeterministicTestProjectGuideAgentRuntime(), then replace the repeated
monkeypatch.setattr blocks in the affected tests with that fixture while keeping
any other monkeypatch usage (like environment setup) alongside it.
ℹ️ Review info
⚙️ Run configuration
Configuration used: defaults
Review profile: CHILL
Plan: Pro Plus
Run ID: f8adcac6-c4e9-479e-9f4a-c967def5ce29
📒 Files selected for processing (14)
.agent-loop/LOOP_STATE.md.agent-loop/initiatives/WS-POL-001-submission-artifact-policy-foundation/CHUNK_MAP.md.agent-loop/initiatives/WS-POL-001-submission-artifact-policy-foundation/STATUS.md.agent-loop/initiatives/WS-POL-001-submission-artifact-policy-foundation/chunks/WS-POL-001-09-openai-agent-sdk-only-project-setup.md.agent-loop/initiatives/WS-POL-001-submission-artifact-policy-foundation/reviews/WS-POL-001-09-internal-review-evidence.md.agent-loop/initiatives/WS-POL-001-submission-artifact-policy-foundation/reviews/WS-POL-001-09-pr-trust-bundle.mdREADME.mdbackend/app/adapters/project_agents/__init__.pybackend/app/adapters/project_agents/local_fixture.pybackend/app/core/config.pybackend/tests/test_projects.pyexamples/terminal_benchmark/LOCAL_VALIDATION_NOTES.mdexamples/terminal_benchmark/README.mdexamples/terminal_benchmark/terminal_benchmark_api_e2e.py
💤 Files with no reviewable changes (2)
- backend/app/adapters/project_agents/local_fixture.py
- backend/app/core/config.py
2 participants
PR Trust Bundle: WS-POL-001-09
Chunk
WS-POL-001-09- OpenAI Agents SDK Only Project Setup RuntimeGoal
Remove the production deterministic project-agent fixture runtime so project
guide sufficiency and submission artifact policy derivation run only through
the OpenAI Agents SDK runtime in current production code.
Human-Approved Intent
The user explicitly rejected the local fixture runtime because it made the
Terminal Benchmark proof look like real agent SDK derivation. The current
product question is whether the OpenAI Agents SDK can derive the project
submission policy from the full guide/source material. Workstream must fail
closed when the model-backed runtime is not configured.
Chunk contract:
.agent-loop/initiatives/WS-POL-001-submission-artifact-policy-foundation/chunks/WS-POL-001-09-openai-agent-sdk-only-project-setup.mdWhat Changed
backend/app/adapters/project_agents/local_fixture.py.project_agent_runtime_adapterfrom active settings.OpenAIAgentSdkProjectGuideRuntime.backend/tests/test_projects.py.WORKSTREAM_PROJECT_AGENT_RUNTIME_ADAPTER=local_fixtureenvironment variableis ignored in
testand cannot restore fixture behavior.model/API-key settings and to avoid fallback/runtime-fixture wording.
contract without rewriting prior chunk history.
Why It Changed
The fixture runtime was useful while plumbing the lifecycle, but it is now
harmful because it can hide whether model-backed setup actually works. A project
guide should trigger Workstream's real project-agent runtime. Tests can still
use explicit fakes, but production code should not expose a deterministic
fixture adapter.
Design Chosen
ProjectGuideAgentRuntimeport/interface.WORKSTREAM_PROJECT_AGENT_OPENAI_AGENT_SDK_MODELis missing.Alternatives Rejected
local_fixtureas dev/test production adapter: rejected because itconfuses real project setup proof.
another supported production path.
fake exists only for tests.
Scope Control
No task lifecycle, checker runtime, post-submit policy, review/revision,
payment, reputation, blockchain, frontend, object storage, or workflow changes
were added.
Historical chunk contracts remain unchanged. The superseding decision is
recorded only in WS-POL-001-09 artifacts and active status/map files.
Product Behavior
Automatic project setup now requires OpenAI Agents SDK configuration. If the
model is missing, setup routes fail closed through the existing sanitized agent
runtime error path. The Week 1 demo keeps project setup autostart disabled so it
does not imply a configured agent worker.
Acceptance Criteria Proof
local_fixturesource remains.WORKSTREAM_PROJECT_AGENT_RUNTIME_ADAPTERis no longer aSettingsfield.OpenAIAgentSdkProjectGuideRuntime.behavior.
DeterministicTestProjectGuideAgentRuntimeonly insidebackend/tests/test_projects.py.fallback.
WORKSTREAM_PROJECT_SETUP_PIPELINE_AUTOSTART=false.Tests/Checks Run
Result summary:
Test Delta
Tests added or strengthened:
test_project_agent_factory_requires_openai_agent_sdk_modeltest_project_agent_factory_ignores_removed_runtime_selectorTests removed or rewritten:
local_fixtureoutside-dev fail-closed test removed becausethe production fixture adapter no longer exists.
CI Integrity
Reviewer Results
Internal review evidence:
.agent-loop/initiatives/WS-POL-001-submission-artifact-policy-foundation/reviews/WS-POL-001-09-internal-review-evidence.mdReviewed code SHA:
4cd69b525665d5c9000734300aa313d9713c575fReviewed at:
2026-07-06T04:59:38ZExternal Review
External review has not run yet for this branch. CodeRabbit/GitHub findings
must be recorded separately if a PR is opened.
Remaining Risks
surface grows.
Human Review Focus
runtime path.
the superseding decision.
Human Merge Ownership
Only the user can approve and merge this PR. Codex must not merge it without
explicit user approval for that specific PR.
Summary by CodeRabbit
New Features
Bug Fixes
Documentation