Fix _GatedLLM.completion override after _return_metrics removal (rel-1.29.0)#3795
Conversation
The `_return_metrics` parameter was removed from LLM/TestLLM.completion in 2bedbd8 (removed_in 1.29.0, on this release branch), but tests/agent_server/test_goal_loop.py's _GatedLLM override (added in #3770) still declared it as a positional param and forwarded it to super().completion(), which pyright flagged as an incompatible override and a 5-positional call against the new 4-positional signature. Drop _return_metrics to match.
all-hands-bot
left a comment
There was a problem hiding this comment.
✅ QA Report: PASS
The PR fixes the _GatedLLM.completion override mismatch on rel-1.29.0: the baseline raises the reported positional-argument failure, while the PR delegates past the signature boundary successfully.
Does this PR achieve its stated goal?
Yes. The stated goal is to remove the stale _return_metrics parameter from _GatedLLM.completion on the rel-1.29.0 branch so it matches TestLLM.completion after the release-branch deprecation removal. Runtime introspection shows the PR branch signature now matches the base class shape, and executing the affected completion path no longer raises the baseline TypeError from forwarding one positional argument too many.
| Phase | Result |
|---|---|
| Environment Setup | ✅ make build completed successfully and installed the uv environment |
| CI Status | pre-commit; Validate PR description is failing and QA checks are in progress |
| Functional Verification | ✅ Before/after execution confirms the override mismatch is fixed |
Functional Verification
Test 1: Runtime signature alignment
Step 1 — Reproduce / establish baseline without the fix:
Ran:
git checkout --detach origin/rel-1.29.0
PYTHONPATH="$PWD" OPENHANDS_SUPPRESS_BANNER=1 uv run python /tmp/qa_goal_loop_signature.pyRelevant output:
Base TestLLM.completion: (self, messages: 'list[Message]', tools: 'Sequence[ToolDefinition] | None' = None, add_security_risk_prediction: 'bool' = False, on_token: 'TokenCallbackType | None' = None, **kwargs: 'Any') -> 'LLMResponse'
_GatedLLM.completion: (self, messages, tools=None, _return_metrics=False, add_security_risk_prediction=False, on_token=None, **kwargs)
This shows the baseline mismatch: the base release-branch method has no _return_metrics, while _GatedLLM still exposes it.
Step 2 — Apply the PR's changes:
Checked out fix/goal-loop-test-return-metrics-rel-1.29.0 at ad1251175418a038ea4828ea616ca63d47df1212.
Step 3 — Re-run with the fix in place:
Ran:
PYTHONPATH="$PWD" OPENHANDS_SUPPRESS_BANNER=1 uv run python /tmp/qa_goal_loop_signature.pyRelevant output:
Base TestLLM.completion: (self, messages: 'list[Message]', tools: 'Sequence[ToolDefinition] | None' = None, add_security_risk_prediction: 'bool' = False, on_token: 'TokenCallbackType | None' = None, **kwargs: 'Any') -> 'LLMResponse'
_GatedLLM.completion: (self, messages, tools=None, add_security_risk_prediction=False, on_token=None, **kwargs)
This shows the stale _return_metrics parameter is gone from the override.
Test 2: Affected completion call path
Step 1 — Reproduce / establish baseline without the fix:
Ran:
git checkout --detach origin/rel-1.29.0
PYTHONPATH="$PWD" OPENHANDS_SUPPRESS_BANNER=1 uv run python /tmp/qa_goal_loop_completion.pyOutput:
TypeError
TestLLM.completion() takes from 2 to 5 positional arguments but 6 were given
This confirms the baseline bug at runtime too: _GatedLLM.completion forwards the removed _return_metrics argument into TestLLM.completion, producing the same kind of signature mismatch the PR describes.
Step 2 — Apply the PR's changes:
Checked out fix/goal-loop-test-return-metrics-rel-1.29.0 at ad1251175418a038ea4828ea616ca63d47df1212.
Step 3 — Re-run with the fix in place:
Ran:
PYTHONPATH="$PWD" OPENHANDS_SUPPRESS_BANNER=1 uv run python /tmp/qa_goal_loop_completion.pyOutput:
TestLLMExhaustedError
TestLLM: no more scripted responses (exhausted after 0 calls)
This is the expected next behavior for the minimal harness with no scripted TestLLM responses. The important part is that the positional-argument TypeError is gone, proving the changed override now delegates through the current release-branch signature.
Issues Found
None.
This review was created by an AI agent (OpenHands) on behalf of the user.
Problem
Pre-commit (
pyright) fails onrel-1.29.0(blocking the v1.29.0 release PR #3787) with two errors intests/agent_server/test_goal_loop.py:This is a logical merge conflict between two changes that are each green on their own:
2bedbd8f"Remove acp_env and_return_metricsdeprecations (removed_in 1.29.0)" — deleted the_return_metricsparameter fromLLM.completion/TestLLM.completion(now 4 positional params). This lives on this release branch.test_goal_loop.py, whose_GatedLLM.completionwas written against the old signature (with_return_metrics) and CI'd before the removal landed.The release branch is the first tree where pyright type-checks both together.
Fix
Drop
_return_metricsfrom the_GatedLLM.completionoverride — both the signature and thesuper().completion(...)call — to match the 4-positional signature on this branch. No behavior change.