Resolve PR 2910: optional Vertex AI bundle#3776
Open
neubig wants to merge 7 commits into
Open
Conversation
Adds google-cloud-aiplatform as a required dep so LiteLLM's
vertex_ai_partner_models handler can route to Vertex AI Model Garden
MaaS endpoints (MiniMax, Qwen, Kimi, etc.). Without it, requests like
vertex_ai/minimaxai/minimax-m2-maas, vertex_ai/qwen/qwen3-coder-*-maas,
vertex_ai/moonshotai/kimi-k2-thinking-maas fail with:
litellm.BadRequestError: Vertex_aiException BadRequestError -
vertexai import failed please run
`pip install -U "google-cloud-aiplatform>=1.38"`.
Got error: No module named 'vertexai'
Changes:
- openhands-sdk/pyproject.toml: add "google-cloud-aiplatform>=1.38"
- uv.lock: regenerated (+7 transitive packages)
- agent-server.spec: collect vertexai/google.cloud.aiplatform/google.api_core
submodules as hidden imports (the vertexai import is deferred inside
litellm function bodies, so PyInstaller static analysis misses it),
pin google.rpc.status_pb2 (dynamic gRPC proto stub), and copy package
metadata.
Supersedes #2373 (closed without merging) and lands the spec-side
changes needed for the bundled binary to actually work at runtime.
Addresses review feedback on #2910. Mirrors the existing `boto3` extra pattern so the base `openhands-sdk` install stays lean (the Vertex SDK adds ~80 MiB to site-packages and ~62 MiB to the PyInstaller binary). - pyproject.toml: move `google-cloud-aiplatform` from base deps into `[project.optional-dependencies]` as `vertex`. - llm.py / vertex_preflight.py: when the inferred provider is `vertex_ai` and `vertexai` is not importable, raise a friendly `LLMBadRequestError` with an install hint (`pip install "openhands-sdk[vertex]"`) instead of letting LiteLLM crash with a low-level `ModuleNotFoundError`. - agent-server.spec: skip the Vertex `collect_all` block (and the `google.rpc.status_pb2` pin) when `vertexai` is not installed, so local PyInstaller builds without the extra still succeed. - docker/Dockerfile: add `--extra vertex` to both `uv sync` calls so the published agent-server binary continues to bundle the Vertex SDK by default. - tests: cover the preflight raise/no-op paths. - uv.lock: regenerated.
Per follow-up review on #2910: don't bundle the Vertex SDK in the default agent-server image — it's a ~62 MiB / ~72% size hit for provider-specific support that most users won't need. - Dockerfile: drop `--extra vertex` from both `uv sync` invocations, so the published image stays as lean as it was on `main`. - agent-server.spec: rewrite the header comment to make the new policy explicit (off by default; opt in with `uv sync ... --extra vertex` before running pyinstaller). The `if _VERTEX_AVAILABLE:` guard already added in the previous commit is what makes the default build succeed without the extra installed; keep it as the supported opt-in path for downstream builds (e.g. fork-specific images) that do want Vertex bundled.
Closes the gap left by the previous commit: with the default Docker
build now lean, there was no way to opt back into a Vertex-bundled
binary without forking the Dockerfile. Adds `ENABLE_VERTEX` (default
0) so downstream builds can do:
docker build --build-arg ENABLE_VERTEX=1 ...
ARG declared globally and re-imported into both the `builder` and
`binary-builder` stages, with a small inline shell conditional that
appends `--extra vertex` to each `uv sync` call. The PyInstaller
spec already auto-detects an installed `vertexai` and bundles
accordingly, so no further wiring is needed.
Spec header comment updated to point at this build arg as the
canonical opt-in path.
# Conflicts: # openhands-agent-server/openhands/agent_server/agent-server.spec # uv.lock
# Conflicts: # openhands-sdk/openhands/sdk/llm/llm.py
4 tasks
Contributor
Python API breakage checks — ✅ PASSEDResult: ✅ PASSED |
Contributor
REST API breakage checks (OpenAPI) — ✅ PASSEDResult: ✅ PASSED |
Contributor
Coverage Report •
|
||||||||||||||||||||||||||||||
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
HUMAN:
Requested follow-up for #2910: fix merge conflicts, reflect worthwhile review comments, and open a replacement PR if the original fork branch cannot be updated.
AGENT:
Why
Replacement for #2910 because I could not push to the original fork branch (
trustle:adam/install-aiplatform-for-vertex-ai; GitHub returned 403). This carries forward the Vertex AI optional-extra work from #2910, merged with currentmain.Summary
mainin the shared LLM transport path.google-cloud-aiplatformas theopenhands-sdk[vertex]optional extra and preserved the lean default agent-server build with Vertex as an opt-in bundle.extend(), and regeneratinguv.lockfrom currentmainso the unrelatedpypdf/lxmldowngrade concern is gone.Issue Number
Related PR: #2910
Fixes #3785
How to Test
Commands run locally:
Result: all commands passed. The pytest run collected 4 tests and all 4 passed.
Video/Screenshots
Not applicable; this is dependency/transport/build configuration work with no UI surface.
Type
Notes
The GitHub connector could not create the PR (
Resource not accessible by integration), so the PR was opened with the authenticatedghCLI. I also commented on #2910 with a link to this replacement.Agent Server images for this PR
• GHCR package: https://github.com/OpenHands/agent-sdk/pkgs/container/agent-server
Variants & Base Images
eclipse-temurin:17-jdknikolaik/python-nodejs:python3.13-nodejs22-slimgolang:1.21-bookwormPull (multi-arch manifest)
# Each variant is a multi-arch manifest supporting both amd64 and arm64 docker pull ghcr.io/openhands/agent-server:c78b23b-pythonRun
All tags pushed for this build
About Multi-Architecture Support
c78b23b-python) is a multi-arch manifest supporting both amd64 and arm64c78b23b-python-amd64) are also available if needed