Skip to content

settings: trust public OpenHands provider models#1280

Merged
enyst merged 1 commit into
mainfrom
openhands/public-openhands-provider
Jun 13, 2026
Merged

settings: trust public OpenHands provider models#1280
enyst merged 1 commit into
mainfrom
openhands/public-openhands-provider

Conversation

@enyst

@enyst enyst commented Jun 9, 2026

Copy link
Copy Markdown
Member

HUMAN:
This PR proposes to continue the simplifying refactoring of openhands/ provider started in agent-sdk: the "translated" form of it will no longer be written on disk, nor present in the LLM object, so the client apps like agent-canvas can just use openhands/ like they use any other LLM providers, and check for openhands/ prefix if they want to offer special features. This removes cruft from the codebase and makes it more robust and maintainable.

  • A human has tested these changes.

AGENT:

Why

The SDK is moving to keep openhands/* as the public/stored LLM model namespace and translate to litellm_proxy/* only at LiteLLM transport time. Canvas no longer needs frontend reverse-mapping or proxy-base-url stamping for the normal OpenHands provider path.

Summary

  • Remove frontend helpers that reverse-map litellm_proxy/* + OpenHands proxy URL back to openhands/*.
  • Initialize the model selector directly from the model returned by the SDK/API.
  • Stop saving the OpenHands LiteLLM proxy base_url from Basic-mode LLM/profile forms.
  • Update tests to cover the public openhands/* end state.

Issue Number

Fixes OpenHands/software-agent-sdk#3514
Related to and works with OpenHands/software-agent-sdk#3548

How to Test

  • npm ci
  • npm test -- --run __tests__/components/modals/settings/model-selector-openhands.test.tsx __tests__/components/settings/llm-profiles/llm-settings-local-view.test.tsx
  • npm run typecheck

Video/Screenshots

Not applicable; this is settings payload normalization behavior with unit coverage.

Type

  • Bug fix
  • Feature
  • Refactor
  • Breaking change
  • Docs / chore

Notes

This PR depends on the SDK-side migration/end-state in OpenHands/software-agent-sdk#3548. This PR was created by an AI agent (OpenHands) on behalf of the user.

@enyst can click here to continue refining the PR


🐳 Docker images for this PR

GHCR package: https://github.com/OpenHands/agent-canvas/pkgs/container/agent-canvas

Component Value
Image ghcr.io/openhands/agent-canvas
Architectures amd64, arm64
Agent Server ghcr.io/openhands/agent-server:1.28.1-python
Automation openhands-automation==1.0.0a9
Commit 7fcfb8fafcf4e95ca11615bcfe0f84c9161dafa9

Pull (multi-arch manifest)

# Multi-arch manifest — Docker automatically pulls the correct architecture
docker pull ghcr.io/openhands/agent-canvas:sha-7fcfb8f

Run

docker run -it --rm \
  -p 8000:8000 \
  ghcr.io/openhands/agent-canvas:sha-7fcfb8f

All tags pushed for this build

ghcr.io/openhands/agent-canvas:sha-7fcfb8f-amd64
ghcr.io/openhands/agent-canvas:openhands-public-openhands-provider-amd64
ghcr.io/openhands/agent-canvas:pr-1280-amd64
ghcr.io/openhands/agent-canvas:sha-7fcfb8f-arm64
ghcr.io/openhands/agent-canvas:openhands-public-openhands-provider-arm64
ghcr.io/openhands/agent-canvas:pr-1280-arm64
ghcr.io/openhands/agent-canvas:sha-7fcfb8f
ghcr.io/openhands/agent-canvas:openhands-public-openhands-provider
ghcr.io/openhands/agent-canvas:pr-1280

About Multi-Architecture Support

  • Each tag (e.g., sha-7fcfb8f) is a multi-arch manifest supporting both amd64 and arm64
  • Docker automatically pulls the correct architecture for your platform
  • Individual architecture tags (e.g., sha-7fcfb8f-amd64) are also available if needed

@vercel

vercel Bot commented Jun 9, 2026

Copy link
Copy Markdown

The latest updates on your projects. Learn more about Vercel for GitHub.

Project Deployment Actions Updated (UTC)
agent-canvas Ready Ready Preview, Comment Jun 13, 2026 1:47am

Request Review

@github-actions

github-actions Bot commented Jun 9, 2026

Copy link
Copy Markdown
Contributor

❌ Mock-LLM E2E Tests

43/44 passed · 1 failed

Commit: da06eb48 · Workflow run · Test artifacts

Status Test Duration
mock-llm-acp-agent.spec.ts › mock-LLM ACP agent conversation › step 1: configure ACP agent via Settings → Agent UI 13.7s
mock-llm-acp-agent.spec.ts › mock-LLM ACP agent conversation › step 2: reload and verify ACP settings are persisted in UI 5.5s
mock-llm-acp-agent.spec.ts › mock-LLM ACP agent conversation › step 3: start ACP conversation and verify agent reply 6.3s
mock-llm-acp-agent.spec.ts › mock-LLM ACP agent conversation › step 4: resume ACP conversation from sidebar after navigating away 5.7s
mock-llm-auth-modes.spec.ts › auth mode: fresh install with runtime-injected key › reaches the onboarding modal without pre-seeded localStorage 1.3s
mock-llm-auth-modes.spec.ts › auth mode: non-public key rotation › recovers when localStorage has a stale session API key 5.3s
mock-llm-auth-modes.spec.ts › auth mode: public gate › shows the auth screen when no key is configured 1.1s
mock-llm-auth-modes.spec.ts › auth mode: public gate › rejects an incorrect key with an inline error 1.4s
mock-llm-auth-modes.spec.ts › auth mode: public gate › allows access after pasting the correct key 1.7s
mock-llm-auth-modes.spec.ts › auth mode: public gate › skips auth screen for returning user with valid stored key 728ms
mock-llm-auth-modes.spec.ts › auth mode: public gate › re-prompts when the server rotates its key (stale localStorage) 1.5s
mock-llm-automation.spec.ts › mock-LLM automation lifecycle › step 1: setup LLM profile and register automation trajectory 7.4s
mock-llm-automation.spec.ts › mock-LLM automation lifecycle › step 2: create automation and dispatch run via the UI 30.4s
mock-llm-automation.spec.ts › mock-LLM automation lifecycle › step 3: verify automation and run on the automations page 6.2s
mock-llm-conversation.spec.ts › mock-LLM agent-server conversation › step 1: create an LLM profile pointing at the mock LLM server 6.2s
mock-llm-conversation.spec.ts › mock-LLM agent-server conversation › step 2: activate the mock-llm profile and verify settings API 6.2s
mock-llm-conversation.spec.ts › mock-LLM agent-server conversation › step 3: run a conversation with the mock LLM 6.5s
mock-llm-conversation.spec.ts › mock-LLM agent-server conversation › step 4: resume conversation from sidebar after navigating away 5.7s
mock-llm-cross-connect.spec.ts › cross-connect: frontend-only → backend-only › frontend-only connects to a separate backend-only instance 15.9s
mock-llm-cross-connect.spec.ts › cross-connect: frontend-only → multiple backends › connects to two separate backends and switches between them 19.8s
mock-llm-folder-workspace.spec.ts › mock-LLM folder browser → workspace → conversation › step 1: browse to a folder, add it as a workspace, and launch a conversation with the correct working_dir 7.8s
mock-llm-image-upload.spec.ts › mock-LLM image upload › attaching an image embeds it as base64 in the LLM completion call 13.5s
mock-llm-model-switch.spec.ts › mock-LLM /model slash command › step 1: configure LLM, create switch-target profile, register trajectory 13.1s
mock-llm-model-switch.spec.ts › mock-LLM /model slash command › step 2: start conversation, switch profile via /model, verify switch 6.8s
mock-llm-onboarding-happy-path.spec.ts › onboarding happy path › completes the full onboarding flow and launches a conversation 4.5s
mock-llm-onboarding-regressions.spec.ts › onboarding recent regressions › keeps the modal open on backdrop click and Escape 1.4s
mock-llm-onboarding-regressions.spec.ts › onboarding recent regressions › defaults the LLM setup step to OpenAI GPT-5.5 1.8s
mock-llm-partial-stack.spec.ts › partial stack: --frontend-only › serves the frontend but returns 503 for backend routes 7.4s
mock-llm-partial-stack.spec.ts › partial stack: --backend-only › serves backend APIs but returns 503 for the frontend root 14.1s
mock-llm-partial-stack.spec.ts › partial stack: port conflict › fails with a clear error when the ingress port is occupied 114ms
mock-llm-partial-stack.spec.ts › partial stack: port conflict › starts successfully on a free port after a conflict 6.0s
mock-llm-preset-automation.spec.ts › preset automation → slash command conversation › automation card sends the correct slash command to a conversation 16.5s
mock-llm-preset-automation.spec.ts › preset automation → slash command conversation › direct slash command from home page triggers skill activation 13.5s
mock-llm-profile-management.spec.ts › active profile deletion + reconciliation › active profile is deletable and reconciliation activates another profile 8.6s
mock-llm-profile-management.spec.ts › same-model profile identity › chat header shows the correct profile when two profiles share the same model 15.0s
mock-llm-profile-management.spec.ts › litellm_proxy proxy base_url preservation › re-saving a litellm_proxy profile from Basic view preserves the proxy base_url 6.9s
mock-llm-skills.spec.ts › skill loading: project, user, and deletion › project skill in workspace/.agents/skills/ triggers on matching keyword 14.4s
mock-llm-skills.spec.ts › skill loading: project, user, and deletion › user skill in ~/.openhands/skills/ triggers on matching keyword 13.5s
mock-llm-skills.spec.ts › skill loading: project, user, and deletion › deleting a user skill removes it from subsequent conversations 13.4s
mock-llm-ui-regressions.spec.ts › UI regressions › scopes standalone styles to the agent-server-ui shell 1.3s
mock-llm-ui-regressions.spec.ts › UI regressions › renders critic results on agent messages and finish actions 1.4s
mock-llm-ui-regressions.spec.ts › UI regressions › loads older events when scrolling up 1.6s
mock-llm-ui-regressions.spec.ts › UI regressions › selected workspace persists after navigating away and returning 2.0s
mock-llm-ui-regressions.spec.ts › UI regressions › cleared sessionStorage yields empty workspace selection 927ms
🔍 Failure details (1)

❌ mock-llm-profile-management.spec.ts › litellm_proxy proxy base_url preservation › re-saving a litellm_proxy profile from Basic view preserves the proxy base_url

Error: base_url must be the All-Hands proxy URL after a Basic-tab re-save; dropping it strands the profile (issue #1146)

expect(received).toBe(expected) // Object.is equality

Expected: "https://llm-proxy.app.all-hands.dev/"
Received: null

Posted by the Mock-LLM E2E workflow · results are deterministic (scripted LLM responses)

@github-actions

github-actions Bot commented Jun 9, 2026

Copy link
Copy Markdown
Contributor

❌ Mock-LLM Docker E2E Test Results

38/44 passed · 1 failed · 5 skipped

Commit: da06eb48 · Workflow run · Test artifacts

Status Test Duration
mock-llm-acp-agent.spec.ts › mock-LLM ACP agent conversation › step 1: configure ACP agent via Settings → Agent UI 13.9s
mock-llm-acp-agent.spec.ts › mock-LLM ACP agent conversation › step 2: reload and verify ACP settings are persisted in UI 5.6s
mock-llm-acp-agent.spec.ts › mock-LLM ACP agent conversation › step 3: start ACP conversation and verify agent reply 6.8s
mock-llm-acp-agent.spec.ts › mock-LLM ACP agent conversation › step 4: resume ACP conversation from sidebar after navigating away 5.8s
mock-llm-auth-modes.spec.ts › auth mode: fresh install with runtime-injected key › reaches the onboarding modal without pre-seeded localStorage 1.3s
mock-llm-auth-modes.spec.ts › auth mode: non-public key rotation › recovers when localStorage has a stale session API key 5.4s
mock-llm-auth-modes.spec.ts › auth mode: public gate › shows the auth screen when no key is configured 1.2s
mock-llm-auth-modes.spec.ts › auth mode: public gate › rejects an incorrect key with an inline error 1.4s
mock-llm-auth-modes.spec.ts › auth mode: public gate › allows access after pasting the correct key 1.7s
mock-llm-auth-modes.spec.ts › auth mode: public gate › skips auth screen for returning user with valid stored key 774ms
mock-llm-auth-modes.spec.ts › auth mode: public gate › re-prompts when the server rotates its key (stale localStorage) 1.5s
mock-llm-automation.spec.ts › mock-LLM automation lifecycle › step 1: setup LLM profile and register automation trajectory 7.4s
mock-llm-automation.spec.ts › mock-LLM automation lifecycle › step 2: create automation and dispatch run via the UI 33.5s
mock-llm-automation.spec.ts › mock-LLM automation lifecycle › step 3: verify automation and run on the automations page 6.1s
mock-llm-conversation.spec.ts › mock-LLM agent-server conversation › step 1: create an LLM profile pointing at the mock LLM server 6.2s
mock-llm-conversation.spec.ts › mock-LLM agent-server conversation › step 2: activate the mock-llm profile and verify settings API 6.2s
mock-llm-conversation.spec.ts › mock-LLM agent-server conversation › step 3: run a conversation with the mock LLM 6.3s
mock-llm-conversation.spec.ts › mock-LLM agent-server conversation › step 4: resume conversation from sidebar after navigating away 5.7s
⏭️ mock-llm-cross-connect.spec.ts › cross-connect: frontend-only → backend-only › frontend-only connects to a separate backend-only instance 189ms
⏭️ mock-llm-cross-connect.spec.ts › cross-connect: frontend-only → multiple backends › connects to two separate backends and switches between them 185ms
mock-llm-folder-workspace.spec.ts › mock-LLM folder browser → workspace → conversation › step 1: browse to a folder, add it as a workspace, and launch a conversation with the correct working_dir 8.2s
mock-llm-image-upload.spec.ts › mock-LLM image upload › attaching an image embeds it as base64 in the LLM completion call 13.3s
mock-llm-model-switch.spec.ts › mock-LLM /model slash command › step 1: configure LLM, create switch-target profile, register trajectory 13.2s
mock-llm-model-switch.spec.ts › mock-LLM /model slash command › step 2: start conversation, switch profile via /model, verify switch 6.7s
mock-llm-onboarding-happy-path.spec.ts › onboarding happy path › completes the full onboarding flow and launches a conversation 3.4s
mock-llm-onboarding-regressions.spec.ts › onboarding recent regressions › keeps the modal open on backdrop click and Escape 1.4s
mock-llm-onboarding-regressions.spec.ts › onboarding recent regressions › defaults the LLM setup step to OpenAI GPT-5.5 1.6s
⏭️ mock-llm-partial-stack.spec.ts › partial stack: --frontend-only › serves the frontend but returns 503 for backend routes 188ms
mock-llm-partial-stack.spec.ts › partial stack: --backend-only › serves backend APIs but returns 503 for the frontend root 26.2s
⏭️ mock-llm-partial-stack.spec.ts › partial stack: port conflict › fails with a clear error when the ingress port is occupied 0ms
⏭️ mock-llm-partial-stack.spec.ts › partial stack: port conflict › starts successfully on a free port after a conflict 2ms
mock-llm-preset-automation.spec.ts › preset automation → slash command conversation › automation card sends the correct slash command to a conversation 16.5s
mock-llm-preset-automation.spec.ts › preset automation → slash command conversation › direct slash command from home page triggers skill activation 13.3s
mock-llm-profile-management.spec.ts › active profile deletion + reconciliation › active profile is deletable and reconciliation activates another profile (1 retries) 18.3s
mock-llm-profile-management.spec.ts › same-model profile identity › chat header shows the correct profile when two profiles share the same model (1 retries) 31.2s
mock-llm-profile-management.spec.ts › litellm_proxy proxy base_url preservation › re-saving a litellm_proxy profile from Basic view preserves the proxy base_url (1 retries) 14.0s
mock-llm-skills.spec.ts › skill loading: project, user, and deletion › project skill in workspace/.agents/skills/ triggers on matching keyword 13.6s
mock-llm-skills.spec.ts › skill loading: project, user, and deletion › user skill in ~/.openhands/skills/ triggers on matching keyword 13.2s
mock-llm-skills.spec.ts › skill loading: project, user, and deletion › deleting a user skill removes it from subsequent conversations 13.2s
mock-llm-ui-regressions.spec.ts › UI regressions › scopes standalone styles to the agent-server-ui shell 1.3s
mock-llm-ui-regressions.spec.ts › UI regressions › renders critic results on agent messages and finish actions 1.4s
mock-llm-ui-regressions.spec.ts › UI regressions › loads older events when scrolling up 1.6s
mock-llm-ui-regressions.spec.ts › UI regressions › selected workspace persists after navigating away and returning 1.9s
mock-llm-ui-regressions.spec.ts › UI regressions › cleared sessionStorage yields empty workspace selection 924ms
🔍 Failure details (1)

❌ mock-llm-profile-management.spec.ts › litellm_proxy proxy base_url preservation › re-saving a litellm_proxy profile from Basic view preserves the proxy base_url

Error: base_url must be the All-Hands proxy URL after a Basic-tab re-save; dropping it strands the profile (issue #1146)

expect(received).toBe(expected) // Object.is equality

Expected: "https://llm-proxy.app.all-hands.dev/"
Received: null

Posted by the Mock-LLM E2E workflow · results are deterministic (scripted LLM responses)

@enyst

enyst commented Jun 9, 2026

Copy link
Copy Markdown
Member Author

@OpenHands Read all comments and see the e2e test failing in a comment: I think maybe that's exactly what this PR changes, so we should change that test: either remove it if it's not useful, or replace it with the actual intended behavior in this PR.

@openhands-ai

openhands-ai Bot commented Jun 9, 2026

Copy link
Copy Markdown

I'm on it! enyst can track my progress at all-hands.dev

all-hands-bot commented Jun 9, 2026

Copy link
Copy Markdown
Contributor

Review complete.

This review was performed through OpenHands Cloud Automation. You can log in and view the conversation here.

Review Summary:

  • ✅ Approved — clean refactor that removes unnecessary complexity
  • 🟢 Risk: LOW
  • The PR removes frontend reverse-mapping logic for litellm_proxy/*openhands/* models, letting the SDK handle transport details at runtime.

View full review

@github-actions

github-actions Bot commented Jun 9, 2026

Copy link
Copy Markdown
Contributor

❌ Mock-LLM E2E Tests

43/44 passed · 1 failed

Commit: 992d57c3 · Workflow run · Test artifacts

Status Test Duration
mock-llm-acp-agent.spec.ts › mock-LLM ACP agent conversation › step 1: configure ACP agent via Settings → Agent UI 13.7s
mock-llm-acp-agent.spec.ts › mock-LLM ACP agent conversation › step 2: reload and verify ACP settings are persisted in UI 5.6s
mock-llm-acp-agent.spec.ts › mock-LLM ACP agent conversation › step 3: start ACP conversation and verify agent reply 6.3s
mock-llm-acp-agent.spec.ts › mock-LLM ACP agent conversation › step 4: resume ACP conversation from sidebar after navigating away 5.7s
mock-llm-auth-modes.spec.ts › auth mode: fresh install with runtime-injected key › reaches the onboarding modal without pre-seeded localStorage 1.3s
mock-llm-auth-modes.spec.ts › auth mode: non-public key rotation › recovers when localStorage has a stale session API key 5.3s
mock-llm-auth-modes.spec.ts › auth mode: public gate › shows the auth screen when no key is configured 1.2s
mock-llm-auth-modes.spec.ts › auth mode: public gate › rejects an incorrect key with an inline error 1.4s
mock-llm-auth-modes.spec.ts › auth mode: public gate › allows access after pasting the correct key 1.4s
mock-llm-auth-modes.spec.ts › auth mode: public gate › skips auth screen for returning user with valid stored key 773ms
mock-llm-auth-modes.spec.ts › auth mode: public gate › re-prompts when the server rotates its key (stale localStorage) 1.5s
mock-llm-automation.spec.ts › mock-LLM automation lifecycle › step 1: setup LLM profile and register automation trajectory 7.3s
mock-llm-automation.spec.ts › mock-LLM automation lifecycle › step 2: create automation and dispatch run via the UI 30.4s
mock-llm-automation.spec.ts › mock-LLM automation lifecycle › step 3: verify automation and run on the automations page 6.3s
mock-llm-conversation.spec.ts › mock-LLM agent-server conversation › step 1: create an LLM profile pointing at the mock LLM server 6.3s
mock-llm-conversation.spec.ts › mock-LLM agent-server conversation › step 2: activate the mock-llm profile and verify settings API 7.8s
mock-llm-conversation.spec.ts › mock-LLM agent-server conversation › step 3: run a conversation with the mock LLM 6.4s
mock-llm-conversation.spec.ts › mock-LLM agent-server conversation › step 4: resume conversation from sidebar after navigating away 5.7s
mock-llm-cross-connect.spec.ts › cross-connect: frontend-only → backend-only › frontend-only connects to a separate backend-only instance 15.9s
mock-llm-cross-connect.spec.ts › cross-connect: frontend-only → multiple backends › connects to two separate backends and switches between them 19.8s
mock-llm-folder-workspace.spec.ts › mock-LLM folder browser → workspace → conversation › step 1: browse to a folder, add it as a workspace, and launch a conversation with the correct working_dir 7.9s
mock-llm-image-upload.spec.ts › mock-LLM image upload › attaching an image embeds it as base64 in the LLM completion call 13.5s
mock-llm-model-switch.spec.ts › mock-LLM /model slash command › step 1: configure LLM, create switch-target profile, register trajectory 13.2s
mock-llm-model-switch.spec.ts › mock-LLM /model slash command › step 2: start conversation, switch profile via /model, verify switch 7.7s
mock-llm-onboarding-happy-path.spec.ts › onboarding happy path › completes the full onboarding flow and launches a conversation 4.4s
mock-llm-onboarding-regressions.spec.ts › onboarding recent regressions › keeps the modal open on backdrop click and Escape 1.4s
mock-llm-onboarding-regressions.spec.ts › onboarding recent regressions › defaults the LLM setup step to OpenAI GPT-5.5 1.6s
mock-llm-partial-stack.spec.ts › partial stack: --frontend-only › serves the frontend but returns 503 for backend routes 7.4s
mock-llm-partial-stack.spec.ts › partial stack: --backend-only › serves backend APIs but returns 503 for the frontend root 14.1s
mock-llm-partial-stack.spec.ts › partial stack: port conflict › fails with a clear error when the ingress port is occupied 116ms
mock-llm-partial-stack.spec.ts › partial stack: port conflict › starts successfully on a free port after a conflict 6.0s
mock-llm-preset-automation.spec.ts › preset automation → slash command conversation › automation card sends the correct slash command to a conversation 16.0s
mock-llm-preset-automation.spec.ts › preset automation → slash command conversation › direct slash command from home page triggers skill activation 13.6s
mock-llm-profile-management.spec.ts › active profile deletion + reconciliation › active profile is deletable and reconciliation activates another profile 8.5s
mock-llm-profile-management.spec.ts › same-model profile identity › chat header shows the correct profile when two profiles share the same model 15.8s
mock-llm-profile-management.spec.ts › OpenHands provider Basic-mode profile save › re-saving an openhands profile from Basic view drops proxy base_url 7.0s
mock-llm-skills.spec.ts › skill loading: project, user, and deletion › project skill in workspace/.agents/skills/ triggers on matching keyword 14.7s
mock-llm-skills.spec.ts › skill loading: project, user, and deletion › user skill in ~/.openhands/skills/ triggers on matching keyword 13.5s
mock-llm-skills.spec.ts › skill loading: project, user, and deletion › deleting a user skill removes it from subsequent conversations 13.5s
mock-llm-ui-regressions.spec.ts › UI regressions › scopes standalone styles to the agent-server-ui shell 1.4s
mock-llm-ui-regressions.spec.ts › UI regressions › renders critic results on agent messages and finish actions 1.5s
mock-llm-ui-regressions.spec.ts › UI regressions › loads older events when scrolling up 1.6s
mock-llm-ui-regressions.spec.ts › UI regressions › selected workspace persists after navigating away and returning 2.0s
mock-llm-ui-regressions.spec.ts › UI regressions › cleared sessionStorage yields empty workspace selection 957ms
🔍 Failure details (1)

❌ mock-llm-profile-management.spec.ts › OpenHands provider Basic-mode profile save › re-saving an openhands profile from Basic view drops proxy base_url

Error: expect(received).toBe(expected) // Object.is equality

Expected: "openhands/claude-opus-4-8"
Received: "litellm_proxy/claude-opus-4-8"

Posted by the Mock-LLM E2E workflow · results are deterministic (scripted LLM responses)

@all-hands-bot all-hands-bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review: settings: trust public OpenHands provider models

🟢 Good taste — Clean refactor that removes unnecessary complexity.

Summary

This PR removes frontend reverse-mapping logic for litellm_proxy/*openhands/* models, letting the SDK handle transport-level details (LiteLLM proxy base URL) at runtime. The changes are well-scoped and the removal of normalizeDisplayModel and related utilities is the right call now that the SDK has settled on openhands/* as the canonical namespace.

Files Changed

File Change
model-selector.tsx Remove currentBaseUrl prop and complex bootstrap effect
llm-settings-local-view.tsx Drop openHandsProxyModel check, simplify Basic-tab save
llm-settings.tsx Remove isOpenHandsProxyModel import and proxy base_url handling
settings-form.tsx Remove currentBaseUrl prop passthrough
normalize-display-model.ts Deleted (no longer needed)
openhands-llm.ts Deleted (no longer needed)
__tests__/... Updated to cover public openhands/* end state

Observations

[model-selector.tsx, Lines 68-79] The removed useEffect was doing extra work: it waited for openhandsVerifiedModels to load before initializing the selector, which added a dependency and an async gate. The new effect simply parses the model ID directly. ✅

[llm-settings-local-view.tsx, Lines 253-260] Removing the conditional that re-stamped the proxy base_url is correct. The SDK now owns this at transport time; the frontend shouldn't be guessing transport details from the model prefix.

[llm-settings.tsx, Lines 40-49] The removal of openhands and litellm_proxy from KNOWN_PROVIDER_DEFAULT_BASE_URLS is consistent with the new model — if the provider prefix is enough, there's no need to hardcode proxy URLs as "known defaults".

Testing

The updated tests correctly reflect the new behavior:

  • Unit tests for model-selector-openhands.test.tsx and llm-settings-local-view.test.tsx now use openhands/<m> directly and verify that base_url is dropped in Basic mode
  • The e2e test was updated to match the new semantics
  • The two deleted utility files had good coverage; the refactored tests preserve that coverage

PR Description

  • ✅ Explains the why (SDK migration to openhands/* as canonical namespace)
  • ✅ Links to related SDK PR (#3548)
  • ✅ Provides test commands
  • ⚠️ The Type section has [x] on Refactor but the checkbox labels don't match the checked state (Breaking change is unchecked, which is correct for a refactor)

[RISK ASSESSMENT]

  • [Overall PR] ⚠️ Risk Assessment: 🟢 LOW

This is a pure refactor with no behavior changes visible to end users. The changes:

  • Remove dead code (deleted files)
  • Simplify existing components (removed props, simplified effects)
  • Update tests to match new expected behavior

No breaking changes, no security implications, no new dependencies. CI should validate the test suite passes.


Worth merging: Clean, well-scoped refactor that reduces codebase complexity.

KEY INSIGHT: The SDK now owns openhands/*litellm_proxy/* translation at transport time, so the frontend doesn't need to maintain proxy base URL awareness — it just uses the public namespace and lets the SDK derive transport details.


This review was generated by an AI agent (OpenHands) on behalf of the user through OpenHands Automation. View conversation

Comment thread src/components/shared/modals/settings/model-selector.tsx
@github-actions

github-actions Bot commented Jun 9, 2026

Copy link
Copy Markdown
Contributor

🛑 Mock-LLM Docker E2E Test Results

31/37 passed · 1 failed · 5 skipped · ⚠️ 7 not run (process killed at 37/44)

Commit: 992d57c3 · Workflow run · Test artifacts

Status Test Duration
chromium › mock-llm-acp-agent.spec.ts › mock-LLM ACP agent conversation › step 1: configure ACP agent via Settings → Agent UI 13.7s
chromium › mock-llm-acp-agent.spec.ts › mock-LLM ACP agent conversation › step 2: reload and verify ACP settings are persisted in UI 5.5s
chromium › mock-llm-acp-agent.spec.ts › mock-LLM ACP agent conversation › step 3: start ACP conversation and verify agent reply 6.7s
chromium › mock-llm-acp-agent.spec.ts › mock-LLM ACP agent conversation › step 4: resume ACP conversation from sidebar after navigating away 5.7s
chromium › mock-llm-auth-modes.spec.ts › auth mode: fresh install with runtime-injected key › reaches the onboarding modal without pre-seeded localStorage 974ms
chromium › mock-llm-auth-modes.spec.ts › auth mode: non-public key rotation › recovers when localStorage has a stale session API key 5.3s
chromium › mock-llm-auth-modes.spec.ts › auth mode: public gate › shows the auth screen when no key is configured 1.2s
chromium › mock-llm-auth-modes.spec.ts › auth mode: public gate › rejects an incorrect key with an inline error 1.4s
chromium › mock-llm-auth-modes.spec.ts › auth mode: public gate › allows access after pasting the correct key 1.6s
chromium › mock-llm-auth-modes.spec.ts › auth mode: public gate › skips auth screen for returning user with valid stored key 706ms
chromium › mock-llm-auth-modes.spec.ts › auth mode: public gate › re-prompts when the server rotates its key (stale localStorage) 1.4s
chromium › mock-llm-automation.spec.ts › mock-LLM automation lifecycle › step 1: setup LLM profile and register automation trajectory 7.3s
chromium › mock-llm-automation.spec.ts › mock-LLM automation lifecycle › step 2: create automation and dispatch run via the UI 32.4s
chromium › mock-llm-automation.spec.ts › mock-LLM automation lifecycle › step 3: verify automation and run on the automations page 6.2s
chromium › mock-llm-conversation.spec.ts › mock-LLM agent-server conversation › step 1: create an LLM profile pointing at the mock LLM server 6.2s
chromium › mock-llm-conversation.spec.ts › mock-LLM agent-server conversation › step 2: activate the mock-llm profile and verify settings API 6.2s
chromium › mock-llm-conversation.spec.ts › mock-LLM agent-server conversation › step 3: run a conversation with the mock LLM 6.3s
chromium › mock-llm-conversation.spec.ts › mock-LLM agent-server conversation › step 4: resume conversation from sidebar after navigating away 5.7s
⏭️ chromium › mock-llm-cross-connect.spec.ts › cross-connect: frontend-only → backend-only › frontend-only connects to a separate backend-only instance 173ms
⏭️ chromium › mock-llm-cross-connect.spec.ts › cross-connect: frontend-only → multiple backends › connects to two separate backends and switches between them 179ms
chromium › mock-llm-folder-workspace.spec.ts › mock-LLM folder browser → workspace → conversation › step 1: browse to a folder, add it as a workspace, and launch a conversation with the correct working_dir 8.0s
chromium › mock-llm-image-upload.spec.ts › mock-LLM image upload › attaching an image embeds it as base64 in the LLM completion call 13.3s
chromium › mock-llm-model-switch.spec.ts › mock-LLM /model slash command › step 1: configure LLM, create switch-target profile, register trajectory 13.0s
chromium › mock-llm-model-switch.spec.ts › mock-LLM /model slash command › step 2: start conversation, switch profile via /model, verify switch 6.6s
chromium › mock-llm-onboarding-happy-path.spec.ts › onboarding happy path › completes the full onboarding flow and launches a conversation 3.0s
chromium › mock-llm-onboarding-regressions.spec.ts › onboarding recent regressions › keeps the modal open on backdrop click and Escape 1.0s
chromium › mock-llm-onboarding-regressions.spec.ts › onboarding recent regressions › defaults the LLM setup step to OpenAI GPT-5.5 1.4s
⏭️ chromium › mock-llm-partial-stack.spec.ts › partial stack: --frontend-only › serves the frontend but returns 503 for backend routes 173ms
chromium › mock-llm-partial-stack.spec.ts › partial stack: --backend-only › serves backend APIs but returns 503 for the frontend root 25.1s
⏭️ chromium › mock-llm-partial-stack.spec.ts › partial stack: port conflict › fails with a clear error when the ingress port is occupied 1ms
⏭️ chromium › mock-llm-partial-stack.spec.ts › partial stack: port conflict › starts successfully on a free port after a conflict 2ms
chromium › mock-llm-preset-automation.spec.ts › preset automation → slash command conversation › automation card sends the correct slash command to a conversation 15.8s
chromium › mock-llm-preset-automation.spec.ts › preset automation → slash command conversation › direct slash command from home page triggers skill activation 13.3s
chromium › mock-llm-profile-management.spec.ts › active profile deletion + reconciliation › active profile is deletable and reconciliation activates another profile 8.5s
chromium › mock-llm-profile-management.spec.ts › same-model profile identity › chat header shows the correct profile when two profiles share the same model 14.8s
chromium › mock-llm-profile-management.spec.ts › OpenHands provider Basic-mode profile save › re-saving an openhands profile from Basic view drops proxy base_url 6.9s
chromium › mock-llm-profile-management.spec.ts › active profile deletion + reconciliation › active profile is deletable and reconciliation activates another profile 9.4s
🔍 Failure details (1)

❌ chromium › mock-llm-profile-management.spec.ts › OpenHands provider Basic-mode profile save › re-saving an openhands profile from Basic view drops proxy base_url

Error: �[2mexpect(�[22m�[31mreceived�[39m�[2m).�[22mtoBe�[2m(�[22m�[32mexpected�[39m�[2m) // Object.is equality�[22m

Expected: �[32m"�[7mopenhands�[27m/claude-opus-4-8"�[39m
Received: �[31m"�[7mlitellm_proxy�[27m/claude-opus-4-8"�[39m

Posted by the Mock-LLM E2E workflow · results are deterministic (scripted LLM responses)

enyst commented Jun 9, 2026

Copy link
Copy Markdown
Member Author

Updated the mock-LLM profile-management E2E in commit 98c8fdb to match this PR's intended behavior: Basic-mode OpenHands profile saves now assert the stale proxy base_url is dropped, while allowing the current released SDK's transitional litellm_proxy model shape until the SDK-side migration lands.\n\nVerified locally:\n- npm test -- --run tests/components/modals/settings/model-selector-openhands.test.tsx tests/components/settings/llm-profiles/llm-settings-local-view.test.tsx\n- npm run typecheck\n- npx prettier --check tests/e2e/mock-llm/mock-llm-profile-management.spec.ts\n- MOCK_LLM_PYTHON=.mock-llm-venv/bin/python3 npm run test:e2e:mock-llm -- mock-llm-profile-management.spec.ts -g "OpenHands provider base_url normalization"\n\nThis comment was created by an AI agent (OpenHands) on behalf of the user.

@github-actions

github-actions Bot commented Jun 9, 2026

Copy link
Copy Markdown
Contributor

⚠️ Mock-LLM Docker E2E Test Results

0/0 passed

Commit: 87058933 · Workflow run

Status Test Duration

Posted by the Mock-LLM E2E workflow · results are deterministic (scripted LLM responses)

@github-actions

github-actions Bot commented Jun 9, 2026

Copy link
Copy Markdown
Contributor

🛑 Mock-LLM E2E Tests

22/22 passed · ⚠️ 22 not run (process killed at 22/44)

Commit: 87058933 · Workflow run · Test artifacts

Status Test Duration
chromium › mock-llm-acp-agent.spec.ts › mock-LLM ACP agent conversation › step 1: configure ACP agent via Settings → Agent UI 13.9s
chromium › mock-llm-acp-agent.spec.ts › mock-LLM ACP agent conversation › step 2: reload and verify ACP settings are persisted in UI 5.6s
chromium › mock-llm-acp-agent.spec.ts › mock-LLM ACP agent conversation › step 3: start ACP conversation and verify agent reply 6.7s
chromium › mock-llm-acp-agent.spec.ts › mock-LLM ACP agent conversation › step 4: resume ACP conversation from sidebar after navigating away 5.7s
chromium › mock-llm-auth-modes.spec.ts › auth mode: fresh install with runtime-injected key › reaches the onboarding modal without pre-seeded localStorage 1.3s
chromium › mock-llm-auth-modes.spec.ts › auth mode: non-public key rotation › recovers when localStorage has a stale session API key 5.3s
chromium › mock-llm-auth-modes.spec.ts › auth mode: public gate › shows the auth screen when no key is configured 1.2s
chromium › mock-llm-auth-modes.spec.ts › auth mode: public gate › rejects an incorrect key with an inline error 1.3s
chromium › mock-llm-auth-modes.spec.ts › auth mode: public gate › allows access after pasting the correct key 1.4s
chromium › mock-llm-auth-modes.spec.ts › auth mode: public gate › skips auth screen for returning user with valid stored key 741ms
chromium › mock-llm-auth-modes.spec.ts › auth mode: public gate › re-prompts when the server rotates its key (stale localStorage) 1.5s
chromium › mock-llm-automation.spec.ts › mock-LLM automation lifecycle › step 1: setup LLM profile and register automation trajectory 7.4s
chromium › mock-llm-automation.spec.ts › mock-LLM automation lifecycle › step 2: create automation and dispatch run via the UI 29.5s
chromium › mock-llm-automation.spec.ts › mock-LLM automation lifecycle › step 3: verify automation and run on the automations page 6.2s
chromium › mock-llm-conversation.spec.ts › mock-LLM agent-server conversation › step 1: create an LLM profile pointing at the mock LLM server 6.2s
chromium › mock-llm-conversation.spec.ts › mock-LLM agent-server conversation › step 2: activate the mock-llm profile and verify settings API 6.2s
chromium › mock-llm-conversation.spec.ts › mock-LLM agent-server conversation › step 3: run a conversation with the mock LLM 6.5s
chromium › mock-llm-conversation.spec.ts › mock-LLM agent-server conversation › step 4: resume conversation from sidebar after navigating away 5.7s
chromium › mock-llm-cross-connect.spec.ts › cross-connect: frontend-only → backend-only › frontend-only connects to a separate backend-only instance 15.8s
chromium › mock-llm-cross-connect.spec.ts › cross-connect: frontend-only → multiple backends › connects to two separate backends and switches between them 21.2s
chromium › mock-llm-folder-workspace.spec.ts › mock-LLM folder browser → workspace → conversation › step 1: browse to a folder, add it as a workspace, and launch a conversation with the correct working_dir 7.9s
chromium › mock-llm-image-upload.spec.ts › mock-LLM image upload › attaching an image embeds it as base64 in the LLM completion call 14.3s

Posted by the Mock-LLM E2E workflow · results are deterministic (scripted LLM responses)

@openhands-ai

openhands-ai Bot commented Jun 9, 2026

Copy link
Copy Markdown

Since my last summary, there were no additional code changes.

Final status:

  • ✅ The PR comment request has been fully addressed.
  • ✅ The failing E2E expectation was updated to match the intended behavior of this PR.
  • ✅ The change was committed and pushed to the existing PR branch.
  • ✅ A PR comment was posted with the fix and verification details.
  • ✅ The final branch is clean and synced with origin/openhands/public-openhands-provider.
  • ✅ The changes are concise and scoped to the relevant mock-LLM E2E test; no extraneous changes remain.

@github-actions

github-actions Bot commented Jun 9, 2026

Copy link
Copy Markdown
Contributor

✅ Mock-LLM E2E Tests

44/44 passed

Commit: 98c8fdbb · Workflow run · Test artifacts

Status Test Duration
mock-llm-acp-agent.spec.ts › mock-LLM ACP agent conversation › step 1: configure ACP agent via Settings → Agent UI 13.8s
mock-llm-acp-agent.spec.ts › mock-LLM ACP agent conversation › step 2: reload and verify ACP settings are persisted in UI 5.6s
mock-llm-acp-agent.spec.ts › mock-LLM ACP agent conversation › step 3: start ACP conversation and verify agent reply 6.3s
mock-llm-acp-agent.spec.ts › mock-LLM ACP agent conversation › step 4: resume ACP conversation from sidebar after navigating away 5.7s
mock-llm-auth-modes.spec.ts › auth mode: fresh install with runtime-injected key › reaches the onboarding modal without pre-seeded localStorage 1.3s
mock-llm-auth-modes.spec.ts › auth mode: non-public key rotation › recovers when localStorage has a stale session API key 5.3s
mock-llm-auth-modes.spec.ts › auth mode: public gate › shows the auth screen when no key is configured 1.2s
mock-llm-auth-modes.spec.ts › auth mode: public gate › rejects an incorrect key with an inline error 1.4s
mock-llm-auth-modes.spec.ts › auth mode: public gate › allows access after pasting the correct key 1.7s
mock-llm-auth-modes.spec.ts › auth mode: public gate › skips auth screen for returning user with valid stored key 4.7s
mock-llm-auth-modes.spec.ts › auth mode: public gate › re-prompts when the server rotates its key (stale localStorage) 1.5s
mock-llm-automation.spec.ts › mock-LLM automation lifecycle › step 1: setup LLM profile and register automation trajectory 7.3s
mock-llm-automation.spec.ts › mock-LLM automation lifecycle › step 2: create automation and dispatch run via the UI 26.4s
mock-llm-automation.spec.ts › mock-LLM automation lifecycle › step 3: verify automation and run on the automations page 6.1s
mock-llm-conversation.spec.ts › mock-LLM agent-server conversation › step 1: create an LLM profile pointing at the mock LLM server 6.2s
mock-llm-conversation.spec.ts › mock-LLM agent-server conversation › step 2: activate the mock-llm profile and verify settings API 7.7s
mock-llm-conversation.spec.ts › mock-LLM agent-server conversation › step 3: run a conversation with the mock LLM 7.3s
mock-llm-conversation.spec.ts › mock-LLM agent-server conversation › step 4: resume conversation from sidebar after navigating away 5.7s
mock-llm-cross-connect.spec.ts › cross-connect: frontend-only → backend-only › frontend-only connects to a separate backend-only instance 15.8s
mock-llm-cross-connect.spec.ts › cross-connect: frontend-only → multiple backends › connects to two separate backends and switches between them 20.8s
mock-llm-folder-workspace.spec.ts › mock-LLM folder browser → workspace → conversation › step 1: browse to a folder, add it as a workspace, and launch a conversation with the correct working_dir 7.8s
mock-llm-image-upload.spec.ts › mock-LLM image upload › attaching an image embeds it as base64 in the LLM completion call 13.6s
mock-llm-model-switch.spec.ts › mock-LLM /model slash command › step 1: configure LLM, create switch-target profile, register trajectory 13.0s
mock-llm-model-switch.spec.ts › mock-LLM /model slash command › step 2: start conversation, switch profile via /model, verify switch 7.6s
mock-llm-onboarding-happy-path.spec.ts › onboarding happy path › completes the full onboarding flow and launches a conversation 4.4s
mock-llm-onboarding-regressions.spec.ts › onboarding recent regressions › keeps the modal open on backdrop click and Escape 1.3s
mock-llm-onboarding-regressions.spec.ts › onboarding recent regressions › defaults the LLM setup step to OpenAI GPT-5.5 1.6s
mock-llm-partial-stack.spec.ts › partial stack: --frontend-only › serves the frontend but returns 503 for backend routes 7.3s
mock-llm-partial-stack.spec.ts › partial stack: --backend-only › serves backend APIs but returns 503 for the frontend root 13.1s
mock-llm-partial-stack.spec.ts › partial stack: port conflict › fails with a clear error when the ingress port is occupied 103ms
mock-llm-partial-stack.spec.ts › partial stack: port conflict › starts successfully on a free port after a conflict 6.0s
mock-llm-preset-automation.spec.ts › preset automation → slash command conversation › automation card sends the correct slash command to a conversation 15.9s
mock-llm-preset-automation.spec.ts › preset automation → slash command conversation › direct slash command from home page triggers skill activation 13.4s
mock-llm-profile-management.spec.ts › active profile deletion + reconciliation › active profile is deletable and reconciliation activates another profile 8.4s
mock-llm-profile-management.spec.ts › same-model profile identity › chat header shows the correct profile when two profiles share the same model 15.0s
mock-llm-profile-management.spec.ts › OpenHands provider base_url normalization › re-saving an OpenHands profile from Basic view drops proxy base_url 7.6s
mock-llm-skills.spec.ts › skill loading: project, user, and deletion › project skill in workspace/.agents/skills/ triggers on matching keyword 15.9s
mock-llm-skills.spec.ts › skill loading: project, user, and deletion › user skill in ~/.openhands/skills/ triggers on matching keyword 15.3s
mock-llm-skills.spec.ts › skill loading: project, user, and deletion › deleting a user skill removes it from subsequent conversations 14.9s
mock-llm-ui-regressions.spec.ts › UI regressions › scopes standalone styles to the agent-server-ui shell 922ms
mock-llm-ui-regressions.spec.ts › UI regressions › renders critic results on agent messages and finish actions 1.4s
mock-llm-ui-regressions.spec.ts › UI regressions › loads older events when scrolling up 1.6s
mock-llm-ui-regressions.spec.ts › UI regressions › selected workspace persists after navigating away and returning 1.9s
mock-llm-ui-regressions.spec.ts › UI regressions › cleared sessionStorage yields empty workspace selection 926ms

Posted by the Mock-LLM E2E workflow · results are deterministic (scripted LLM responses)

@github-actions

github-actions Bot commented Jun 9, 2026

Copy link
Copy Markdown
Contributor

🔶 Mock-LLM Docker E2E Test Results

39/44 passed · 5 skipped

Commit: 98c8fdbb · Workflow run · Test artifacts

Status Test Duration
mock-llm-acp-agent.spec.ts › mock-LLM ACP agent conversation › step 1: configure ACP agent via Settings → Agent UI 13.7s
mock-llm-acp-agent.spec.ts › mock-LLM ACP agent conversation › step 2: reload and verify ACP settings are persisted in UI 5.6s
mock-llm-acp-agent.spec.ts › mock-LLM ACP agent conversation › step 3: start ACP conversation and verify agent reply 6.7s
mock-llm-acp-agent.spec.ts › mock-LLM ACP agent conversation › step 4: resume ACP conversation from sidebar after navigating away 5.7s
mock-llm-auth-modes.spec.ts › auth mode: fresh install with runtime-injected key › reaches the onboarding modal without pre-seeded localStorage 1.3s
mock-llm-auth-modes.spec.ts › auth mode: non-public key rotation › recovers when localStorage has a stale session API key 5.3s
mock-llm-auth-modes.spec.ts › auth mode: public gate › shows the auth screen when no key is configured 1.2s
mock-llm-auth-modes.spec.ts › auth mode: public gate › rejects an incorrect key with an inline error 1.4s
mock-llm-auth-modes.spec.ts › auth mode: public gate › allows access after pasting the correct key 1.7s
mock-llm-auth-modes.spec.ts › auth mode: public gate › skips auth screen for returning user with valid stored key 757ms
mock-llm-auth-modes.spec.ts › auth mode: public gate › re-prompts when the server rotates its key (stale localStorage) 1.5s
mock-llm-automation.spec.ts › mock-LLM automation lifecycle › step 1: setup LLM profile and register automation trajectory 7.4s
mock-llm-automation.spec.ts › mock-LLM automation lifecycle › step 2: create automation and dispatch run via the UI 33.5s
mock-llm-automation.spec.ts › mock-LLM automation lifecycle › step 3: verify automation and run on the automations page 6.1s
mock-llm-conversation.spec.ts › mock-LLM agent-server conversation › step 1: create an LLM profile pointing at the mock LLM server 6.2s
mock-llm-conversation.spec.ts › mock-LLM agent-server conversation › step 2: activate the mock-llm profile and verify settings API 6.2s
mock-llm-conversation.spec.ts › mock-LLM agent-server conversation › step 3: run a conversation with the mock LLM 6.3s
mock-llm-conversation.spec.ts › mock-LLM agent-server conversation › step 4: resume conversation from sidebar after navigating away 5.7s
⏭️ mock-llm-cross-connect.spec.ts › cross-connect: frontend-only → backend-only › frontend-only connects to a separate backend-only instance 179ms
⏭️ mock-llm-cross-connect.spec.ts › cross-connect: frontend-only → multiple backends › connects to two separate backends and switches between them 189ms
mock-llm-folder-workspace.spec.ts › mock-LLM folder browser → workspace → conversation › step 1: browse to a folder, add it as a workspace, and launch a conversation with the correct working_dir 8.0s
mock-llm-image-upload.spec.ts › mock-LLM image upload › attaching an image embeds it as base64 in the LLM completion call 13.3s
mock-llm-model-switch.spec.ts › mock-LLM /model slash command › step 1: configure LLM, create switch-target profile, register trajectory 13.0s
mock-llm-model-switch.spec.ts › mock-LLM /model slash command › step 2: start conversation, switch profile via /model, verify switch 6.6s
mock-llm-onboarding-happy-path.spec.ts › onboarding happy path › completes the full onboarding flow and launches a conversation 3.5s
mock-llm-onboarding-regressions.spec.ts › onboarding recent regressions › keeps the modal open on backdrop click and Escape 1.4s
mock-llm-onboarding-regressions.spec.ts › onboarding recent regressions › defaults the LLM setup step to OpenAI GPT-5.5 1.6s
⏭️ mock-llm-partial-stack.spec.ts › partial stack: --frontend-only › serves the frontend but returns 503 for backend routes 178ms
mock-llm-partial-stack.spec.ts › partial stack: --backend-only › serves backend APIs but returns 503 for the frontend root 25.2s
⏭️ mock-llm-partial-stack.spec.ts › partial stack: port conflict › fails with a clear error when the ingress port is occupied 1ms
⏭️ mock-llm-partial-stack.spec.ts › partial stack: port conflict › starts successfully on a free port after a conflict 8ms
mock-llm-preset-automation.spec.ts › preset automation → slash command conversation › automation card sends the correct slash command to a conversation 16.0s
mock-llm-preset-automation.spec.ts › preset automation → slash command conversation › direct slash command from home page triggers skill activation 13.3s
mock-llm-profile-management.spec.ts › active profile deletion + reconciliation › active profile is deletable and reconciliation activates another profile 8.4s
mock-llm-profile-management.spec.ts › same-model profile identity › chat header shows the correct profile when two profiles share the same model 14.9s
mock-llm-profile-management.spec.ts › OpenHands provider base_url normalization › re-saving an OpenHands profile from Basic view drops proxy base_url 7.7s
mock-llm-skills.spec.ts › skill loading: project, user, and deletion › project skill in workspace/.agents/skills/ triggers on matching keyword 13.4s
mock-llm-skills.spec.ts › skill loading: project, user, and deletion › user skill in ~/.openhands/skills/ triggers on matching keyword 13.1s
mock-llm-skills.spec.ts › skill loading: project, user, and deletion › deleting a user skill removes it from subsequent conversations 13.2s
mock-llm-ui-regressions.spec.ts › UI regressions › scopes standalone styles to the agent-server-ui shell 1.3s
mock-llm-ui-regressions.spec.ts › UI regressions › renders critic results on agent messages and finish actions 1.4s
mock-llm-ui-regressions.spec.ts › UI regressions › loads older events when scrolling up 1.5s
mock-llm-ui-regressions.spec.ts › UI regressions › selected workspace persists after navigating away and returning 1.9s
mock-llm-ui-regressions.spec.ts › UI regressions › cleared sessionStorage yields empty workspace selection 1.1s

Posted by the Mock-LLM E2E workflow · results are deterministic (scripted LLM responses)

@enyst enyst marked this pull request as ready for review June 9, 2026 14:25
@enyst

enyst commented Jun 9, 2026

Copy link
Copy Markdown
Member Author

@OpenHands look at SDK PR OpenHands/software-agent-sdk#3548 and pin temporarily pyproject to it; then ask for review from all-hands-bot

I want it to do an end to end test on the actual SDK commit for the dependency.

@openhands-ai

openhands-ai Bot commented Jun 9, 2026

Copy link
Copy Markdown

I'm on it! enyst can track my progress at all-hands.dev

@enyst enyst requested a review from all-hands-bot June 9, 2026 16:10

enyst commented Jun 9, 2026

Copy link
Copy Markdown
Member Author

Done in 691f208: temporarily pinned the mock-LLM E2E paths to OpenHands/software-agent-sdk#3548 (SDK git ref 2e3e5e17c70eb4822e290596a8de8ec37a8cdf46) and the matching agent-server PR image for Docker builds, then requested review from @all-hands-bot.\n\nVerified locally:\n- Installed openhands-sdk from the pinned git ref and confirmed OpenHands models remain public/stored as openhands/*\n- npm test -- --run tests/components/modals/settings/model-selector-openhands.test.tsx tests/components/settings/llm-profiles/llm-settings-local-view.test.tsx\n- npm run typecheck\n\n_This comment was created by an AI agent (OpenHands) on behalf of the user._

all-hands-bot commented Jun 9, 2026

Copy link
Copy Markdown
Contributor

Review complete.

This review was performed through OpenHands Cloud Automation. You can log in and view the conversation here.

@all-hands-bot all-hands-bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review: settings: trust public OpenHands provider models

🟢 Good taste — Clean refactor that eliminates special-case handling.

Summary

This PR removes the frontend's reverse-mapping layer for litellm_proxy/*openhands/* models, delegating transport-level translation to the SDK. The deletion of normalize-display-model.ts and openhands-llm.ts is the right call now that openhands/* is the canonical namespace.

Key Observations

[config/defaults.json, Lines 16-19] The temporaryPins block is intentional scaffolding to validate against SDK PR #3548 before release. The _comment field makes the purpose clear. ✅

[src/components/features/settings/llm-profiles/llm-settings-local-view.tsx, Lines 249-255] The new save logic is simpler and correct: Basic mode drops base_url entirely, letting the SDK derive transport details. The removed isOpenHandsProxyModel check was trying to guess transport layer details in the frontend — that's not the frontend's job.

[src/routes/llm-settings.tsx, Lines 40-49] Removing openhands and litellm_proxy from KNOWN_PROVIDER_DEFAULT_BASE_URLS is consistent with the new model — if the provider prefix is sufficient, there's no need to hardcode proxy URLs as "known defaults".

[.github/workflows/mock-llm-e2e.yml, Lines 74-83] The conditional install path (git ref vs. PyPI version) is the correct pattern for testing unreleased SDK changes. ✅

Testing

The updated tests correctly reflect the new behavior:

  • Unit tests for model-selector-openhands.test.tsx and llm-settings-local-view.test.tsx now use openhands/<m> directly
  • The e2e test was renamed and updated to match the new semantics
  • The CI status shows mock-llm-docker-e2e passing with the updated test expectations ✅

Minor Notes

  1. The _comment field in defaults.json is unconventional but acceptable for temporary pins. Consider removing it before merge.

  2. The PR description's Type section checkbox labels are slightly misaligned (the checked state doesn't match the label), though this is cosmetic.


[RISK ASSESSMENT]

  • [Overall PR] ⚠️ Risk Assessment: 🟢 LOW

This is a pure refactor:

  • Removes dead code (2 deleted files, ~111 lines removed)
  • Simplifies existing components by removing special-case logic
  • Updates tests to match the new expected behavior
  • No breaking changes to user-visible behavior
  • No new dependencies
  • CI is validating the test suite (in progress)

The temporaryPins in defaults.json are scoped to this PR's testing needs and don't affect production behavior.


Worth merging: Well-scoped cleanup that reduces codebase complexity and aligns with the SDK's canonical model namespace.

KEY INSIGHT: The SDK now owns openhands/*litellm_proxy/* translation at transport time. The frontend just needs to trust the public openhands/* namespace — no reverse-mapping required.


This review was generated by an AI agent (OpenHands) on behalf of the user through OpenHands Automation. View conversation

@github-actions

github-actions Bot commented Jun 9, 2026

Copy link
Copy Markdown
Contributor

❌ Mock-LLM E2E Tests

42/44 passed · 1 failed · 1 skipped

Commit: 691f208d · Workflow run · Test artifacts

Status Test Duration
mock-llm-acp-agent.spec.ts › mock-LLM ACP agent conversation › step 1: configure ACP agent via Settings → Agent UI 13.8s
mock-llm-acp-agent.spec.ts › mock-LLM ACP agent conversation › step 2: reload and verify ACP settings are persisted in UI 5.6s
mock-llm-acp-agent.spec.ts › mock-LLM ACP agent conversation › step 3: start ACP conversation and verify agent reply 6.3s
mock-llm-acp-agent.spec.ts › mock-LLM ACP agent conversation › step 4: resume ACP conversation from sidebar after navigating away 5.8s
mock-llm-auth-modes.spec.ts › auth mode: fresh install with runtime-injected key › reaches the onboarding modal without pre-seeded localStorage 1.3s
mock-llm-auth-modes.spec.ts › auth mode: non-public key rotation › recovers when localStorage has a stale session API key 5.3s
mock-llm-auth-modes.spec.ts › auth mode: public gate › shows the auth screen when no key is configured 1.2s
mock-llm-auth-modes.spec.ts › auth mode: public gate › rejects an incorrect key with an inline error 1.4s
mock-llm-auth-modes.spec.ts › auth mode: public gate › allows access after pasting the correct key 1.7s
mock-llm-auth-modes.spec.ts › auth mode: public gate › skips auth screen for returning user with valid stored key 731ms
mock-llm-auth-modes.spec.ts › auth mode: public gate › re-prompts when the server rotates its key (stale localStorage) 1.5s
mock-llm-automation.spec.ts › mock-LLM automation lifecycle › step 1: setup LLM profile and register automation trajectory 7.4s
mock-llm-automation.spec.ts › mock-LLM automation lifecycle › step 2: create automation and dispatch run via the UI 99.9s
⏭️ mock-llm-automation.spec.ts › mock-LLM automation lifecycle › step 3: verify automation and run on the automations page 0ms
mock-llm-conversation.spec.ts › mock-LLM agent-server conversation › step 1: create an LLM profile pointing at the mock LLM server 6.4s
mock-llm-conversation.spec.ts › mock-LLM agent-server conversation › step 2: activate the mock-llm profile and verify settings API 6.2s
mock-llm-conversation.spec.ts › mock-LLM agent-server conversation › step 3: run a conversation with the mock LLM 7.4s
mock-llm-conversation.spec.ts › mock-LLM agent-server conversation › step 4: resume conversation from sidebar after navigating away 5.7s
mock-llm-cross-connect.spec.ts › cross-connect: frontend-only → backend-only › frontend-only connects to a separate backend-only instance 17.8s
mock-llm-cross-connect.spec.ts › cross-connect: frontend-only → multiple backends › connects to two separate backends and switches between them 21.5s
mock-llm-folder-workspace.spec.ts › mock-LLM folder browser → workspace → conversation › step 1: browse to a folder, add it as a workspace, and launch a conversation with the correct working_dir 7.9s
mock-llm-image-upload.spec.ts › mock-LLM image upload › attaching an image embeds it as base64 in the LLM completion call 14.4s
mock-llm-model-switch.spec.ts › mock-LLM /model slash command › step 1: configure LLM, create switch-target profile, register trajectory 13.3s
mock-llm-model-switch.spec.ts › mock-LLM /model slash command › step 2: start conversation, switch profile via /model, verify switch 7.7s
mock-llm-onboarding-happy-path.spec.ts › onboarding happy path › completes the full onboarding flow and launches a conversation 4.5s
mock-llm-onboarding-regressions.spec.ts › onboarding recent regressions › keeps the modal open on backdrop click and Escape 1.4s
mock-llm-onboarding-regressions.spec.ts › onboarding recent regressions › defaults the LLM setup step to OpenAI GPT-5.5 1.7s
mock-llm-partial-stack.spec.ts › partial stack: --frontend-only › serves the frontend but returns 503 for backend routes 7.4s
mock-llm-partial-stack.spec.ts › partial stack: --backend-only › serves backend APIs but returns 503 for the frontend root 14.1s
mock-llm-partial-stack.spec.ts › partial stack: port conflict › fails with a clear error when the ingress port is occupied 126ms
mock-llm-partial-stack.spec.ts › partial stack: port conflict › starts successfully on a free port after a conflict 6.0s
mock-llm-preset-automation.spec.ts › preset automation → slash command conversation › automation card sends the correct slash command to a conversation 15.9s
mock-llm-preset-automation.spec.ts › preset automation → slash command conversation › direct slash command from home page triggers skill activation 13.7s
mock-llm-profile-management.spec.ts › active profile deletion + reconciliation › active profile is deletable and reconciliation activates another profile 8.5s
mock-llm-profile-management.spec.ts › same-model profile identity › chat header shows the correct profile when two profiles share the same model 15.1s
mock-llm-profile-management.spec.ts › OpenHands provider base_url normalization › re-saving an OpenHands profile from Basic view drops proxy base_url 8.1s
mock-llm-skills.spec.ts › skill loading: project, user, and deletion › project skill in workspace/.agents/skills/ triggers on matching keyword 14.5s
mock-llm-skills.spec.ts › skill loading: project, user, and deletion › user skill in ~/.openhands/skills/ triggers on matching keyword 14.7s
mock-llm-skills.spec.ts › skill loading: project, user, and deletion › deleting a user skill removes it from subsequent conversations 13.3s
mock-llm-ui-regressions.spec.ts › UI regressions › scopes standalone styles to the agent-server-ui shell 1.3s
mock-llm-ui-regressions.spec.ts › UI regressions › renders critic results on agent messages and finish actions 1.5s
mock-llm-ui-regressions.spec.ts › UI regressions › loads older events when scrolling up 1.6s
mock-llm-ui-regressions.spec.ts › UI regressions › selected workspace persists after navigating away and returning 1.9s
mock-llm-ui-regressions.spec.ts › UI regressions › cleared sessionStorage yields empty workspace selection 960ms
🔍 Failure details (1)

❌ mock-llm-automation.spec.ts › mock-LLM automation lifecycle › step 2: create automation and dispatch run via the UI

Error: No run with status "COMPLETED" after 90000ms

Posted by the Mock-LLM E2E workflow · results are deterministic (scripted LLM responses)

@github-actions

github-actions Bot commented Jun 9, 2026

Copy link
Copy Markdown
Contributor

🛑 Mock-LLM Docker E2E Test Results

26/35 passed · 2 failed · 7 skipped · ⚠️ 9 not run (process killed at 35/44)

Commit: 691f208d · Workflow run · Test artifacts

Status Test Duration
chromium › mock-llm-acp-agent.spec.ts › mock-LLM ACP agent conversation › step 1: configure ACP agent via Settings → Agent UI 14.1s
chromium › mock-llm-acp-agent.spec.ts › mock-LLM ACP agent conversation › step 2: reload and verify ACP settings are persisted in UI 5.6s
chromium › mock-llm-acp-agent.spec.ts › mock-LLM ACP agent conversation › step 3: start ACP conversation and verify agent reply 6.8s
chromium › mock-llm-acp-agent.spec.ts › mock-LLM ACP agent conversation › step 4: resume ACP conversation from sidebar after navigating away 5.8s
chromium › mock-llm-auth-modes.spec.ts › auth mode: fresh install with runtime-injected key › reaches the onboarding modal without pre-seeded localStorage 1.3s
chromium › mock-llm-auth-modes.spec.ts › auth mode: non-public key rotation › recovers when localStorage has a stale session API key 5.3s
chromium › mock-llm-auth-modes.spec.ts › auth mode: public gate › shows the auth screen when no key is configured 1.2s
chromium › mock-llm-auth-modes.spec.ts › auth mode: public gate › rejects an incorrect key with an inline error 1.4s
chromium › mock-llm-auth-modes.spec.ts › auth mode: public gate › allows access after pasting the correct key 1.4s
chromium › mock-llm-auth-modes.spec.ts › auth mode: public gate › skips auth screen for returning user with valid stored key 755ms
chromium › mock-llm-auth-modes.spec.ts › auth mode: public gate › re-prompts when the server rotates its key (stale localStorage) 1.5s
chromium › mock-llm-automation.spec.ts › mock-LLM automation lifecycle › step 1: setup LLM profile and register automation trajectory 7.5s
chromium › mock-llm-automation.spec.ts › mock-LLM automation lifecycle › step 2: create automation and dispatch run via the UI 99.2s
⏭️ chromium › mock-llm-automation.spec.ts › mock-LLM automation lifecycle › step 3: verify automation and run on the automations page 0ms
chromium › mock-llm-automation.spec.ts › mock-LLM automation lifecycle › step 1: setup LLM profile and register automation trajectory 8.6s
chromium › mock-llm-automation.spec.ts › mock-LLM automation lifecycle › step 2: create automation and dispatch run via the UI 99.8s
⏭️ chromium › mock-llm-automation.spec.ts › mock-LLM automation lifecycle › step 3: verify automation and run on the automations page 0ms
chromium › mock-llm-conversation.spec.ts › mock-LLM agent-server conversation › step 1: create an LLM profile pointing at the mock LLM server 6.6s
chromium › mock-llm-conversation.spec.ts › mock-LLM agent-server conversation › step 2: activate the mock-llm profile and verify settings API 6.4s
chromium › mock-llm-conversation.spec.ts › mock-LLM agent-server conversation › step 3: run a conversation with the mock LLM 6.4s
chromium › mock-llm-conversation.spec.ts › mock-LLM agent-server conversation › step 4: resume conversation from sidebar after navigating away 5.9s
⏭️ chromium › mock-llm-cross-connect.spec.ts › cross-connect: frontend-only → backend-only › frontend-only connects to a separate backend-only instance 230ms
⏭️ chromium › mock-llm-cross-connect.spec.ts › cross-connect: frontend-only → multiple backends › connects to two separate backends and switches between them 200ms
chromium › mock-llm-folder-workspace.spec.ts › mock-LLM folder browser → workspace → conversation › step 1: browse to a folder, add it as a workspace, and launch a conversation with the correct working_dir 8.3s
chromium › mock-llm-image-upload.spec.ts › mock-LLM image upload › attaching an image embeds it as base64 in the LLM completion call 13.7s
chromium › mock-llm-model-switch.spec.ts › mock-LLM /model slash command › step 1: configure LLM, create switch-target profile, register trajectory 13.7s
chromium › mock-llm-model-switch.spec.ts › mock-LLM /model slash command › step 2: start conversation, switch profile via /model, verify switch 7.8s
chromium › mock-llm-onboarding-happy-path.spec.ts › onboarding happy path › completes the full onboarding flow and launches a conversation 3.7s
chromium › mock-llm-onboarding-regressions.spec.ts › onboarding recent regressions › keeps the modal open on backdrop click and Escape 1.4s
chromium › mock-llm-onboarding-regressions.spec.ts › onboarding recent regressions › defaults the LLM setup step to OpenAI GPT-5.5 1.7s
⏭️ chromium › mock-llm-partial-stack.spec.ts › partial stack: --frontend-only › serves the frontend but returns 503 for backend routes 210ms
chromium › mock-llm-partial-stack.spec.ts › partial stack: --backend-only › serves backend APIs but returns 503 for the frontend root 28.2s
⏭️ chromium › mock-llm-partial-stack.spec.ts › partial stack: port conflict › fails with a clear error when the ingress port is occupied 1ms
⏭️ chromium › mock-llm-partial-stack.spec.ts › partial stack: port conflict › starts successfully on a free port after a conflict 2ms
chromium › mock-llm-preset-automation.spec.ts › preset automation → slash command conversation › automation card sends the correct slash command to a conversation 16.9s
🔍 Failure details (2)

❌ chromium › mock-llm-automation.spec.ts › mock-LLM automation lifecycle › step 2: create automation and dispatch run via the UI

Error: No run with status "COMPLETED" after 90000ms

❌ chromium › mock-llm-automation.spec.ts › mock-LLM automation lifecycle › step 2: create automation and dispatch run via the UI

Error: No run with status "COMPLETED" after 90000ms

Posted by the Mock-LLM E2E workflow · results are deterministic (scripted LLM responses)

@github-actions

Copy link
Copy Markdown
Contributor

❌ Mock-LLM E2E Tests

52/54 passed · 1 failed · 1 skipped

Commit: b36736a1 · Workflow run · Test artifacts

Status Test Duration
mock-llm-acp-agent.spec.ts › mock-LLM ACP agent conversation › step 1: configure ACP agent via Settings → Agent UI 13.6s
mock-llm-acp-agent.spec.ts › mock-LLM ACP agent conversation › step 2: reload and verify ACP settings are persisted in UI 5.5s
mock-llm-acp-agent.spec.ts › mock-LLM ACP agent conversation › step 3: start ACP conversation and verify agent reply 6.2s
mock-llm-acp-agent.spec.ts › mock-LLM ACP agent conversation › step 4: resume ACP conversation from sidebar after navigating away 5.7s
mock-llm-auth-modes.spec.ts › auth mode: fresh install with runtime-injected key › reaches the onboarding modal without pre-seeded localStorage 1.3s
mock-llm-auth-modes.spec.ts › auth mode: non-public key rotation › recovers when localStorage has a stale session API key 5.3s
mock-llm-auth-modes.spec.ts › auth mode: public gate › shows the auth screen when no key is configured 1.2s
mock-llm-auth-modes.spec.ts › auth mode: public gate › rejects an incorrect key with an inline error 1.3s
mock-llm-auth-modes.spec.ts › auth mode: public gate › allows access after pasting the correct key 1.6s
mock-llm-auth-modes.spec.ts › auth mode: public gate › skips auth screen for returning user with valid stored key 710ms
mock-llm-auth-modes.spec.ts › auth mode: public gate › re-prompts when the server rotates its key (stale localStorage) 1.4s
mock-llm-automation.spec.ts › mock-LLM automation lifecycle › step 1: setup LLM profile and register automation trajectory 7.0s
mock-llm-automation.spec.ts › mock-LLM automation lifecycle › step 2: create automation and dispatch run via the UI 99.8s
⏭️ mock-llm-automation.spec.ts › mock-LLM automation lifecycle › step 3: verify automation and run on the automations page 0ms
mock-llm-conversation.spec.ts › mock-LLM agent-server conversation › step 1: create an LLM profile pointing at the mock LLM server 6.3s
mock-llm-conversation.spec.ts › mock-LLM agent-server conversation › step 2: activate the mock-llm profile and verify settings API 6.2s
mock-llm-conversation.spec.ts › mock-LLM agent-server conversation › step 3: run a conversation with the mock LLM 6.5s
mock-llm-conversation.spec.ts › mock-LLM agent-server conversation › step 4: resume conversation from sidebar after navigating away 5.7s
mock-llm-cross-connect.spec.ts › cross-connect: frontend-only → backend-only › frontend-only connects to a separate backend-only instance 15.8s
mock-llm-cross-connect.spec.ts › cross-connect: frontend-only → multiple backends › connects to two separate backends and switches between them 19.6s
mock-llm-files-and-git.spec.ts › files tab, git control bar, and browser tab › step 1: ensure mock LLM profile is configured 205ms
mock-llm-files-and-git.spec.ts › files tab, git control bar, and browser tab › step 2: start conversation and attach workspace metadata 11.6s
mock-llm-files-and-git.spec.ts › files tab, git control bar, and browser tab › step 3: git control bar shows workspace pill and git actions 25.3s
mock-llm-files-and-git.spec.ts › files tab, git control bar, and browser tab › step 4: files tab defaults to diff view for attached workspace 5.9s
mock-llm-files-and-git.spec.ts › files tab, git control bar, and browser tab › step 5: browser tab shows empty state 6.2s
mock-llm-files-and-git.spec.ts › files tab, git control bar, and browser tab › step 6: files tab defaults to file-tree view without attached workspace 7.4s
mock-llm-folder-workspace.spec.ts › mock-LLM folder browser → workspace → conversation › step 1: browse to a folder, add it as a workspace, and launch a conversation with the correct working_dir 7.7s
mock-llm-image-upload.spec.ts › mock-LLM image upload › attaching an image embeds it as base64 in the LLM completion call 13.3s
mock-llm-mcp-github.spec.ts › MCP GitHub server install flow › step 1: GitHub card is visible on the MCP marketplace page 5.6s
mock-llm-mcp-github.spec.ts › MCP GitHub server install flow › step 2: clicking GitHub card opens the install modal with correct fields 5.7s
mock-llm-mcp-github.spec.ts › MCP GitHub server install flow › step 3: full install flow — fill PAT, submit, verify installed 12.6s
mock-llm-mcp-github.spec.ts › MCP GitHub server install flow › step 4: installed GitHub server can be deleted 5.8s
mock-llm-model-switch.spec.ts › mock-LLM /model slash command › step 1: configure LLM, create switch-target profile, register trajectory 12.9s
mock-llm-model-switch.spec.ts › mock-LLM /model slash command › step 2: start conversation, switch profile via /model, verify switch 6.8s
mock-llm-onboarding-happy-path.spec.ts › onboarding happy path › completes the full onboarding flow and launches a conversation 3.6s
mock-llm-onboarding-regressions.spec.ts › onboarding recent regressions › keeps the modal open on backdrop click and Escape 1.3s
mock-llm-onboarding-regressions.spec.ts › onboarding recent regressions › defaults the LLM setup step to OpenAI GPT-5.5 1.6s
mock-llm-partial-stack.spec.ts › partial stack: --frontend-only › serves the frontend but returns 503 for backend routes 7.3s
mock-llm-partial-stack.spec.ts › partial stack: --backend-only › serves backend APIs but returns 503 for the frontend root 13.1s
mock-llm-partial-stack.spec.ts › partial stack: port conflict › fails with a clear error when the ingress port is occupied 103ms
mock-llm-partial-stack.spec.ts › partial stack: port conflict › starts successfully on a free port after a conflict 6.0s
mock-llm-preset-automation.spec.ts › preset automation → slash command conversation › automation card sends the correct slash command to a conversation 15.8s
mock-llm-preset-automation.spec.ts › preset automation → slash command conversation › direct slash command from home page triggers skill activation 13.5s
mock-llm-profile-management.spec.ts › active profile deletion + reconciliation › active profile is deletable and reconciliation activates another profile 8.4s
mock-llm-profile-management.spec.ts › same-model profile identity › chat header shows the correct profile when two profiles share the same model 14.9s
mock-llm-profile-management.spec.ts › OpenHands provider base_url normalization › re-saving an OpenHands profile from Basic view drops proxy base_url 8.0s
mock-llm-skills.spec.ts › skill loading: project, user, and deletion › project skill in workspace/.agents/skills/ triggers on matching keyword 13.5s
mock-llm-skills.spec.ts › skill loading: project, user, and deletion › user skill in ~/.openhands/skills/ triggers on matching keyword 13.4s
mock-llm-skills.spec.ts › skill loading: project, user, and deletion › deleting a user skill removes it from subsequent conversations 13.3s
mock-llm-ui-regressions.spec.ts › UI regressions › scopes standalone styles to the agent-server-ui shell 924ms
mock-llm-ui-regressions.spec.ts › UI regressions › renders critic results on agent messages and finish actions 1.4s
mock-llm-ui-regressions.spec.ts › UI regressions › loads older events when scrolling up 1.6s
mock-llm-ui-regressions.spec.ts › UI regressions › selected workspace persists after navigating away and returning 2.3s
mock-llm-ui-regressions.spec.ts › UI regressions › cleared sessionStorage yields empty workspace selection 890ms
🔍 Failure details (1)

❌ mock-llm-automation.spec.ts › mock-LLM automation lifecycle › step 2: create automation and dispatch run via the UI

Error: No run with status "COMPLETED" after 90000ms

Posted by the Mock-LLM E2E workflow · results are deterministic (scripted LLM responses)

github-actions Bot added a commit that referenced this pull request Jun 11, 2026
@github-actions

Copy link
Copy Markdown
Contributor

📸 Snapshot Test Report

Warning

Snapshot comparison step crashed (timeout, OOM, or runner error) — diff results below may be incomplete or absent.
Check the CI logs for the full error output (look for the "Run snapshot comparison" step).

Warning

One or more snapshot tests crashed during generation — some snapshots below may be incomplete.
Check the CI logs for the full error output (look for the "Generate current PR snapshots" step).

❌ 3 snapshots differ from the main branch baselines. Add the update-snapshots label to acknowledge intentional changes.

Category Count
🔴 Changed 3
🆕 New 0
✅ Unchanged 71
Total 74

How to resolve:

  • Unintentional diffs — the baselines on main may have moved since this branch was created. Merge the latest main into this branch and re-run CI.
  • Intentional changes — add the update-snapshots label. CI will pass and the new screenshots become the baseline when this PR merges.
🔴 Changed snapshots (3)

changes-tab

changes-empty

Expected (main) Actual (PR) Diff
expected actual diff

settings-page — 2 snapshots

analytics-consent-modal

Expected (main) Actual (PR) Diff
expected actual diff

settings-app-page

Expected (main) Actual (PR) Diff
expected actual diff
✅ Unchanged snapshots (71)

archived-conversation

  • conversation-panel-with-archived-badges
  • conversation-view-archived
  • conversation-view-sandbox-error

automations

  • automations-delete-modal
  • automations-list-active-inactive
  • automations-no-automations
  • automations-search-no-results

backends-extended

  • backend-add-blank-disabled
  • backend-add-cloud-advanced-open
  • backend-add-cloud-no-key-disabled
  • backend-add-cloud-with-key-enabled
  • backend-add-form-partially-filled
  • backend-add-invalid-url-disabled
  • backend-add-local-ready
  • backend-add-name-only-disabled
  • backend-add-two-column-layout
  • backend-add-whitespace-host-disabled
  • backend-after-switch
  • backend-cancel-nothing-saved
  • backend-dropdown-two-backends
  • backend-edit-prefilled
  • backend-manage-after-removal
  • backend-manage-two-listed
  • backend-remove-cancelled
  • backend-remove-confirmation
  • backend-switch-overlay

backends

  • backend-add-modal
  • backend-manage-modal
  • backend-selector-open

changes-tab

  • changes-deleted-file
  • changes-diff-viewer

collapsible-thinking

  • reasoning-content-collapsed
  • reasoning-content-expanded
  • think-action-collapsed
  • think-action-expanded

mcp-page

  • mcp-custom-server-1-editor-open
  • mcp-custom-server-2-url-filled
  • mcp-custom-server-3-all-filled
  • mcp-custom-server-4-installed
  • mcp-custom-server-editor
  • mcp-empty-installed
  • mcp-search-filtered
  • mcp-slack-install-1-marketplace
  • mcp-slack-install-2-modal
  • mcp-slack-install-3-filled
  • mcp-slack-install-4-installed

onboarding

  • onboarding-step-0-check-backend
  • onboarding-step-1-choose-agent
  • onboarding-step-2-setup-llm
  • onboarding-step-3-say-hello

projects-workspace-browser

  • projects-workspace-browser

settings-page

  • add-backend-modal
  • home-screen
  • settings-page

settings-secrets

  • secrets-add-form-filled
  • secrets-add-form
  • secrets-after-save
  • secrets-delete-confirm
  • secrets-list

settings-verification

  • condenser-settings
  • verification-settings-critic-enabled
  • verification-settings-off
  • verification-settings-on

sidebar

  • sidebar-collapsed
  • sidebar-conversation-panel
  • sidebar-filter-menu

skills-page

  • skills-empty
  • skills-loaded
  • skills-no-match
  • skills-search-filtered
  • skills-type-filter

Generated by the Snapshot Tests workflow. This comment was created by an AI agent (OpenHands) on behalf of the repo maintainers.

@github-actions

Copy link
Copy Markdown
Contributor

🛑 Mock-LLM Docker E2E Test Results

24/30 passed · 2 failed · 4 skipped · ⚠️ 24 not run (process killed at 30/54)

Commit: b36736a1 · Workflow run · Test artifacts

Status Test Duration
chromium › mock-llm-acp-agent.spec.ts › mock-LLM ACP agent conversation › step 1: configure ACP agent via Settings → Agent UI 13.9s
chromium › mock-llm-acp-agent.spec.ts › mock-LLM ACP agent conversation › step 2: reload and verify ACP settings are persisted in UI 5.5s
chromium › mock-llm-acp-agent.spec.ts › mock-LLM ACP agent conversation › step 3: start ACP conversation and verify agent reply 6.7s
chromium › mock-llm-acp-agent.spec.ts › mock-LLM ACP agent conversation › step 4: resume ACP conversation from sidebar after navigating away 5.7s
chromium › mock-llm-auth-modes.spec.ts › auth mode: fresh install with runtime-injected key › reaches the onboarding modal without pre-seeded localStorage 1.3s
chromium › mock-llm-auth-modes.spec.ts › auth mode: non-public key rotation › recovers when localStorage has a stale session API key 5.3s
chromium › mock-llm-auth-modes.spec.ts › auth mode: public gate › shows the auth screen when no key is configured 1.2s
chromium › mock-llm-auth-modes.spec.ts › auth mode: public gate › rejects an incorrect key with an inline error 1.3s
chromium › mock-llm-auth-modes.spec.ts › auth mode: public gate › allows access after pasting the correct key 1.6s
chromium › mock-llm-auth-modes.spec.ts › auth mode: public gate › skips auth screen for returning user with valid stored key 739ms
chromium › mock-llm-auth-modes.spec.ts › auth mode: public gate › re-prompts when the server rotates its key (stale localStorage) 1.5s
chromium › mock-llm-automation.spec.ts › mock-LLM automation lifecycle › step 1: setup LLM profile and register automation trajectory 7.1s
chromium › mock-llm-automation.spec.ts › mock-LLM automation lifecycle › step 2: create automation and dispatch run via the UI 98.9s
⏭️ chromium › mock-llm-automation.spec.ts › mock-LLM automation lifecycle › step 3: verify automation and run on the automations page 0ms
chromium › mock-llm-automation.spec.ts › mock-LLM automation lifecycle › step 1: setup LLM profile and register automation trajectory 8.0s
chromium › mock-llm-automation.spec.ts › mock-LLM automation lifecycle › step 2: create automation and dispatch run via the UI 99.4s
⏭️ chromium › mock-llm-automation.spec.ts › mock-LLM automation lifecycle › step 3: verify automation and run on the automations page 0ms
chromium › mock-llm-conversation.spec.ts › mock-LLM agent-server conversation › step 1: create an LLM profile pointing at the mock LLM server 6.4s
chromium › mock-llm-conversation.spec.ts › mock-LLM agent-server conversation › step 2: activate the mock-llm profile and verify settings API 6.2s
chromium › mock-llm-conversation.spec.ts › mock-LLM agent-server conversation › step 3: run a conversation with the mock LLM 6.3s
chromium › mock-llm-conversation.spec.ts › mock-LLM agent-server conversation › step 4: resume conversation from sidebar after navigating away 5.7s
⏭️ chromium › mock-llm-cross-connect.spec.ts › cross-connect: frontend-only → backend-only › frontend-only connects to a separate backend-only instance 190ms
⏭️ chromium › mock-llm-cross-connect.spec.ts › cross-connect: frontend-only → multiple backends › connects to two separate backends and switches between them 189ms
chromium › mock-llm-files-and-git.spec.ts › files tab, git control bar, and browser tab › step 1: ensure mock LLM profile is configured 201ms
chromium › mock-llm-files-and-git.spec.ts › files tab, git control bar, and browser tab › step 2: start conversation and attach workspace metadata 11.4s
chromium › mock-llm-files-and-git.spec.ts › files tab, git control bar, and browser tab › step 3: git control bar shows workspace pill and git actions 25.3s
chromium › mock-llm-files-and-git.spec.ts › files tab, git control bar, and browser tab › step 4: files tab defaults to diff view for attached workspace 5.9s
chromium › mock-llm-files-and-git.spec.ts › files tab, git control bar, and browser tab › step 5: browser tab shows empty state 6.3s
chromium › mock-llm-files-and-git.spec.ts › files tab, git control bar, and browser tab › step 6: files tab defaults to file-tree view without attached workspace 7.2s
chromium › mock-llm-folder-workspace.spec.ts › mock-LLM folder browser → workspace → conversation › step 1: browse to a folder, add it as a workspace, and launch a conversation with the correct working_dir 7.9s
🔍 Failure details (2)

❌ chromium › mock-llm-automation.spec.ts › mock-LLM automation lifecycle › step 2: create automation and dispatch run via the UI

Error: No run with status "COMPLETED" after 90000ms

❌ chromium › mock-llm-automation.spec.ts › mock-LLM automation lifecycle › step 2: create automation and dispatch run via the UI

Error: No run with status "COMPLETED" after 90000ms

Posted by the Mock-LLM E2E workflow · results are deterministic (scripted LLM responses)

@enyst

enyst commented Jun 12, 2026

Copy link
Copy Markdown
Member Author

@OpenHands what PRs and what exactly affected us, caused the merge conflicts with main?

Respond on this pr as a comment

@openhands-ai

openhands-ai Bot commented Jun 12, 2026

Copy link
Copy Markdown

I'm on it! enyst can track my progress at all-hands.dev

enyst commented Jun 12, 2026

Copy link
Copy Markdown
Member Author

I reproduced the merge against current origin/main with git merge-tree --write-tree HEAD origin/main. Short version: the conflicts are from two main-branch PRs, with one historical/revert wrinkle:

  1. chore: align package version with npm RC #1303chore: align package version with npm RC

    • Affected us in config/defaults.json by moving versions.agentCanvas to the current RC line (1.0.0-rc.6).
    • Our branch still had the older value (1.0.0-rc.3) because it also touched the same versions object while bumping the agent-server pin.
  2. chore: bump agent-server → 1.28.1, automation → 1.0.0a9, extensions → 0.4.1 #1319chore: bump agent-server → 1.28.1, automation → 1.0.0a9, extensions → 0.4.1

Actual textual conflict files from merge-tree:

  • AGENTS.md
  • __tests__/scripts/dev-safe.test.ts
  • config/defaults.json
  • scripts/check-sdk-version-sync.mjs
  • scripts/dev-safe.mjs
  • src/components/features/settings/llm-profiles/llm-settings-local-view.tsx
  • tests/e2e/mock-llm/mock-llm-profile-management.spec.ts

No other merged PR materially caused the conflicts. #1263 and #1297 are also between our merge-base and main, but they do not touch the conflicting OpenHands-provider/version areas; any overlap there auto-merged.

Resolution direction I would take: keep main's current version pins/docs (1.28.1, 1.0.0a9, 1.28.1, rc.6), and resolve the LLM-profile code/tests according to this PR's intended end-state: Canvas should stop re-stamping the LiteLLM proxy base_url for the public OpenHands provider path, while keeping only whatever transitional test tolerance we still need until the SDK-side migration is fully released.

This PR comment was created by an AI agent (OpenHands) on behalf of the user.

@openhands-ai

openhands-ai Bot commented Jun 12, 2026

Copy link
Copy Markdown

Since the last summary, I completed the requested PR follow-up by posting a comment on PR #1280.

Summary:

No code changes were made, so there are no extraneous repository changes to revert.

@enyst

enyst commented Jun 13, 2026

Copy link
Copy Markdown
Member Author

@OpenHands There are many conflicts with main in this branch now. Look on main history from ~24h or so and find why, was there a PR doing anything similar or? Respond directly here as a comment.

@openhands-ai

openhands-ai Bot commented Jun 13, 2026

Copy link
Copy Markdown

I'm on it! enyst can track my progress at all-hands.dev

enyst commented Jun 13, 2026

Copy link
Copy Markdown
Member Author

I checked main history since this branch's current merge base (a0e69751, branch is currently 15 ahead / 23 behind origin/main). Yes: there was a very similar PR merged into main:

  • Remove obsolete OpenHands proxy base URL handling #1321, "Remove obsolete OpenHands proxy base URL handling", merged as c5b05b54 on 2026-06-12 00:40 UTC.
    • It overlaps this PR in llm-settings-local-view.tsx, llm-settings-local-view.test.tsx, src/routes/llm-settings.tsx, src/utils/openhands-llm.ts, __tests__/utils/openhands-llm.test.ts, and tests/e2e/mock-llm/mock-llm-profile-management.spec.ts.
    • It removed the Basic-mode profile-save logic that preserved/injected the OpenHands LiteLLM proxy base_url, plus the obsolete proxy-base-url E2E expectation.
    • It was a follow-up to the agent-server 1.28.1 bump (chore: bump agent-server → 1.28.1, automation → 1.0.0a9, extensions → 0.4.1 #1319) and assumes the SDK/server owns proxy base_url behavior.

That said, #1321 is narrower than this PR. main still has the normalizeDisplayModel() path that reverse-maps litellm_proxy/* + the All-Hands proxy URL back to openhands/* for display, and still has OPENHANDS_LLM_PROXY_BASE_URL / isOpenHandsProxyBaseUrl. This PR goes further by trusting the public/stored openhands/* model namespace directly and removing that reverse-display compatibility layer. So I would treat #1321 as partially superseding this PR, not fully replacing it.

The other conflict sources are mostly unrelated recent main churn:

I also ran a merge-tree check; the actual content/modify-delete conflicts are concentrated in the LLM profile/settings files, src/utils/openhands-llm.ts, the profile-management E2E, version/config scripts, and AGENTS.md. For conflict resolution, I would take main's #1321 and version-bump cleanup as already landed, drop this branch's temporary SDK pin commits, keep #744's subscription UI additions, and reapply only the remaining public-openhands/* end-state pieces from this PR.

This comment was created by an AI agent (OpenHands) on behalf of the user.

@openhands-ai

This comment was marked as duplicate.

Co-authored-by: openhands <openhands@all-hands.dev>
@enyst enyst force-pushed the openhands/public-openhands-provider branch from b36736a to 7fcfb8f Compare June 13, 2026 01:46

enyst commented Jun 13, 2026

Copy link
Copy Markdown
Member Author

Done — I rebuilt this branch on current main and force-pushed the cleaned version as 7fcfb8fa.

What changed in the cleanup:

  • Dropped the temporary SDK/agent-server pin commits and kept main's current version/default config.
  • Kept the overlapping Remove obsolete OpenHands proxy base URL handling #1321 cleanup as already landed.
  • Preserved Add ChatGPT subscription LLM support #744's ChatGPT subscription LLM settings UI from main.
  • Reapplied only the remaining public-openhands/* end-state behavior: removed frontend reverse-display mapping / OpenHands proxy utility code, initialized the model selector directly from the SDK/API model, and kept Basic-mode saves from re-stamping proxy base_url.
  • Updated unit and mock-LLM E2E coverage for the public OpenHands provider behavior.

Verified locally:

npm test -- --run __tests__/components/modals/settings/model-selector-openhands.test.tsx __tests__/components/settings/llm-profiles/llm-settings-local-view.test.tsx __tests__/routes/llm-settings.test.tsx
npm run typecheck
npx eslint src/components/shared/modals/settings/model-selector.tsx src/components/shared/modals/settings/settings-form.tsx src/routes/llm-settings.tsx tests/e2e/mock-llm/mock-llm-profile-management.spec.ts
npm run build

This comment was created by an AI agent (OpenHands) on behalf of the user.

@enyst enyst removed review-this qa-this Trigger OpenHands QA validation labels Jun 13, 2026
@enyst enyst added review-this qa-this Trigger OpenHands QA validation labels Jun 13, 2026 — with OpenHands AI

all-hands-bot commented Jun 13, 2026

Copy link
Copy Markdown
Contributor

Review complete.

This review was performed through OpenHands Cloud Automation. You can log in and view the conversation here.

@all-hands-bot all-hands-bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review: settings: trust public OpenHands provider models

🟢 Good taste — Clean, well-scoped refactor that removes unnecessary complexity.

Summary

This PR removes the frontend "reverse-mapping" layer that was translating litellm_proxy/<model> back to openhands/<model> for display. This translation layer was a workaround for the old SDK behavior that stored OpenHands models in litellm_proxy/ format. Now that the SDK keeps openhands/ as the public namespace (per SDK PR #3548), the frontend no longer needs this transformation.

Analysis

[REMOVED CODE]

  • src/utils/normalize-display-model.ts — The reverse-mapping utility that is no longer needed
  • src/utils/openhands-llm.ts — The proxy base URL helper (now handled by SDK)
  • currentBaseUrl prop from ModelSelector — Removed the workaround parameter

[PRESERVED BEHAVIOR]

  • The ModelSelector component now directly uses the model ID without transformation
  • The bootstrap effect is simplified (no longer needs to wait for openhandsVerifiedModels)
  • The test counts (providersCount, verifiedCount, modelsCount) still verify no redundant API calls

[TEST IMPROVEMENTS]

  • Unit tests are updated to use the new openhands/<model> format
  • New e2e test covers the full save flow confirming base_url is dropped for OpenHands models
  • Test helper getProfileConfig() is a good addition for verifying persisted state

Existing Review Comments

I reviewed the existing comments and the author's responses:

  1. Line 72 (defensive handling for extractModelAndProvider): Author's response "I think it's okay and can be addressed later" is reasonable — the function already handles empty/null cases
  2. Line 52 (test assertions): Author's explanation that semantics haven't changed is correct

Dependency Note

This PR references SDK PR #3548. Ensure that SDK PR is merged and the updated SDK is deployed before merging this PR, as the frontend changes depend on the new SDK behavior.


[RISK ASSESSMENT]

  • ⚠️ Risk Assessment: 🟢 LOW

This is a pure refactor that removes code rather than adding it. The behavior change (dropping base_url for OpenHands models) is the desired outcome aligned with the SDK migration. Test coverage is comprehensive.

Recommendation: Safe to merge once SDK PR #3548 is deployed.


Worth merging: Clean architectural simplification that removes a workaround now handled by the SDK.

KEY INSIGHT: The frontend no longer needs to reverse-translate model IDs because the SDK now stores the public openhands/<model> namespace directly.


This review was generated by an AI agent (OpenHands) on behalf of the user through OpenHands Automation. View conversation

@github-actions

Copy link
Copy Markdown
Contributor

✅ Mock-LLM E2E Tests

54/54 passed

Commit: 7fcfb8fa · Workflow run · Test artifacts

Status Test Duration
mock-llm-acp-agent.spec.ts › mock-LLM ACP agent conversation › step 1: configure ACP agent via Settings → Agent UI 13.9s
mock-llm-acp-agent.spec.ts › mock-LLM ACP agent conversation › step 2: reload and verify ACP settings are persisted in UI 5.6s
mock-llm-acp-agent.spec.ts › mock-LLM ACP agent conversation › step 3: start ACP conversation and verify agent reply 6.8s
mock-llm-acp-agent.spec.ts › mock-LLM ACP agent conversation › step 4: resume ACP conversation from sidebar after navigating away 5.9s
mock-llm-auth-modes.spec.ts › auth mode: fresh install with runtime-injected key › reaches the onboarding modal without pre-seeded localStorage 1.3s
mock-llm-auth-modes.spec.ts › auth mode: non-public key rotation › recovers when localStorage has a stale session API key 5.3s
mock-llm-auth-modes.spec.ts › auth mode: public gate › shows the auth screen when no key is configured 1.2s
mock-llm-auth-modes.spec.ts › auth mode: public gate › rejects an incorrect key with an inline error 1.4s
mock-llm-auth-modes.spec.ts › auth mode: public gate › allows access after pasting the correct key 1.4s
mock-llm-auth-modes.spec.ts › auth mode: public gate › skips auth screen for returning user with valid stored key 756ms
mock-llm-auth-modes.spec.ts › auth mode: public gate › re-prompts when the server rotates its key (stale localStorage) 1.4s
mock-llm-automation.spec.ts › mock-LLM automation lifecycle › step 1: setup LLM profile and register automation trajectory 7.2s
mock-llm-automation.spec.ts › mock-LLM automation lifecycle › step 2: create automation and dispatch run via the UI 29.5s
mock-llm-automation.spec.ts › mock-LLM automation lifecycle › step 3: verify automation and run on the automations page 6.1s
mock-llm-conversation.spec.ts › mock-LLM agent-server conversation › step 1: create an LLM profile pointing at the mock LLM server 6.2s
mock-llm-conversation.spec.ts › mock-LLM agent-server conversation › step 2: activate the mock-llm profile and verify settings API 7.9s
mock-llm-conversation.spec.ts › mock-LLM agent-server conversation › step 3: run a conversation with the mock LLM 6.6s
mock-llm-conversation.spec.ts › mock-LLM agent-server conversation › step 4: resume conversation from sidebar after navigating away 5.9s
mock-llm-cross-connect.spec.ts › cross-connect: frontend-only → backend-only › frontend-only connects to a separate backend-only instance 15.8s
mock-llm-cross-connect.spec.ts › cross-connect: frontend-only → multiple backends › connects to two separate backends and switches between them 19.7s
mock-llm-files-and-git.spec.ts › files tab, git control bar, and browser tab › step 1: ensure mock LLM profile is configured 166ms
mock-llm-files-and-git.spec.ts › files tab, git control bar, and browser tab › step 2: start conversation and attach workspace metadata 11.5s
mock-llm-files-and-git.spec.ts › files tab, git control bar, and browser tab › step 3: git control bar shows workspace pill and git actions 25.3s
mock-llm-files-and-git.spec.ts › files tab, git control bar, and browser tab › step 4: files tab defaults to diff view for attached workspace 5.8s
mock-llm-files-and-git.spec.ts › files tab, git control bar, and browser tab › step 5: browser tab shows empty state 6.3s
mock-llm-files-and-git.spec.ts › files tab, git control bar, and browser tab › step 6: files tab defaults to file-tree view without attached workspace 7.4s
mock-llm-folder-workspace.spec.ts › mock-LLM folder browser → workspace → conversation › step 1: browse to a folder, add it as a workspace, and launch a conversation with the correct working_dir 8.7s
mock-llm-image-upload.spec.ts › mock-LLM image upload › attaching an image embeds it as base64 in the LLM completion call 13.6s
mock-llm-mcp-github.spec.ts › MCP GitHub server install flow › step 1: GitHub card is visible on the MCP marketplace page 5.5s
mock-llm-mcp-github.spec.ts › MCP GitHub server install flow › step 2: clicking GitHub card opens the install modal with correct fields 5.7s
mock-llm-mcp-github.spec.ts › MCP GitHub server install flow › step 3: full install flow — fill PAT, submit, verify installed 13.3s
mock-llm-mcp-github.spec.ts › MCP GitHub server install flow › step 4: installed GitHub server can be deleted 5.8s
mock-llm-model-switch.spec.ts › mock-LLM /model slash command › step 1: configure LLM, create switch-target profile, register trajectory 13.3s
mock-llm-model-switch.spec.ts › mock-LLM /model slash command › step 2: start conversation, switch profile via /model, verify switch 7.7s
mock-llm-onboarding-happy-path.spec.ts › onboarding happy path › completes the full onboarding flow and launches a conversation 4.6s
mock-llm-onboarding-regressions.spec.ts › onboarding recent regressions › keeps the modal open on backdrop click and Escape 1.4s
mock-llm-onboarding-regressions.spec.ts › onboarding recent regressions › defaults the LLM setup step to OpenAI GPT-5.5 1.6s
mock-llm-partial-stack.spec.ts › partial stack: --frontend-only › serves the frontend but returns 503 for backend routes 7.3s
mock-llm-partial-stack.spec.ts › partial stack: --backend-only › serves backend APIs but returns 503 for the frontend root 14.1s
mock-llm-partial-stack.spec.ts › partial stack: port conflict › fails with a clear error when the ingress port is occupied 104ms
mock-llm-partial-stack.spec.ts › partial stack: port conflict › starts successfully on a free port after a conflict 6.0s
mock-llm-preset-automation.spec.ts › preset automation → slash command conversation › automation card sends the correct slash command to a conversation 16.9s
mock-llm-preset-automation.spec.ts › preset automation → slash command conversation › direct slash command from home page triggers skill activation 13.5s
mock-llm-profile-management.spec.ts › active profile deletion + reconciliation › active profile is deletable and reconciliation activates another profile 8.5s
mock-llm-profile-management.spec.ts › same-model profile identity › chat header shows the correct profile when two profiles share the same model 15.1s
mock-llm-profile-management.spec.ts › OpenHands provider base_url normalization › re-saving an OpenHands profile from Basic view drops proxy base_url 8.2s
mock-llm-skills.spec.ts › skill loading: project, user, and deletion › project skill in workspace/.agents/skills/ triggers on matching keyword 13.6s
mock-llm-skills.spec.ts › skill loading: project, user, and deletion › user skill in ~/.openhands/skills/ triggers on matching keyword 13.7s
mock-llm-skills.spec.ts › skill loading: project, user, and deletion › deleting a user skill removes it from subsequent conversations 13.6s
mock-llm-ui-regressions.spec.ts › UI regressions › scopes standalone styles to the agent-server-ui shell 1.4s
mock-llm-ui-regressions.spec.ts › UI regressions › renders critic results on agent messages and finish actions 1.5s
mock-llm-ui-regressions.spec.ts › UI regressions › loads older events when scrolling up 1.7s
mock-llm-ui-regressions.spec.ts › UI regressions › selected workspace persists after navigating away and returning 2.3s
mock-llm-ui-regressions.spec.ts › UI regressions › cleared sessionStorage yields empty workspace selection 1.1s

Posted by the Mock-LLM E2E workflow · results are deterministic (scripted LLM responses)

@enyst enyst merged commit 8779380 into main Jun 13, 2026
18 of 19 checks passed
@enyst enyst deleted the openhands/public-openhands-provider branch June 13, 2026 01:56
@github-actions

Copy link
Copy Markdown
Contributor

🔶 Mock-LLM Docker E2E Test Results

49/54 passed · 5 skipped

Commit: 7fcfb8fa · Workflow run · Test artifacts

Status Test Duration
mock-llm-acp-agent.spec.ts › mock-LLM ACP agent conversation › step 1: configure ACP agent via Settings → Agent UI 13.9s
mock-llm-acp-agent.spec.ts › mock-LLM ACP agent conversation › step 2: reload and verify ACP settings are persisted in UI 5.5s
mock-llm-acp-agent.spec.ts › mock-LLM ACP agent conversation › step 3: start ACP conversation and verify agent reply 6.7s
mock-llm-acp-agent.spec.ts › mock-LLM ACP agent conversation › step 4: resume ACP conversation from sidebar after navigating away 5.7s
mock-llm-auth-modes.spec.ts › auth mode: fresh install with runtime-injected key › reaches the onboarding modal without pre-seeded localStorage 1.3s
mock-llm-auth-modes.spec.ts › auth mode: non-public key rotation › recovers when localStorage has a stale session API key 5.3s
mock-llm-auth-modes.spec.ts › auth mode: public gate › shows the auth screen when no key is configured 1.2s
mock-llm-auth-modes.spec.ts › auth mode: public gate › rejects an incorrect key with an inline error 1.4s
mock-llm-auth-modes.spec.ts › auth mode: public gate › allows access after pasting the correct key 1.8s
mock-llm-auth-modes.spec.ts › auth mode: public gate › skips auth screen for returning user with valid stored key 779ms
mock-llm-auth-modes.spec.ts › auth mode: public gate › re-prompts when the server rotates its key (stale localStorage) 1.5s
mock-llm-automation.spec.ts › mock-LLM automation lifecycle › step 1: setup LLM profile and register automation trajectory 8.2s
mock-llm-automation.spec.ts › mock-LLM automation lifecycle › step 2: create automation and dispatch run via the UI 32.4s
mock-llm-automation.spec.ts › mock-LLM automation lifecycle › step 3: verify automation and run on the automations page 6.2s
mock-llm-conversation.spec.ts › mock-LLM agent-server conversation › step 1: create an LLM profile pointing at the mock LLM server 6.2s
mock-llm-conversation.spec.ts › mock-LLM agent-server conversation › step 2: activate the mock-llm profile and verify settings API 6.2s
mock-llm-conversation.spec.ts › mock-LLM agent-server conversation › step 3: run a conversation with the mock LLM 6.3s
mock-llm-conversation.spec.ts › mock-LLM agent-server conversation › step 4: resume conversation from sidebar after navigating away 5.7s
⏭️ mock-llm-cross-connect.spec.ts › cross-connect: frontend-only → backend-only › frontend-only connects to a separate backend-only instance 188ms
⏭️ mock-llm-cross-connect.spec.ts › cross-connect: frontend-only → multiple backends › connects to two separate backends and switches between them 192ms
mock-llm-files-and-git.spec.ts › files tab, git control bar, and browser tab › step 1: ensure mock LLM profile is configured 194ms
mock-llm-files-and-git.spec.ts › files tab, git control bar, and browser tab › step 2: start conversation and attach workspace metadata 11.5s
mock-llm-files-and-git.spec.ts › files tab, git control bar, and browser tab › step 3: git control bar shows workspace pill and git actions 25.3s
mock-llm-files-and-git.spec.ts › files tab, git control bar, and browser tab › step 4: files tab defaults to diff view for attached workspace 6.9s
mock-llm-files-and-git.spec.ts › files tab, git control bar, and browser tab › step 5: browser tab shows empty state 6.3s
mock-llm-files-and-git.spec.ts › files tab, git control bar, and browser tab › step 6: files tab defaults to file-tree view without attached workspace 7.2s
mock-llm-folder-workspace.spec.ts › mock-LLM folder browser → workspace → conversation › step 1: browse to a folder, add it as a workspace, and launch a conversation with the correct working_dir 7.4s
mock-llm-image-upload.spec.ts › mock-LLM image upload › attaching an image embeds it as base64 in the LLM completion call 13.4s
mock-llm-mcp-github.spec.ts › MCP GitHub server install flow › step 1: GitHub card is visible on the MCP marketplace page 5.6s
mock-llm-mcp-github.spec.ts › MCP GitHub server install flow › step 2: clicking GitHub card opens the install modal with correct fields 5.8s
mock-llm-mcp-github.spec.ts › MCP GitHub server install flow › step 3: full install flow — fill PAT, submit, verify installed 12.9s
mock-llm-mcp-github.spec.ts › MCP GitHub server install flow › step 4: installed GitHub server can be deleted 5.9s
mock-llm-model-switch.spec.ts › mock-LLM /model slash command › step 1: configure LLM, create switch-target profile, register trajectory 13.3s
mock-llm-model-switch.spec.ts › mock-LLM /model slash command › step 2: start conversation, switch profile via /model, verify switch 6.6s
mock-llm-onboarding-happy-path.spec.ts › onboarding happy path › completes the full onboarding flow and launches a conversation 3.6s
mock-llm-onboarding-regressions.spec.ts › onboarding recent regressions › keeps the modal open on backdrop click and Escape 1.4s
mock-llm-onboarding-regressions.spec.ts › onboarding recent regressions › defaults the LLM setup step to OpenAI GPT-5.5 1.6s
⏭️ mock-llm-partial-stack.spec.ts › partial stack: --frontend-only › serves the frontend but returns 503 for backend routes 207ms
mock-llm-partial-stack.spec.ts › partial stack: --backend-only › serves backend APIs but returns 503 for the frontend root 26.2s
⏭️ mock-llm-partial-stack.spec.ts › partial stack: port conflict › fails with a clear error when the ingress port is occupied 0ms
⏭️ mock-llm-partial-stack.spec.ts › partial stack: port conflict › starts successfully on a free port after a conflict 2ms
mock-llm-preset-automation.spec.ts › preset automation → slash command conversation › automation card sends the correct slash command to a conversation 15.9s
mock-llm-preset-automation.spec.ts › preset automation → slash command conversation › direct slash command from home page triggers skill activation 13.4s
mock-llm-profile-management.spec.ts › active profile deletion + reconciliation › active profile is deletable and reconciliation activates another profile 8.6s
mock-llm-profile-management.spec.ts › same-model profile identity › chat header shows the correct profile when two profiles share the same model 15.0s
mock-llm-profile-management.spec.ts › OpenHands provider base_url normalization › re-saving an OpenHands profile from Basic view drops proxy base_url 8.1s
mock-llm-skills.spec.ts › skill loading: project, user, and deletion › project skill in workspace/.agents/skills/ triggers on matching keyword 13.5s
mock-llm-skills.spec.ts › skill loading: project, user, and deletion › user skill in ~/.openhands/skills/ triggers on matching keyword 13.4s
mock-llm-skills.spec.ts › skill loading: project, user, and deletion › deleting a user skill removes it from subsequent conversations 13.4s
mock-llm-ui-regressions.spec.ts › UI regressions › scopes standalone styles to the agent-server-ui shell 1.3s
mock-llm-ui-regressions.spec.ts › UI regressions › renders critic results on agent messages and finish actions 1.6s
mock-llm-ui-regressions.spec.ts › UI regressions › loads older events when scrolling up 1.7s
mock-llm-ui-regressions.spec.ts › UI regressions › selected workspace persists after navigating away and returning 2.2s
mock-llm-ui-regressions.spec.ts › UI regressions › cleared sessionStorage yields empty workspace selection 1.3s

Posted by the Mock-LLM E2E workflow · results are deterministic (scripted LLM responses)

@all-hands-bot all-hands-bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

✅ QA Report: PASS

Verified the LLM profile settings UI preserves public openhands/* models while dropping stale proxy base_url, and no functional issues were found.

Does this PR achieve its stated goal?

Yes. I exercised the affected settings/profile workflow in a real browser against the mock frontend, using a profile that included stale https://llm-proxy.app.all-hands.dev/ transport data. On the PR branch, saving a public openhands/claude-opus-4-5-20251101 profile from Basic mode sent model: "openhands/..." and omitted base_url; the legacy litellm_proxy/... scenario also initialized directly from the API model instead of reverse-mapping to OpenHands.

Phase Result
Environment Setup ✅ Started npm run dev:mock -- --host 127.0.0.1 --port 3001 successfully
CI Status ✅ GitHub checks were green for CI, Docker, Mock-LLM E2E, Mock-LLM Docker E2E, Vercel; live-e2e skipped; QA workflow in progress at inspection time
Functional Verification ✅ Browser UI save payloads matched the intended public-provider behavior
Functional Verification

Test 1: Public openhands/* profile with stale proxy base_url

Step 1 — Establish baseline without the PR:
Checked out PR parent cb6f370d111d05723fd6946ed17f1be1c969bf27, started npm run dev:mock -- --host 127.0.0.1 --port 3001, opened /settings/llm in Chromium, edited profile qa-openhands-stale-base-url, and saved from Basic mode. The captured save request was:

{
  "method": "POST",
  "path": "/api/profiles/qa-openhands-stale-base-url",
  "body": {
    "llm": {
      "model": "openhands/claude-opus-4-5-20251101",
      "api_key": "gAAAA_encrypted_key",
      "temperature": null,
      "auth_type": "api_key",
      "subscription_vendor": null
    },
    "include_secrets": true
  }
}

This established the user workflow and fixture: a stale profile-level proxy base_url was present in the API response, but the Basic-mode save path could be exercised end-to-end through the UI.

Step 2 — Apply the PR's changes:
Checked out openhands/public-openhands-provider at 7fcfb8fafcf4e95ca11615bcfe0f84c9161dafa9 and restarted the same mock frontend command.

Step 3 — Re-run with the PR in place:
Repeated the same browser workflow. The UI showed provider OpenHands, model claude-opus-4-5-20251101, and the captured save request was:

{
  "method": "POST",
  "path": "/api/profiles/qa-openhands-stale-base-url",
  "body": {
    "llm": {
      "model": "openhands/claude-opus-4-5-20251101",
      "api_key": "gAAAA_encrypted_key",
      "temperature": null,
      "auth_type": "api_key",
      "subscription_vendor": null
    },
    "include_secrets": true
  }
}

This confirms the PR preserves the public openhands/* model identity and does not re-save the stale OpenHands LiteLLM proxy base_url from the Basic profile form.

Test 2: Legacy litellm_proxy/* profile no longer reverse-maps in the frontend

Step 1 — Reproduce old behavior without the PR:
On PR parent cb6f370d111d05723fd6946ed17f1be1c969bf27, I used the same settings page with a legacy profile returned as model: "litellm_proxy/claude-opus-4-5-20251101" plus the stale OpenHands proxy base_url. The editor initialized as:

{
  "providerValue": "OpenHands",
  "modelValue": "claude-opus-4-5-20251101"
}

This shows the old frontend reverse-mapping behavior: the stored/API model was litellm_proxy/*, but the UI presented it as OpenHands.

Step 2 — Apply the PR's changes:
Checked out openhands/public-openhands-provider at 7fcfb8fafcf4e95ca11615bcfe0f84c9161dafa9 and restarted the same mock frontend command.

Step 3 — Re-run with the PR in place:
Repeated the same browser edit/save workflow. The editor initialized directly from the API model and the save payload preserved that identity while omitting base_url:

{
  "providerValue": "litellm_proxy",
  "modelValue": "claude-opus-4-5-20251101",
  "capturedSave": {
    "body": {
      "llm": {
        "model": "litellm_proxy/claude-opus-4-5-20251101",
        "api_key": "gAAAA_encrypted_key",
        "temperature": null,
        "auth_type": "api_key",
        "subscription_vendor": null
      },
      "include_secrets": true
    }
  }
}

This confirms the specific refactor goal: Canvas no longer reverse-maps litellm_proxy/* + OpenHands proxy URL to openhands/*; it trusts the model returned by the SDK/API and still drops hidden transport details on Basic-mode save.

Issues Found

None.

This review was created by an AI agent (OpenHands) on behalf of the user.

Comment thread tests/e2e/mock-llm/mock-llm-profile-management.spec.ts

enyst commented Jun 13, 2026

Copy link
Copy Markdown
Member Author

Thanks — verified against OpenHands/OpenHands#14776 and software-agent-sdk#3548. The concern was valid: the merged behavior wiped hidden base_url on same-model Basic saves.

I opened a follow-up PR from main: #1347

The fix preserves hidden base_url unless the Basic-view model actually changes, and updates unit + mock-LLM E2E coverage accordingly. Labels qa-this and review-this are applied and the PR is nondraft.

This comment was generated by an AI agent (OpenHands) on behalf of the user.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

qa-this Trigger OpenHands QA validation review-this

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Keep openhands/* as public LLM model; translate to litellm_proxy only at transport time

2 participants