Skip to content

Add lock-to-cloud backend setup mode#1389

Open
neubig wants to merge 11 commits into
fix-public-onboardingfrom
lock-to-cloud-backend
Open

Add lock-to-cloud backend setup mode#1389
neubig wants to merge 11 commits into
fix-public-onboardingfrom
lock-to-cloud-backend

Conversation

@neubig

@neubig neubig commented Jun 16, 2026

Copy link
Copy Markdown
Member

HUMAN:
I tested this manually and it works as expected, only showing the cloud login when this option is present.

  • A human has tested these changes.

AGENT:

Why

Frontend-only/public deployments sometimes need backend setup locked to one OpenHands Cloud URL so users cannot configure arbitrary local or custom backend targets during onboarding or Add Backend.

Fixes #1392.

Summary

  • Add compile-time VITE_LOCK_TO_CLOUD=<cloud-url> configuration for locking backend setup to a single OpenHands Cloud URL.
  • Add runtime scripts/static-server.mjs --lock-to-cloud <cloud-url> support for pre-built/frontend-only deployments.
  • When locked, onboarding and the Add Backend modal hide the manual/local backend form and the Advanced custom Cloud URL control, leaving only Cloud login against the configured URL.
  • Add config, static-server injection, onboarding, and Add Backend modal tests.
  • Keep this branch stacked on Show onboarding before public backend auth gate #1385 by merging the updated fix-public-onboarding parent branch into lock-to-cloud-backend.

Stacked on #1385. Current stack heads: parent e81bcd6a, child 6f994e3b.

How to Test

  • PATH=/usr/bin:$PATH npm run typecheck
  • PATH=/usr/bin:$PATH npm run test -- __tests__/api/agent-server-config.test.ts __tests__/components/backends/backend-form-modal.test.tsx __tests__/components/onboarding/onboarding-modal.test.tsx __tests__/scripts/static-server.test.ts __tests__/root.test.tsx
  • PATH=/usr/bin:$PATH npm run build

Manual QA

  • Verified --lock-to-cloud https://app.all-hands.dev on https://work-2-tuizajshdoegydst.prod-runtime.all-hands.dev/: first-run onboarding shows only OpenHands Cloud login, with no local backend form and no Advanced custom Cloud URL field.

This PR description was updated by an AI agent (OpenHands) on behalf of the user.


🐳 Docker images for this PR

GHCR package: https://github.com/OpenHands/agent-canvas/pkgs/container/agent-canvas

Component Value
Image ghcr.io/openhands/agent-canvas
Architectures amd64, arm64
Agent Server ghcr.io/openhands/agent-server:1.28.1-python
Automation openhands-automation==1.0.0a10
Commit 6e7bac6faf13e0fa1e6fa98c975cdc56bcfddd7b

Pull (multi-arch manifest)

# Multi-arch manifest — Docker automatically pulls the correct architecture
docker pull ghcr.io/openhands/agent-canvas:sha-6e7bac6

Run

docker run -it --rm \
  -p 8000:8000 \
  ghcr.io/openhands/agent-canvas:sha-6e7bac6

All tags pushed for this build

ghcr.io/openhands/agent-canvas:sha-6e7bac6-amd64
ghcr.io/openhands/agent-canvas:lock-to-cloud-backend-amd64
ghcr.io/openhands/agent-canvas:pr-1389-amd64
ghcr.io/openhands/agent-canvas:sha-6e7bac6-arm64
ghcr.io/openhands/agent-canvas:lock-to-cloud-backend-arm64
ghcr.io/openhands/agent-canvas:pr-1389-arm64
ghcr.io/openhands/agent-canvas:sha-6e7bac6
ghcr.io/openhands/agent-canvas:lock-to-cloud-backend
ghcr.io/openhands/agent-canvas:pr-1389

About Multi-Architecture Support

  • Each tag (e.g., sha-6e7bac6) is a multi-arch manifest supporting both amd64 and arm64
  • Docker automatically pulls the correct architecture for your platform
  • Individual architecture tags (e.g., sha-6e7bac6-amd64) are also available if needed

@vercel

vercel Bot commented Jun 16, 2026

Copy link
Copy Markdown

The latest updates on your projects. Learn more about Vercel for GitHub.

Project Deployment Actions Updated (UTC)
agent-canvas Ready Ready Preview, Comment Jun 18, 2026 7:55pm

Request Review

@github-actions

Copy link
Copy Markdown
Contributor

⚠️ Mock-LLM Docker E2E Test Results

0/0 passed

Commit: a415caf9 · Workflow run

Status Test Duration

Posted by the Mock-LLM E2E workflow · results are deterministic (scripted LLM responses)

@github-actions

Copy link
Copy Markdown
Contributor

🛑 Mock-LLM E2E Tests

6/6 passed · ⚠️ 54 not run (process killed at 6/60)

Commit: a415caf9 · Workflow run · Test artifacts

Status Test Duration
chromium › automations/mock-llm-automation.spec.ts › mock-LLM automation lifecycle › step 1: setup LLM profile and register automation trajectory 7.4s
chromium › automations/mock-llm-automation.spec.ts › mock-LLM automation lifecycle › step 2: create automation and dispatch run via the UI 27.5s
chromium › automations/mock-llm-automation.spec.ts › mock-LLM automation lifecycle › step 3: verify automation and run on the automations page 6.1s
chromium › automations/mock-llm-preset-automation.spec.ts › preset automation → slash command conversation › automation card sends the correct slash command to a conversation 16.6s
chromium › automations/mock-llm-preset-automation.spec.ts › preset automation → slash command conversation › direct slash command from home page triggers skill activation 14.9s
chromium › backends/mock-llm-auth-modes.spec.ts › auth mode: fresh install with runtime-injected key › reaches the onboarding modal without pre-seeded localStorage 1.5s

Posted by the Mock-LLM E2E workflow · results are deterministic (scripted LLM responses)

@github-actions

Copy link
Copy Markdown
Contributor

⚠️ Mock-LLM Docker E2E Test Results

0/0 passed

Commit: b39a6bbf · Workflow run

Status Test Duration

Posted by the Mock-LLM E2E workflow · results are deterministic (scripted LLM responses)

@github-actions

Copy link
Copy Markdown
Contributor

🛑 Mock-LLM E2E Tests

8/13 passed · 1 failed · 4 skipped · ⚠️ 47 not run (process killed at 13/60)

Commit: b39a6bbf · Workflow run · Test artifacts

Status Test Duration
chromium › automations/mock-llm-automation.spec.ts › mock-LLM automation lifecycle › step 1: setup LLM profile and register automation trajectory 7.4s
chromium › automations/mock-llm-automation.spec.ts › mock-LLM automation lifecycle › step 2: create automation and dispatch run via the UI 27.5s
chromium › automations/mock-llm-automation.spec.ts › mock-LLM automation lifecycle › step 3: verify automation and run on the automations page 6.2s
chromium › automations/mock-llm-preset-automation.spec.ts › preset automation → slash command conversation › automation card sends the correct slash command to a conversation 16.5s
chromium › automations/mock-llm-preset-automation.spec.ts › preset automation → slash command conversation › direct slash command from home page triggers skill activation 13.7s
chromium › backends/mock-llm-auth-modes.spec.ts › auth mode: fresh install with runtime-injected key › reaches the onboarding modal without pre-seeded localStorage 1.3s
chromium › backends/mock-llm-auth-modes.spec.ts › auth mode: non-public key rotation › recovers when localStorage has a stale session API key 5.4s
chromium › backends/mock-llm-auth-modes.spec.ts › auth mode: public gate › shows the auth screen when no key is configured 32.0s
⏭️ chromium › backends/mock-llm-auth-modes.spec.ts › auth mode: public gate › rejects an incorrect key with an inline error 0ms
⏭️ chromium › backends/mock-llm-auth-modes.spec.ts › auth mode: public gate › allows access after pasting the correct key 0ms
⏭️ chromium › backends/mock-llm-auth-modes.spec.ts › auth mode: public gate › skips auth screen for returning user with valid stored key 0ms
⏭️ chromium › backends/mock-llm-auth-modes.spec.ts › auth mode: public gate › re-prompts when the server rotates its key (stale localStorage) 0ms
chromium › backends/mock-llm-cross-connect.spec.ts › cross-connect: frontend-only → backend-only › frontend-only connects to a separate backend-only instance 16.0s
🔍 Failure details (1)

❌ chromium › backends/mock-llm-auth-modes.spec.ts › auth mode: public gate › shows the auth screen when no key is configured

Error: �[2mexpect(�[22m�[31mlocator�[39m�[2m).�[22mtoBeVisible�[2m(�[22m�[2m)�[22m failed

Locator: getByTestId('api-key-entry-screen')
Expected: visible
Timeout: 30000ms
Error: element(s) not found

Call log:
�[2m  - Expect "toBeVisible" with timeout 30000ms�[22m
�[2m  - waiting for getByTestId('api-key-entry-screen')�[22m

Posted by the Mock-LLM E2E workflow · results are deterministic (scripted LLM responses)

@github-actions

Copy link
Copy Markdown
Contributor

❌ Mock-LLM E2E Tests

55/60 passed · 1 failed · 4 skipped

Commit: 89456704 · Workflow run · Test artifacts

Status Test Duration
automations/mock-llm-automation.spec.ts › mock-LLM automation lifecycle › step 1: setup LLM profile and register automation trajectory 7.4s
automations/mock-llm-automation.spec.ts › mock-LLM automation lifecycle › step 2: create automation and dispatch run via the UI 28.5s
automations/mock-llm-automation.spec.ts › mock-LLM automation lifecycle › step 3: verify automation and run on the automations page 6.3s
automations/mock-llm-preset-automation.spec.ts › preset automation → slash command conversation › automation card sends the correct slash command to a conversation 16.7s
automations/mock-llm-preset-automation.spec.ts › preset automation → slash command conversation › direct slash command from home page triggers skill activation 14.5s
backends/mock-llm-auth-modes.spec.ts › auth mode: fresh install with runtime-injected key › reaches the onboarding modal without pre-seeded localStorage 1.4s
backends/mock-llm-auth-modes.spec.ts › auth mode: non-public key rotation › recovers when localStorage has a stale session API key 5.4s
backends/mock-llm-auth-modes.spec.ts › auth mode: public gate › shows the auth screen when no key is configured 32.0s
⏭️ backends/mock-llm-auth-modes.spec.ts › auth mode: public gate › rejects an incorrect key with an inline error 0ms
⏭️ backends/mock-llm-auth-modes.spec.ts › auth mode: public gate › allows access after pasting the correct key 0ms
⏭️ backends/mock-llm-auth-modes.spec.ts › auth mode: public gate › skips auth screen for returning user with valid stored key 0ms
⏭️ backends/mock-llm-auth-modes.spec.ts › auth mode: public gate › re-prompts when the server rotates its key (stale localStorage) 0ms
backends/mock-llm-cross-connect.spec.ts › cross-connect: frontend-only → backend-only › frontend-only connects to a separate backend-only instance 16.1s
backends/mock-llm-cross-connect.spec.ts › cross-connect: frontend-only → multiple backends › connects to two separate backends and switches between them 27.6s
backends/mock-llm-partial-stack.spec.ts › partial stack: --frontend-only › serves the frontend but returns 503 for backend routes 7.6s
backends/mock-llm-partial-stack.spec.ts › partial stack: --backend-only › serves backend APIs but returns 503 for the frontend root 15.1s
backends/mock-llm-partial-stack.spec.ts › partial stack: port conflict › fails with a clear error when the ingress port is occupied 111ms
backends/mock-llm-partial-stack.spec.ts › partial stack: port conflict › starts successfully on a free port after a conflict 6.1s
conversations/mock-llm-conversation.spec.ts › mock-LLM agent-server conversation › step 1: create an LLM profile pointing at the mock LLM server 6.4s
conversations/mock-llm-conversation.spec.ts › mock-LLM agent-server conversation › step 2: activate the mock-llm profile and verify settings API 6.3s
conversations/mock-llm-conversation.spec.ts › mock-LLM agent-server conversation › step 3: run a conversation with the mock LLM 7.4s
conversations/mock-llm-conversation.spec.ts › mock-LLM agent-server conversation › step 4: resume conversation from sidebar after navigating away 5.9s
conversations/mock-llm-image-upload.spec.ts › mock-LLM image upload › attaching an image embeds it as base64 in the LLM completion call 14.5s
files/mock-llm-files-and-git.spec.ts › files tab, git control bar, and browser tab › step 1: ensure mock LLM profile is configured 213ms
files/mock-llm-files-and-git.spec.ts › files tab, git control bar, and browser tab › step 2: start conversation and attach workspace metadata 12.5s
files/mock-llm-files-and-git.spec.ts › files tab, git control bar, and browser tab › step 3: git control bar shows workspace pill and git actions 25.3s
files/mock-llm-files-and-git.spec.ts › files tab, git control bar, and browser tab › step 4: files tab defaults to diff view for attached workspace 6.0s
files/mock-llm-files-and-git.spec.ts › files tab, git control bar, and browser tab › step 5: browser tab shows empty state 6.4s
files/mock-llm-files-and-git.spec.ts › files tab, git control bar, and browser tab › step 6: files tab defaults to file-tree view without attached workspace 8.3s
home/mock-llm-folder-workspace.spec.ts › mock-LLM folder browser → workspace → conversation › step 1: browse to a folder, add it as a workspace, and launch a conversation with the correct working_dir 8.8s
mcp/mock-llm-mcp-github.spec.ts › MCP GitHub server install flow › step 1: GitHub card is visible on the MCP marketplace page 5.6s
mcp/mock-llm-mcp-github.spec.ts › MCP GitHub server install flow › step 2: clicking GitHub card opens the install modal with correct fields 5.8s
mcp/mock-llm-mcp-github.spec.ts › MCP GitHub server install flow › step 3: full install flow — fill PAT, submit, verify installed 13.2s
mcp/mock-llm-mcp-github.spec.ts › MCP GitHub server install flow › step 4: installed GitHub server can be deleted 5.9s
mcp/mock-llm-mcp-slack-credentials.spec.ts › MCP Test Connection credential verification (Slack) › install: invalid Slack credentials are blocked with a credential-check error 5.9s
mcp/mock-llm-mcp-slack-credentials.spec.ts › MCP Test Connection credential verification (Slack) › install: a valid token missing only a scope still installs (missing_scope is not a credential failure) 6.0s
mcp/mock-llm-mcp-slack-credentials.spec.ts › MCP Test Connection credential verification (Slack) › install: an older agent server that omits tool_result still installs (compat) 5.9s
mcp/mock-llm-mcp-slack-credentials.spec.ts › MCP Test Connection credential verification (Slack) › edit: Test Connection verifies the stored credentials and surfaces a credential failure 5.8s
mcp/mock-llm-mcp-slack-credentials.spec.ts › MCP Test Connection credential verification (Slack) › edit: Test Connection reports success for valid stored credentials 5.8s
mcp/mock-llm-mcp-slack-credentials.spec.ts › MCP Test Connection credential verification (Slack) › custom (non-catalog) server: Test Connection attaches no verification probe 5.8s
onboarding/mock-llm-onboarding-happy-path.spec.ts › onboarding happy path › completes the full onboarding flow and launches a conversation 4.5s
onboarding/mock-llm-onboarding-regressions.spec.ts › onboarding recent regressions › keeps the modal open on backdrop click and Escape 1.4s
onboarding/mock-llm-onboarding-regressions.spec.ts › onboarding recent regressions › defaults the LLM setup step to OpenAI GPT-5.5 1.6s
regressions/mock-llm-ui-regressions.spec.ts › UI regressions › scopes standalone styles to the agent-server-ui shell 1.4s
regressions/mock-llm-ui-regressions.spec.ts › UI regressions › renders critic results on agent messages and finish actions 1.5s
regressions/mock-llm-ui-regressions.spec.ts › UI regressions › loads older events when scrolling up 1.8s
regressions/mock-llm-ui-regressions.spec.ts › UI regressions › selected workspace persists after navigating away and returning 2.5s
regressions/mock-llm-ui-regressions.spec.ts › UI regressions › cleared sessionStorage yields empty workspace selection 1.0s
settings/mock-llm-acp-agent.spec.ts › mock-LLM ACP agent conversation › step 1: configure ACP agent via Settings → Agent UI 14.1s
settings/mock-llm-acp-agent.spec.ts › mock-LLM ACP agent conversation › step 2: reload and verify ACP settings are persisted in UI 5.6s
settings/mock-llm-acp-agent.spec.ts › mock-LLM ACP agent conversation › step 3: start ACP conversation and verify agent reply 6.7s
settings/mock-llm-acp-agent.spec.ts › mock-LLM ACP agent conversation › step 4: resume ACP conversation from sidebar after navigating away 5.8s
settings/mock-llm-model-switch.spec.ts › mock-LLM /model slash command › step 1: configure LLM, create switch-target profile, register trajectory 13.2s
settings/mock-llm-model-switch.spec.ts › mock-LLM /model slash command › step 2: start conversation, switch profile via /model, verify switch 7.0s
settings/mock-llm-profile-management.spec.ts › active profile deletion + reconciliation › active profile is deletable and reconciliation activates another profile 8.6s
settings/mock-llm-profile-management.spec.ts › same-model profile identity › chat header shows the correct profile when two profiles share the same model 16.0s
settings/mock-llm-profile-management.spec.ts › OpenHands provider hidden base_url preservation › re-saving an OpenHands profile from Basic view preserves hidden base_url 7.8s
skills/mock-llm-skills.spec.ts › skill loading: project, user, and deletion › project skill in workspace/.agents/skills/ triggers on matching keyword 13.9s
skills/mock-llm-skills.spec.ts › skill loading: project, user, and deletion › user skill in ~/.openhands/skills/ triggers on matching keyword 14.2s
skills/mock-llm-skills.spec.ts › skill loading: project, user, and deletion › deleting a user skill removes it from subsequent conversations 13.8s
🔍 Failure details (1)

❌ backends/mock-llm-auth-modes.spec.ts › auth mode: public gate › shows the auth screen when no key is configured

Error: expect(locator).toBeVisible() failed

Locator: getByTestId('api-key-entry-screen')
Expected: visible
Timeout: 30000ms
Error: element(s) not found

Call log:
  - Expect "toBeVisible" with timeout 30000ms
  - waiting for getByTestId('api-key-entry-screen')

Posted by the Mock-LLM E2E workflow · results are deterministic (scripted LLM responses)

@github-actions

Copy link
Copy Markdown
Contributor

⚠️ Mock-LLM Docker E2E Test Results

0/0 passed

Commit: 89456704 · Workflow run

Status Test Duration

Posted by the Mock-LLM E2E workflow · results are deterministic (scripted LLM responses)

@github-actions

Copy link
Copy Markdown
Contributor

❌ Mock-LLM E2E Tests

52/60 passed · 3 failed · 5 skipped

Commit: 8af6854f · Workflow run · Test artifacts

Status Test Duration
automations/mock-llm-automation.spec.ts › mock-LLM automation lifecycle › step 1: setup LLM profile and register automation trajectory 7.2s
automations/mock-llm-automation.spec.ts › mock-LLM automation lifecycle › step 2: create automation and dispatch run via the UI 27.7s
automations/mock-llm-automation.spec.ts › mock-LLM automation lifecycle › step 3: verify automation and run on the automations page 6.2s
automations/mock-llm-preset-automation.spec.ts › preset automation → slash command conversation › automation card sends the correct slash command to a conversation 16.8s
automations/mock-llm-preset-automation.spec.ts › preset automation → slash command conversation › direct slash command from home page triggers skill activation 13.7s
backends/mock-llm-auth-modes.spec.ts › auth mode: fresh install with runtime-injected key › reaches the onboarding modal without pre-seeded localStorage 1.3s
backends/mock-llm-auth-modes.spec.ts › auth mode: non-public key rotation › recovers when localStorage has a stale session API key 5.3s
backends/mock-llm-auth-modes.spec.ts › auth mode: public gate › shows the auth screen when no key is configured 32.8s
⏭️ backends/mock-llm-auth-modes.spec.ts › auth mode: public gate › rejects an incorrect key with an inline error 0ms
⏭️ backends/mock-llm-auth-modes.spec.ts › auth mode: public gate › allows access after pasting the correct key 0ms
⏭️ backends/mock-llm-auth-modes.spec.ts › auth mode: public gate › skips auth screen for returning user with valid stored key 0ms
⏭️ backends/mock-llm-auth-modes.spec.ts › auth mode: public gate › re-prompts when the server rotates its key (stale localStorage) 0ms
backends/mock-llm-cross-connect.spec.ts › cross-connect: frontend-only → backend-only › frontend-only connects to a separate backend-only instance 16.0s
backends/mock-llm-cross-connect.spec.ts › cross-connect: frontend-only → multiple backends › connects to two separate backends and switches between them 19.8s
backends/mock-llm-partial-stack.spec.ts › partial stack: --frontend-only › serves the frontend but returns 503 for backend routes 7.4s
backends/mock-llm-partial-stack.spec.ts › partial stack: --backend-only › serves backend APIs but returns 503 for the frontend root 13.1s
backends/mock-llm-partial-stack.spec.ts › partial stack: port conflict › fails with a clear error when the ingress port is occupied 104ms
backends/mock-llm-partial-stack.spec.ts › partial stack: port conflict › starts successfully on a free port after a conflict 6.0s
conversations/mock-llm-conversation.spec.ts › mock-LLM agent-server conversation › step 1: create an LLM profile pointing at the mock LLM server 6.3s
conversations/mock-llm-conversation.spec.ts › mock-LLM agent-server conversation › step 2: activate the mock-llm profile and verify settings API 6.2s
conversations/mock-llm-conversation.spec.ts › mock-LLM agent-server conversation › step 3: run a conversation with the mock LLM 7.3s
conversations/mock-llm-conversation.spec.ts › mock-LLM agent-server conversation › step 4: resume conversation from sidebar after navigating away 5.7s
conversations/mock-llm-image-upload.spec.ts › mock-LLM image upload › attaching an image embeds it as base64 in the LLM completion call 13.5s
files/mock-llm-files-and-git.spec.ts › files tab, git control bar, and browser tab › step 1: ensure mock LLM profile is configured 201ms
files/mock-llm-files-and-git.spec.ts › files tab, git control bar, and browser tab › step 2: start conversation and attach workspace metadata 11.6s
files/mock-llm-files-and-git.spec.ts › files tab, git control bar, and browser tab › step 3: git control bar shows workspace pill and git actions 25.3s
files/mock-llm-files-and-git.spec.ts › files tab, git control bar, and browser tab › step 4: files tab defaults to diff view for attached workspace 5.9s
files/mock-llm-files-and-git.spec.ts › files tab, git control bar, and browser tab › step 5: browser tab shows empty state 6.3s
files/mock-llm-files-and-git.spec.ts › files tab, git control bar, and browser tab › step 6: files tab defaults to file-tree view without attached workspace 8.2s
home/mock-llm-folder-workspace.spec.ts › mock-LLM folder browser → workspace → conversation › step 1: browse to a folder, add it as a workspace, and launch a conversation with the correct working_dir 7.7s
mcp/mock-llm-mcp-github.spec.ts › MCP GitHub server install flow › step 1: GitHub card is visible on the MCP marketplace page 6.8s
mcp/mock-llm-mcp-github.spec.ts › MCP GitHub server install flow › step 2: clicking GitHub card opens the install modal with correct fields 5.7s
mcp/mock-llm-mcp-github.spec.ts › MCP GitHub server install flow › step 3: full install flow — fill PAT, submit, verify installed 12.9s
mcp/mock-llm-mcp-github.spec.ts › MCP GitHub server install flow › step 4: installed GitHub server can be deleted 5.8s
mcp/mock-llm-mcp-slack-credentials.spec.ts › MCP Test Connection credential verification (Slack) › install: invalid Slack credentials are blocked with a credential-check error 5.8s
mcp/mock-llm-mcp-slack-credentials.spec.ts › MCP Test Connection credential verification (Slack) › install: a valid token missing only a scope still installs (missing_scope is not a credential failure) 5.9s
mcp/mock-llm-mcp-slack-credentials.spec.ts › MCP Test Connection credential verification (Slack) › install: an older agent server that omits tool_result still installs (compat) 6.0s
mcp/mock-llm-mcp-slack-credentials.spec.ts › MCP Test Connection credential verification (Slack) › edit: Test Connection verifies the stored credentials and surfaces a credential failure 5.8s
mcp/mock-llm-mcp-slack-credentials.spec.ts › MCP Test Connection credential verification (Slack) › edit: Test Connection reports success for valid stored credentials 5.8s
mcp/mock-llm-mcp-slack-credentials.spec.ts › MCP Test Connection credential verification (Slack) › custom (non-catalog) server: Test Connection attaches no verification probe 5.8s
⏱️ onboarding/mock-llm-onboarding-happy-path.spec.ts › onboarding happy path › completes the full onboarding flow and launches a conversation 120.1s
onboarding/mock-llm-onboarding-regressions.spec.ts › onboarding recent regressions › keeps the modal open on backdrop click and Escape 1.6s
onboarding/mock-llm-onboarding-regressions.spec.ts › onboarding recent regressions › defaults the LLM setup step to OpenAI GPT-5.5 11.4s
regressions/mock-llm-ui-regressions.spec.ts › UI regressions › scopes standalone styles to the agent-server-ui shell 1.5s
regressions/mock-llm-ui-regressions.spec.ts › UI regressions › renders critic results on agent messages and finish actions 1.5s
regressions/mock-llm-ui-regressions.spec.ts › UI regressions › loads older events when scrolling up 1.7s
regressions/mock-llm-ui-regressions.spec.ts › UI regressions › selected workspace persists after navigating away and returning 2.1s
regressions/mock-llm-ui-regressions.spec.ts › UI regressions › cleared sessionStorage yields empty workspace selection 985ms
settings/mock-llm-acp-agent.spec.ts › mock-LLM ACP agent conversation › step 1: configure ACP agent via Settings → Agent UI 13.8s
settings/mock-llm-acp-agent.spec.ts › mock-LLM ACP agent conversation › step 2: reload and verify ACP settings are persisted in UI 5.6s
settings/mock-llm-acp-agent.spec.ts › mock-LLM ACP agent conversation › step 3: start ACP conversation and verify agent reply 6.7s
settings/mock-llm-acp-agent.spec.ts › mock-LLM ACP agent conversation › step 4: resume ACP conversation from sidebar after navigating away 5.8s
settings/mock-llm-model-switch.spec.ts › mock-LLM /model slash command › step 1: configure LLM, create switch-target profile, register trajectory 13.1s
settings/mock-llm-model-switch.spec.ts › mock-LLM /model slash command › step 2: start conversation, switch profile via /model, verify switch 7.0s
⏭️ settings/mock-llm-profile-management.spec.ts › active profile deletion + reconciliation › active profile is deletable and reconciliation activates another profile 0ms
settings/mock-llm-profile-management.spec.ts › same-model profile identity › chat header shows the correct profile when two profiles share the same model 0ms
settings/mock-llm-profile-management.spec.ts › OpenHands provider hidden base_url preservation › re-saving an OpenHands profile from Basic view preserves hidden base_url 0ms
skills/mock-llm-skills.spec.ts › skill loading: project, user, and deletion › project skill in workspace/.agents/skills/ triggers on matching keyword 0ms
skills/mock-llm-skills.spec.ts › skill loading: project, user, and deletion › user skill in ~/.openhands/skills/ triggers on matching keyword 0ms
skills/mock-llm-skills.spec.ts › skill loading: project, user, and deletion › deleting a user skill removes it from subsequent conversations 0ms
🔍 Failure details (3)

❌ backends/mock-llm-auth-modes.spec.ts › auth mode: public gate › shows the auth screen when no key is configured

Error: expect(locator).toBeVisible() failed

Locator: getByTestId('api-key-entry-screen')
Expected: visible
Timeout: 30000ms
Error: element(s) not found

Call log:
  - Expect "toBeVisible" with timeout 30000ms
  - waiting for getByTestId('api-key-entry-screen')

⏱️ onboarding/mock-llm-onboarding-happy-path.spec.ts › onboarding happy path › completes the full onboarding flow and launches a conversation

Test timeout of 120000ms exceeded.

❌ onboarding/mock-llm-onboarding-regressions.spec.ts › onboarding recent regressions › defaults the LLM setup step to OpenAI GPT-5.5

Error: onboarding backend health probe should report connected

expect(locator).toBeVisible() failed

Locator: getByTestId('onboarding-backend-connected')
Expected: visible
Timeout: 10000ms
Error: element(s) not found

Call log:
  - onboarding backend health probe should report connected with timeout 10000ms
  - waiting for getByTestId('onboarding-backend-connected')

Posted by the Mock-LLM E2E workflow · results are deterministic (scripted LLM responses)

@github-actions

Copy link
Copy Markdown
Contributor

⚠️ Mock-LLM Docker E2E Test Results

0/0 passed

Commit: 8af6854f · Workflow run

Status Test Duration

Posted by the Mock-LLM E2E workflow · results are deterministic (scripted LLM responses)

@github-actions

Copy link
Copy Markdown
Contributor

❌ Mock-LLM E2E Tests

53/60 passed · 3 failed · 4 skipped

Commit: 6eb36f5d · Workflow run · Test artifacts

Status Test Duration
automations/mock-llm-automation.spec.ts › mock-LLM automation lifecycle › step 1: setup LLM profile and register automation trajectory 7.4s
automations/mock-llm-automation.spec.ts › mock-LLM automation lifecycle › step 2: create automation and dispatch run via the UI 28.5s
automations/mock-llm-automation.spec.ts › mock-LLM automation lifecycle › step 3: verify automation and run on the automations page 6.2s
automations/mock-llm-preset-automation.spec.ts › preset automation → slash command conversation › automation card sends the correct slash command to a conversation 16.6s
automations/mock-llm-preset-automation.spec.ts › preset automation → slash command conversation › direct slash command from home page triggers skill activation 13.7s
backends/mock-llm-auth-modes.spec.ts › auth mode: fresh install with runtime-injected key › reaches the onboarding modal without pre-seeded localStorage 1.4s
backends/mock-llm-auth-modes.spec.ts › auth mode: non-public key rotation › recovers when localStorage has a stale session API key 5.4s
backends/mock-llm-auth-modes.spec.ts › auth mode: public gate › shows first-run onboarding before the auth screen when no key is configured 1.5s
backends/mock-llm-auth-modes.spec.ts › auth mode: public gate › rejects an incorrect key with an inline error 32.0s
⏭️ backends/mock-llm-auth-modes.spec.ts › auth mode: public gate › allows access after pasting the correct key 0ms
⏭️ backends/mock-llm-auth-modes.spec.ts › auth mode: public gate › skips auth screen for returning user with valid stored key 0ms
⏭️ backends/mock-llm-auth-modes.spec.ts › auth mode: public gate › re-prompts when the server rotates its key (stale localStorage) 0ms
backends/mock-llm-cross-connect.spec.ts › cross-connect: frontend-only → backend-only › frontend-only connects to a separate backend-only instance 16.0s
backends/mock-llm-cross-connect.spec.ts › cross-connect: frontend-only → multiple backends › connects to two separate backends and switches between them 20.8s
backends/mock-llm-partial-stack.spec.ts › partial stack: --frontend-only › serves the frontend but returns 503 for backend routes 7.5s
backends/mock-llm-partial-stack.spec.ts › partial stack: --backend-only › serves backend APIs but returns 503 for the frontend root 14.1s
backends/mock-llm-partial-stack.spec.ts › partial stack: port conflict › fails with a clear error when the ingress port is occupied 102ms
backends/mock-llm-partial-stack.spec.ts › partial stack: port conflict › starts successfully on a free port after a conflict 6.0s
conversations/mock-llm-conversation.spec.ts › mock-LLM agent-server conversation › step 1: create an LLM profile pointing at the mock LLM server 6.4s
conversations/mock-llm-conversation.spec.ts › mock-LLM agent-server conversation › step 2: activate the mock-llm profile and verify settings API 6.4s
conversations/mock-llm-conversation.spec.ts › mock-LLM agent-server conversation › step 3: run a conversation with the mock LLM 6.7s
conversations/mock-llm-conversation.spec.ts › mock-LLM agent-server conversation › step 4: resume conversation from sidebar after navigating away 5.9s
conversations/mock-llm-image-upload.spec.ts › mock-LLM image upload › attaching an image embeds it as base64 in the LLM completion call 13.7s
files/mock-llm-files-and-git.spec.ts › files tab, git control bar, and browser tab › step 1: ensure mock LLM profile is configured 7.4s
files/mock-llm-files-and-git.spec.ts › files tab, git control bar, and browser tab › step 2: start conversation and attach workspace metadata 11.9s
files/mock-llm-files-and-git.spec.ts › files tab, git control bar, and browser tab › step 3: git control bar shows workspace pill and git actions 25.4s
files/mock-llm-files-and-git.spec.ts › files tab, git control bar, and browser tab › step 4: files tab defaults to diff view for attached workspace 5.9s
files/mock-llm-files-and-git.spec.ts › files tab, git control bar, and browser tab › step 5: browser tab shows empty state 6.3s
files/mock-llm-files-and-git.spec.ts › files tab, git control bar, and browser tab › step 6: files tab defaults to file-tree view without attached workspace 7.7s
home/mock-llm-folder-workspace.spec.ts › mock-LLM folder browser → workspace → conversation › step 1: browse to a folder, add it as a workspace, and launch a conversation with the correct working_dir 8.7s
mcp/mock-llm-mcp-github.spec.ts › MCP GitHub server install flow › step 1: GitHub card is visible on the MCP marketplace page 5.6s
mcp/mock-llm-mcp-github.spec.ts › MCP GitHub server install flow › step 2: clicking GitHub card opens the install modal with correct fields 5.8s
mcp/mock-llm-mcp-github.spec.ts › MCP GitHub server install flow › step 3: full install flow — fill PAT, submit, verify installed 13.1s
mcp/mock-llm-mcp-github.spec.ts › MCP GitHub server install flow › step 4: installed GitHub server can be deleted 5.9s
mcp/mock-llm-mcp-slack-credentials.spec.ts › MCP Test Connection credential verification (Slack) › install: invalid Slack credentials are blocked with a credential-check error 5.8s
mcp/mock-llm-mcp-slack-credentials.spec.ts › MCP Test Connection credential verification (Slack) › install: a valid token missing only a scope still installs (missing_scope is not a credential failure) 5.9s
mcp/mock-llm-mcp-slack-credentials.spec.ts › MCP Test Connection credential verification (Slack) › install: an older agent server that omits tool_result still installs (compat) 6.0s
mcp/mock-llm-mcp-slack-credentials.spec.ts › MCP Test Connection credential verification (Slack) › edit: Test Connection verifies the stored credentials and surfaces a credential failure 5.9s
mcp/mock-llm-mcp-slack-credentials.spec.ts › MCP Test Connection credential verification (Slack) › edit: Test Connection reports success for valid stored credentials 5.8s
mcp/mock-llm-mcp-slack-credentials.spec.ts › MCP Test Connection credential verification (Slack) › custom (non-catalog) server: Test Connection attaches no verification probe 5.8s
⏱️ onboarding/mock-llm-onboarding-happy-path.spec.ts › onboarding happy path › completes the full onboarding flow and launches a conversation 120.1s
onboarding/mock-llm-onboarding-regressions.spec.ts › onboarding recent regressions › keeps the modal open on backdrop click and Escape 1.6s
onboarding/mock-llm-onboarding-regressions.spec.ts › onboarding recent regressions › defaults the LLM setup step to OpenAI GPT-5.5 11.4s
regressions/mock-llm-ui-regressions.spec.ts › UI regressions › scopes standalone styles to the agent-server-ui shell 1.5s
regressions/mock-llm-ui-regressions.spec.ts › UI regressions › renders critic results on agent messages and finish actions 1.6s
regressions/mock-llm-ui-regressions.spec.ts › UI regressions › loads older events when scrolling up 1.7s
regressions/mock-llm-ui-regressions.spec.ts › UI regressions › selected workspace persists after navigating away and returning 2.3s
regressions/mock-llm-ui-regressions.spec.ts › UI regressions › cleared sessionStorage yields empty workspace selection 1.5s
settings/mock-llm-acp-agent.spec.ts › mock-LLM ACP agent conversation › step 1: configure ACP agent via Settings → Agent UI 14.0s
settings/mock-llm-acp-agent.spec.ts › mock-LLM ACP agent conversation › step 2: reload and verify ACP settings are persisted in UI 5.6s
settings/mock-llm-acp-agent.spec.ts › mock-LLM ACP agent conversation › step 3: start ACP conversation and verify agent reply 6.8s
settings/mock-llm-acp-agent.spec.ts › mock-LLM ACP agent conversation › step 4: resume ACP conversation from sidebar after navigating away 5.8s
settings/mock-llm-model-switch.spec.ts › mock-LLM /model slash command › step 1: configure LLM, create switch-target profile, register trajectory 15.0s
⏭️ settings/mock-llm-model-switch.spec.ts › mock-LLM /model slash command › step 2: start conversation, switch profile via /model, verify switch 0ms
settings/mock-llm-profile-management.spec.ts › active profile deletion + reconciliation › active profile is deletable and reconciliation activates another profile 0ms
settings/mock-llm-profile-management.spec.ts › same-model profile identity › chat header shows the correct profile when two profiles share the same model 0ms
settings/mock-llm-profile-management.spec.ts › OpenHands provider hidden base_url preservation › re-saving an OpenHands profile from Basic view preserves hidden base_url 0ms
skills/mock-llm-skills.spec.ts › skill loading: project, user, and deletion › project skill in workspace/.agents/skills/ triggers on matching keyword 0ms
skills/mock-llm-skills.spec.ts › skill loading: project, user, and deletion › user skill in ~/.openhands/skills/ triggers on matching keyword 0ms
skills/mock-llm-skills.spec.ts › skill loading: project, user, and deletion › deleting a user skill removes it from subsequent conversations 0ms
🔍 Failure details (3)

❌ backends/mock-llm-auth-modes.spec.ts › auth mode: public gate › rejects an incorrect key with an inline error

Error: expect(locator).toBeVisible() failed

Locator: getByTestId('api-key-entry-screen')
Expected: visible
Timeout: 30000ms
Error: element(s) not found

Call log:
  - Expect "toBeVisible" with timeout 30000ms
  - waiting for getByTestId('api-key-entry-screen')

⏱️ onboarding/mock-llm-onboarding-happy-path.spec.ts › onboarding happy path › completes the full onboarding flow and launches a conversation

Test timeout of 120000ms exceeded.

❌ onboarding/mock-llm-onboarding-regressions.spec.ts › onboarding recent regressions › defaults the LLM setup step to OpenAI GPT-5.5

Error: onboarding backend health probe should report connected

expect(locator).toBeVisible() failed

Locator: getByTestId('onboarding-backend-connected')
Expected: visible
Timeout: 10000ms
Error: element(s) not found

Call log:
  - onboarding backend health probe should report connected with timeout 10000ms
  - waiting for getByTestId('onboarding-backend-connected')

Posted by the Mock-LLM E2E workflow · results are deterministic (scripted LLM responses)

@github-actions

Copy link
Copy Markdown
Contributor

⚠️ Mock-LLM Docker E2E Test Results

0/0 passed

Commit: 6eb36f5d · Workflow run

Status Test Duration

Posted by the Mock-LLM E2E workflow · results are deterministic (scripted LLM responses)

@github-actions

Copy link
Copy Markdown
Contributor

❌ Mock-LLM E2E Tests

57/60 passed · 2 failed · 1 skipped

Commit: 04b5d244 · Workflow run · Test artifacts

Status Test Duration
automations/mock-llm-automation.spec.ts › mock-LLM automation lifecycle › step 1: setup LLM profile and register automation trajectory 7.5s
automations/mock-llm-automation.spec.ts › mock-LLM automation lifecycle › step 2: create automation and dispatch run via the UI 27.5s
automations/mock-llm-automation.spec.ts › mock-LLM automation lifecycle › step 3: verify automation and run on the automations page 6.3s
automations/mock-llm-preset-automation.spec.ts › preset automation → slash command conversation › automation card sends the correct slash command to a conversation 16.1s
automations/mock-llm-preset-automation.spec.ts › preset automation → slash command conversation › direct slash command from home page triggers skill activation 13.8s
backends/mock-llm-auth-modes.spec.ts › auth mode: fresh install with runtime-injected key › reaches the onboarding modal without pre-seeded localStorage 1.4s
backends/mock-llm-auth-modes.spec.ts › auth mode: non-public key rotation › recovers when localStorage has a stale session API key 5.4s
backends/mock-llm-auth-modes.spec.ts › auth mode: public gate › shows first-run onboarding before the auth screen when no key is configured 1.4s
backends/mock-llm-auth-modes.spec.ts › auth mode: public gate › rejects an incorrect key with an inline error 1.7s
backends/mock-llm-auth-modes.spec.ts › auth mode: public gate › allows access after pasting the correct key 1.9s
backends/mock-llm-auth-modes.spec.ts › auth mode: public gate › skips auth screen for returning user with valid stored key 780ms
backends/mock-llm-auth-modes.spec.ts › auth mode: public gate › re-prompts when the server rotates its key (stale localStorage) 3.8s
backends/mock-llm-cross-connect.spec.ts › cross-connect: frontend-only → backend-only › frontend-only connects to a separate backend-only instance 16.2s
backends/mock-llm-cross-connect.spec.ts › cross-connect: frontend-only → multiple backends › connects to two separate backends and switches between them 20.8s
backends/mock-llm-partial-stack.spec.ts › partial stack: --frontend-only › serves the frontend but returns 503 for backend routes 7.7s
backends/mock-llm-partial-stack.spec.ts › partial stack: --backend-only › serves backend APIs but returns 503 for the frontend root 13.1s
backends/mock-llm-partial-stack.spec.ts › partial stack: port conflict › fails with a clear error when the ingress port is occupied 107ms
backends/mock-llm-partial-stack.spec.ts › partial stack: port conflict › starts successfully on a free port after a conflict 6.0s
conversations/mock-llm-conversation.spec.ts › mock-LLM agent-server conversation › step 1: create an LLM profile pointing at the mock LLM server 6.3s
conversations/mock-llm-conversation.spec.ts › mock-LLM agent-server conversation › step 2: activate the mock-llm profile and verify settings API 6.2s
conversations/mock-llm-conversation.spec.ts › mock-LLM agent-server conversation › step 3: run a conversation with the mock LLM 7.3s
conversations/mock-llm-conversation.spec.ts › mock-LLM agent-server conversation › step 4: resume conversation from sidebar after navigating away 5.8s
conversations/mock-llm-image-upload.spec.ts › mock-LLM image upload › attaching an image embeds it as base64 in the LLM completion call 13.8s
files/mock-llm-files-and-git.spec.ts › files tab, git control bar, and browser tab › step 1: ensure mock LLM profile is configured 7.4s
files/mock-llm-files-and-git.spec.ts › files tab, git control bar, and browser tab › step 2: start conversation and attach workspace metadata 12.5s
files/mock-llm-files-and-git.spec.ts › files tab, git control bar, and browser tab › step 3: git control bar shows workspace pill and git actions 25.3s
files/mock-llm-files-and-git.spec.ts › files tab, git control bar, and browser tab › step 4: files tab defaults to diff view for attached workspace 5.9s
files/mock-llm-files-and-git.spec.ts › files tab, git control bar, and browser tab › step 5: browser tab shows empty state 6.3s
files/mock-llm-files-and-git.spec.ts › files tab, git control bar, and browser tab › step 6: files tab defaults to file-tree view without attached workspace 8.3s
home/mock-llm-folder-workspace.spec.ts › mock-LLM folder browser → workspace → conversation › step 1: browse to a folder, add it as a workspace, and launch a conversation with the correct working_dir 8.7s
mcp/mock-llm-mcp-github.spec.ts › MCP GitHub server install flow › step 1: GitHub card is visible on the MCP marketplace page 5.6s
mcp/mock-llm-mcp-github.spec.ts › MCP GitHub server install flow › step 2: clicking GitHub card opens the install modal with correct fields 5.8s
mcp/mock-llm-mcp-github.spec.ts › MCP GitHub server install flow › step 3: full install flow — fill PAT, submit, verify installed 13.0s
mcp/mock-llm-mcp-github.spec.ts › MCP GitHub server install flow › step 4: installed GitHub server can be deleted 7.4s
mcp/mock-llm-mcp-slack-credentials.spec.ts › MCP Test Connection credential verification (Slack) › install: invalid Slack credentials are blocked with a credential-check error 5.8s
mcp/mock-llm-mcp-slack-credentials.spec.ts › MCP Test Connection credential verification (Slack) › install: a valid token missing only a scope still installs (missing_scope is not a credential failure) 6.1s
mcp/mock-llm-mcp-slack-credentials.spec.ts › MCP Test Connection credential verification (Slack) › install: an older agent server that omits tool_result still installs (compat) 5.9s
mcp/mock-llm-mcp-slack-credentials.spec.ts › MCP Test Connection credential verification (Slack) › edit: Test Connection verifies the stored credentials and surfaces a credential failure 5.8s
mcp/mock-llm-mcp-slack-credentials.spec.ts › MCP Test Connection credential verification (Slack) › edit: Test Connection reports success for valid stored credentials 5.8s
mcp/mock-llm-mcp-slack-credentials.spec.ts › MCP Test Connection credential verification (Slack) › custom (non-catalog) server: Test Connection attaches no verification probe 6.4s
⏱️ onboarding/mock-llm-onboarding-happy-path.spec.ts › onboarding happy path › completes the full onboarding flow and launches a conversation 120.1s
onboarding/mock-llm-onboarding-regressions.spec.ts › onboarding recent regressions › keeps the modal open on backdrop click and Escape 1.5s
onboarding/mock-llm-onboarding-regressions.spec.ts › onboarding recent regressions › defaults the LLM setup step to OpenAI GPT-5.5 11.4s
regressions/mock-llm-ui-regressions.spec.ts › UI regressions › scopes standalone styles to the agent-server-ui shell 1.6s
regressions/mock-llm-ui-regressions.spec.ts › UI regressions › renders critic results on agent messages and finish actions 1.5s
regressions/mock-llm-ui-regressions.spec.ts › UI regressions › loads older events when scrolling up 1.8s
regressions/mock-llm-ui-regressions.spec.ts › UI regressions › selected workspace persists after navigating away and returning 2.5s
regressions/mock-llm-ui-regressions.spec.ts › UI regressions › cleared sessionStorage yields empty workspace selection 1.0s
settings/mock-llm-acp-agent.spec.ts › mock-LLM ACP agent conversation › step 1: configure ACP agent via Settings → Agent UI 14.0s
settings/mock-llm-acp-agent.spec.ts › mock-LLM ACP agent conversation › step 2: reload and verify ACP settings are persisted in UI 5.6s
settings/mock-llm-acp-agent.spec.ts › mock-LLM ACP agent conversation › step 3: start ACP conversation and verify agent reply 6.9s
settings/mock-llm-acp-agent.spec.ts › mock-LLM ACP agent conversation › step 4: resume ACP conversation from sidebar after navigating away 5.8s
settings/mock-llm-model-switch.spec.ts › mock-LLM /model slash command › step 1: configure LLM, create switch-target profile, register trajectory 13.4s
settings/mock-llm-model-switch.spec.ts › mock-LLM /model slash command › step 2: start conversation, switch profile via /model, verify switch 8.2s
settings/mock-llm-profile-management.spec.ts › active profile deletion + reconciliation › active profile is deletable and reconciliation activates another profile 8.8s
⏭️ settings/mock-llm-profile-management.spec.ts › same-model profile identity › chat header shows the correct profile when two profiles share the same model 0ms
settings/mock-llm-profile-management.spec.ts › OpenHands provider hidden base_url preservation › re-saving an OpenHands profile from Basic view preserves hidden base_url 0ms
skills/mock-llm-skills.spec.ts › skill loading: project, user, and deletion › project skill in workspace/.agents/skills/ triggers on matching keyword 0ms
skills/mock-llm-skills.spec.ts › skill loading: project, user, and deletion › user skill in ~/.openhands/skills/ triggers on matching keyword 0ms
skills/mock-llm-skills.spec.ts › skill loading: project, user, and deletion › deleting a user skill removes it from subsequent conversations 0ms
🔍 Failure details (2)

⏱️ onboarding/mock-llm-onboarding-happy-path.spec.ts › onboarding happy path › completes the full onboarding flow and launches a conversation

Test timeout of 120000ms exceeded.

❌ onboarding/mock-llm-onboarding-regressions.spec.ts › onboarding recent regressions › defaults the LLM setup step to OpenAI GPT-5.5

Error: onboarding backend health probe should report connected

expect(locator).toBeVisible() failed

Locator: getByTestId('onboarding-backend-connected')
Expected: visible
Timeout: 10000ms
Error: element(s) not found

Call log:
  - onboarding backend health probe should report connected with timeout 10000ms
  - waiting for getByTestId('onboarding-backend-connected')

Posted by the Mock-LLM E2E workflow · results are deterministic (scripted LLM responses)

@github-actions

Copy link
Copy Markdown
Contributor

⚠️ Mock-LLM Docker E2E Test Results

0/0 passed

Commit: 04b5d244 · Workflow run

Status Test Duration

Posted by the Mock-LLM E2E workflow · results are deterministic (scripted LLM responses)

@github-actions

Copy link
Copy Markdown
Contributor

✅ Mock-LLM E2E Tests

60/60 passed

Commit: 6f994e3b · Workflow run · Test artifacts

Status Test Duration
automations/mock-llm-automation.spec.ts › mock-LLM automation lifecycle › step 1: setup LLM profile and register automation trajectory 7.2s
automations/mock-llm-automation.spec.ts › mock-LLM automation lifecycle › step 2: create automation and dispatch run via the UI 28.5s
automations/mock-llm-automation.spec.ts › mock-LLM automation lifecycle › step 3: verify automation and run on the automations page 6.2s
automations/mock-llm-preset-automation.spec.ts › preset automation → slash command conversation › automation card sends the correct slash command to a conversation 16.5s
automations/mock-llm-preset-automation.spec.ts › preset automation → slash command conversation › direct slash command from home page triggers skill activation 13.7s
backends/mock-llm-auth-modes.spec.ts › auth mode: fresh install with runtime-injected key › reaches the onboarding modal without pre-seeded localStorage 1.3s
backends/mock-llm-auth-modes.spec.ts › auth mode: non-public key rotation › recovers when localStorage has a stale session API key 5.3s
backends/mock-llm-auth-modes.spec.ts › auth mode: public gate › shows first-run onboarding before the auth screen when no key is configured 1.4s
backends/mock-llm-auth-modes.spec.ts › auth mode: public gate › rejects an incorrect key with an inline error 1.7s
backends/mock-llm-auth-modes.spec.ts › auth mode: public gate › allows access after pasting the correct key 1.8s
backends/mock-llm-auth-modes.spec.ts › auth mode: public gate › skips auth screen for returning user with valid stored key 755ms
backends/mock-llm-auth-modes.spec.ts › auth mode: public gate › re-prompts when the server rotates its key (stale localStorage) 1.5s
backends/mock-llm-cross-connect.spec.ts › cross-connect: frontend-only → backend-only › frontend-only connects to a separate backend-only instance 16.0s
backends/mock-llm-cross-connect.spec.ts › cross-connect: frontend-only → multiple backends › connects to two separate backends and switches between them 19.8s
backends/mock-llm-partial-stack.spec.ts › partial stack: --frontend-only › serves the frontend but returns 503 for backend routes 7.4s
backends/mock-llm-partial-stack.spec.ts › partial stack: --backend-only › serves backend APIs but returns 503 for the frontend root 13.1s
backends/mock-llm-partial-stack.spec.ts › partial stack: port conflict › fails with a clear error when the ingress port is occupied 106ms
backends/mock-llm-partial-stack.spec.ts › partial stack: port conflict › starts successfully on a free port after a conflict 6.0s
conversations/mock-llm-conversation.spec.ts › mock-LLM agent-server conversation › step 1: create an LLM profile pointing at the mock LLM server 6.3s
conversations/mock-llm-conversation.spec.ts › mock-LLM agent-server conversation › step 2: activate the mock-llm profile and verify settings API 6.3s
conversations/mock-llm-conversation.spec.ts › mock-LLM agent-server conversation › step 3: run a conversation with the mock LLM 6.6s
conversations/mock-llm-conversation.spec.ts › mock-LLM agent-server conversation › step 4: resume conversation from sidebar after navigating away 5.9s
conversations/mock-llm-image-upload.spec.ts › mock-LLM image upload › attaching an image embeds it as base64 in the LLM completion call 14.0s
files/mock-llm-files-and-git.spec.ts › files tab, git control bar, and browser tab › step 1: ensure mock LLM profile is configured 7.2s
files/mock-llm-files-and-git.spec.ts › files tab, git control bar, and browser tab › step 2: start conversation and attach workspace metadata 11.7s
files/mock-llm-files-and-git.spec.ts › files tab, git control bar, and browser tab › step 3: git control bar shows workspace pill and git actions 25.3s
files/mock-llm-files-and-git.spec.ts › files tab, git control bar, and browser tab › step 4: files tab defaults to diff view for attached workspace 5.9s
files/mock-llm-files-and-git.spec.ts › files tab, git control bar, and browser tab › step 5: browser tab shows empty state 6.3s
files/mock-llm-files-and-git.spec.ts › files tab, git control bar, and browser tab › step 6: files tab defaults to file-tree view without attached workspace 7.5s
home/mock-llm-folder-workspace.spec.ts › mock-LLM folder browser → workspace → conversation › step 1: browse to a folder, add it as a workspace, and launch a conversation with the correct working_dir 7.8s
mcp/mock-llm-mcp-github.spec.ts › MCP GitHub server install flow › step 1: GitHub card is visible on the MCP marketplace page 5.5s
mcp/mock-llm-mcp-github.spec.ts › MCP GitHub server install flow › step 2: clicking GitHub card opens the install modal with correct fields 5.8s
mcp/mock-llm-mcp-github.spec.ts › MCP GitHub server install flow › step 3: full install flow — fill PAT, submit, verify installed 13.0s
mcp/mock-llm-mcp-github.spec.ts › MCP GitHub server install flow › step 4: installed GitHub server can be deleted 5.8s
mcp/mock-llm-mcp-slack-credentials.spec.ts › MCP Test Connection credential verification (Slack) › install: invalid Slack credentials are blocked with a credential-check error 5.8s
mcp/mock-llm-mcp-slack-credentials.spec.ts › MCP Test Connection credential verification (Slack) › install: a valid token missing only a scope still installs (missing_scope is not a credential failure) 5.9s
mcp/mock-llm-mcp-slack-credentials.spec.ts › MCP Test Connection credential verification (Slack) › install: an older agent server that omits tool_result still installs (compat) 5.9s
mcp/mock-llm-mcp-slack-credentials.spec.ts › MCP Test Connection credential verification (Slack) › edit: Test Connection verifies the stored credentials and surfaces a credential failure 5.8s
mcp/mock-llm-mcp-slack-credentials.spec.ts › MCP Test Connection credential verification (Slack) › edit: Test Connection reports success for valid stored credentials 5.8s
mcp/mock-llm-mcp-slack-credentials.spec.ts › MCP Test Connection credential verification (Slack) › custom (non-catalog) server: Test Connection attaches no verification probe 5.8s
onboarding/mock-llm-onboarding-happy-path.spec.ts › onboarding happy path › completes the full onboarding flow and launches a conversation 4.4s
onboarding/mock-llm-onboarding-regressions.spec.ts › onboarding recent regressions › keeps the modal open on backdrop click and Escape 1.4s
onboarding/mock-llm-onboarding-regressions.spec.ts › onboarding recent regressions › defaults the LLM setup step to OpenAI GPT-5.5 1.5s
regressions/mock-llm-ui-regressions.spec.ts › UI regressions › scopes standalone styles to the agent-server-ui shell 1.4s
regressions/mock-llm-ui-regressions.spec.ts › UI regressions › renders critic results on agent messages and finish actions 1.5s
regressions/mock-llm-ui-regressions.spec.ts › UI regressions › loads older events when scrolling up 1.7s
regressions/mock-llm-ui-regressions.spec.ts › UI regressions › selected workspace persists after navigating away and returning 2.5s
regressions/mock-llm-ui-regressions.spec.ts › UI regressions › cleared sessionStorage yields empty workspace selection 1.4s
settings/mock-llm-acp-agent.spec.ts › mock-LLM ACP agent conversation › step 1: configure ACP agent via Settings → Agent UI 13.9s
settings/mock-llm-acp-agent.spec.ts › mock-LLM ACP agent conversation › step 2: reload and verify ACP settings are persisted in UI 5.6s
settings/mock-llm-acp-agent.spec.ts › mock-LLM ACP agent conversation › step 3: start ACP conversation and verify agent reply 6.8s
settings/mock-llm-acp-agent.spec.ts › mock-LLM ACP agent conversation › step 4: resume ACP conversation from sidebar after navigating away 5.7s
settings/mock-llm-model-switch.spec.ts › mock-LLM /model slash command › step 1: configure LLM, create switch-target profile, register trajectory 13.2s
settings/mock-llm-model-switch.spec.ts › mock-LLM /model slash command › step 2: start conversation, switch profile via /model, verify switch 6.9s
settings/mock-llm-profile-management.spec.ts › active profile deletion + reconciliation › active profile is deletable and reconciliation activates another profile 9.1s
settings/mock-llm-profile-management.spec.ts › same-model profile identity › chat header shows the correct profile when two profiles share the same model 15.9s
settings/mock-llm-profile-management.spec.ts › OpenHands provider hidden base_url preservation › re-saving an OpenHands profile from Basic view preserves hidden base_url 7.5s
skills/mock-llm-skills.spec.ts › skill loading: project, user, and deletion › project skill in workspace/.agents/skills/ triggers on matching keyword 13.9s
skills/mock-llm-skills.spec.ts › skill loading: project, user, and deletion › user skill in ~/.openhands/skills/ triggers on matching keyword 13.5s
skills/mock-llm-skills.spec.ts › skill loading: project, user, and deletion › deleting a user skill removes it from subsequent conversations 15.6s

Posted by the Mock-LLM E2E workflow · results are deterministic (scripted LLM responses)

@github-actions

Copy link
Copy Markdown
Contributor

⚠️ Mock-LLM Docker E2E Test Results

0/0 passed

Commit: 6f994e3b · Workflow run

Status Test Duration

Posted by the Mock-LLM E2E workflow · results are deterministic (scripted LLM responses)

@github-actions

Copy link
Copy Markdown
Contributor

✅ Mock-LLM Docker E2E Test Results

60/60 passed

Commit: 6f994e3b · Workflow run · Test artifacts

Status Test Duration
automations/mock-llm-automation.spec.ts › mock-LLM automation lifecycle › step 1: setup LLM profile and register automation trajectory 7.3s
automations/mock-llm-automation.spec.ts › mock-LLM automation lifecycle › step 2: create automation and dispatch run via the UI 31.5s
automations/mock-llm-automation.spec.ts › mock-LLM automation lifecycle › step 3: verify automation and run on the automations page 6.2s
automations/mock-llm-preset-automation.spec.ts › preset automation → slash command conversation › automation card sends the correct slash command to a conversation 15.8s
automations/mock-llm-preset-automation.spec.ts › preset automation → slash command conversation › direct slash command from home page triggers skill activation 13.2s
backends/mock-llm-auth-modes.spec.ts › auth mode: fresh install with runtime-injected key › reaches the onboarding modal without pre-seeded localStorage 1.3s
backends/mock-llm-auth-modes.spec.ts › auth mode: non-public key rotation › recovers when localStorage has a stale session API key 5.3s
backends/mock-llm-auth-modes.spec.ts › auth mode: public gate › shows first-run onboarding before the auth screen when no key is configured 1.5s
backends/mock-llm-auth-modes.spec.ts › auth mode: public gate › rejects an incorrect key with an inline error 1.6s
backends/mock-llm-auth-modes.spec.ts › auth mode: public gate › allows access after pasting the correct key 1.8s
backends/mock-llm-auth-modes.spec.ts › auth mode: public gate › skips auth screen for returning user with valid stored key 766ms
backends/mock-llm-auth-modes.spec.ts › auth mode: public gate › re-prompts when the server rotates its key (stale localStorage) 1.5s
backends/mock-llm-cross-connect.spec.ts › cross-connect: frontend-only → backend-only › frontend-only connects to a separate backend-only instance 22.1s
backends/mock-llm-cross-connect.spec.ts › cross-connect: frontend-only → multiple backends › connects to two separate backends and switches between them 20.8s
backends/mock-llm-partial-stack.spec.ts › partial stack: --frontend-only › serves the frontend but returns 503 for backend routes 7.4s
backends/mock-llm-partial-stack.spec.ts › partial stack: --backend-only › serves backend APIs but returns 503 for the frontend root 13.1s
backends/mock-llm-partial-stack.spec.ts › partial stack: port conflict › fails with a clear error when the ingress port is occupied 103ms
backends/mock-llm-partial-stack.spec.ts › partial stack: port conflict › starts successfully on a free port after a conflict 6.0s
conversations/mock-llm-conversation.spec.ts › mock-LLM agent-server conversation › step 1: create an LLM profile pointing at the mock LLM server 6.3s
conversations/mock-llm-conversation.spec.ts › mock-LLM agent-server conversation › step 2: activate the mock-llm profile and verify settings API 6.5s
conversations/mock-llm-conversation.spec.ts › mock-LLM agent-server conversation › step 3: run a conversation with the mock LLM 6.3s
conversations/mock-llm-conversation.spec.ts › mock-LLM agent-server conversation › step 4: resume conversation from sidebar after navigating away 5.7s
conversations/mock-llm-image-upload.spec.ts › mock-LLM image upload › attaching an image embeds it as base64 in the LLM completion call 13.5s
files/mock-llm-files-and-git.spec.ts › files tab, git control bar, and browser tab › step 1: ensure mock LLM profile is configured 7.5s
files/mock-llm-files-and-git.spec.ts › files tab, git control bar, and browser tab › step 2: start conversation and attach workspace metadata 11.4s
files/mock-llm-files-and-git.spec.ts › files tab, git control bar, and browser tab › step 3: git control bar shows workspace pill and git actions 25.3s
files/mock-llm-files-and-git.spec.ts › files tab, git control bar, and browser tab › step 4: files tab defaults to diff view for attached workspace 5.9s
files/mock-llm-files-and-git.spec.ts › files tab, git control bar, and browser tab › step 5: browser tab shows empty state 6.7s
files/mock-llm-files-and-git.spec.ts › files tab, git control bar, and browser tab › step 6: files tab defaults to file-tree view without attached workspace 7.5s
home/mock-llm-folder-workspace.spec.ts › mock-LLM folder browser → workspace → conversation › step 1: browse to a folder, add it as a workspace, and launch a conversation with the correct working_dir 7.3s
mcp/mock-llm-mcp-github.spec.ts › MCP GitHub server install flow › step 1: GitHub card is visible on the MCP marketplace page 5.5s
mcp/mock-llm-mcp-github.spec.ts › MCP GitHub server install flow › step 2: clicking GitHub card opens the install modal with correct fields 5.7s
mcp/mock-llm-mcp-github.spec.ts › MCP GitHub server install flow › step 3: full install flow — fill PAT, submit, verify installed 13.0s
mcp/mock-llm-mcp-github.spec.ts › MCP GitHub server install flow › step 4: installed GitHub server can be deleted 5.8s
mcp/mock-llm-mcp-slack-credentials.spec.ts › MCP Test Connection credential verification (Slack) › install: invalid Slack credentials are blocked with a credential-check error 5.8s
mcp/mock-llm-mcp-slack-credentials.spec.ts › MCP Test Connection credential verification (Slack) › install: a valid token missing only a scope still installs (missing_scope is not a credential failure) 6.0s
mcp/mock-llm-mcp-slack-credentials.spec.ts › MCP Test Connection credential verification (Slack) › install: an older agent server that omits tool_result still installs (compat) 5.9s
mcp/mock-llm-mcp-slack-credentials.spec.ts › MCP Test Connection credential verification (Slack) › edit: Test Connection verifies the stored credentials and surfaces a credential failure 5.8s
mcp/mock-llm-mcp-slack-credentials.spec.ts › MCP Test Connection credential verification (Slack) › edit: Test Connection reports success for valid stored credentials 5.8s
mcp/mock-llm-mcp-slack-credentials.spec.ts › MCP Test Connection credential verification (Slack) › custom (non-catalog) server: Test Connection attaches no verification probe 5.9s
onboarding/mock-llm-onboarding-happy-path.spec.ts › onboarding happy path › completes the full onboarding flow and launches a conversation 3.5s
onboarding/mock-llm-onboarding-regressions.spec.ts › onboarding recent regressions › keeps the modal open on backdrop click and Escape 1.4s
onboarding/mock-llm-onboarding-regressions.spec.ts › onboarding recent regressions › defaults the LLM setup step to OpenAI GPT-5.5 1.5s
regressions/mock-llm-ui-regressions.spec.ts › UI regressions › scopes standalone styles to the agent-server-ui shell 2.9s
regressions/mock-llm-ui-regressions.spec.ts › UI regressions › renders critic results on agent messages and finish actions 1.6s
regressions/mock-llm-ui-regressions.spec.ts › UI regressions › loads older events when scrolling up 1.6s
regressions/mock-llm-ui-regressions.spec.ts › UI regressions › selected workspace persists after navigating away and returning 2.5s
regressions/mock-llm-ui-regressions.spec.ts › UI regressions › cleared sessionStorage yields empty workspace selection 1.3s
settings/mock-llm-acp-agent.spec.ts › mock-LLM ACP agent conversation › step 1: configure ACP agent via Settings → Agent UI 14.2s
settings/mock-llm-acp-agent.spec.ts › mock-LLM ACP agent conversation › step 2: reload and verify ACP settings are persisted in UI 5.6s
settings/mock-llm-acp-agent.spec.ts › mock-LLM ACP agent conversation › step 3: start ACP conversation and verify agent reply 6.7s
settings/mock-llm-acp-agent.spec.ts › mock-LLM ACP agent conversation › step 4: resume ACP conversation from sidebar after navigating away 5.7s
settings/mock-llm-model-switch.spec.ts › mock-LLM /model slash command › step 1: configure LLM, create switch-target profile, register trajectory 13.1s
settings/mock-llm-model-switch.spec.ts › mock-LLM /model slash command › step 2: start conversation, switch profile via /model, verify switch 6.7s
settings/mock-llm-profile-management.spec.ts › active profile deletion + reconciliation › active profile is deletable and reconciliation activates another profile 8.5s
settings/mock-llm-profile-management.spec.ts › same-model profile identity › chat header shows the correct profile when two profiles share the same model 15.0s
settings/mock-llm-profile-management.spec.ts › OpenHands provider hidden base_url preservation › re-saving an OpenHands profile from Basic view preserves hidden base_url 7.4s
skills/mock-llm-skills.spec.ts › skill loading: project, user, and deletion › project skill in workspace/.agents/skills/ triggers on matching keyword 13.8s
skills/mock-llm-skills.spec.ts › skill loading: project, user, and deletion › user skill in ~/.openhands/skills/ triggers on matching keyword 13.3s
skills/mock-llm-skills.spec.ts › skill loading: project, user, and deletion › deleting a user skill removes it from subsequent conversations 13.4s

Posted by the Mock-LLM E2E workflow · results are deterministic (scripted LLM responses)

@neubig neubig added the qa-this Trigger OpenHands QA validation label Jun 17, 2026
@neubig neubig force-pushed the fix-public-onboarding branch from e81bcd6 to 012625d Compare June 17, 2026 17:36
@neubig neubig force-pushed the lock-to-cloud-backend branch from 6f994e3 to 42af044 Compare June 17, 2026 17:36
@neubig neubig added qa-this Trigger OpenHands QA validation and removed qa-this Trigger OpenHands QA validation labels Jun 17, 2026
@github-actions

Copy link
Copy Markdown
Contributor

✅ Mock-LLM E2E Tests

60/60 passed

Commit: 04958f3d · Workflow run · Test artifacts

Status Test Duration
automations/mock-llm-automation.spec.ts › mock-LLM automation lifecycle › step 1: setup LLM profile and register automation trajectory 7.4s
automations/mock-llm-automation.spec.ts › mock-LLM automation lifecycle › step 2: create automation and dispatch run via the UI 28.5s
automations/mock-llm-automation.spec.ts › mock-LLM automation lifecycle › step 3: verify automation and run on the automations page 6.1s
automations/mock-llm-preset-automation.spec.ts › preset automation → slash command conversation › automation card sends the correct slash command to a conversation 16.5s
automations/mock-llm-preset-automation.spec.ts › preset automation → slash command conversation › direct slash command from home page triggers skill activation 13.8s
backends/mock-llm-auth-modes.spec.ts › auth mode: fresh install with runtime-injected key › reaches the onboarding modal without pre-seeded localStorage 1.4s
backends/mock-llm-auth-modes.spec.ts › auth mode: non-public key rotation › recovers when localStorage has a stale session API key 5.3s
backends/mock-llm-auth-modes.spec.ts › auth mode: public gate › shows first-run onboarding before the auth screen when no key is configured 1.4s
backends/mock-llm-auth-modes.spec.ts › auth mode: public gate › rejects an incorrect key with an inline error 1.6s
backends/mock-llm-auth-modes.spec.ts › auth mode: public gate › allows access after pasting the correct key 1.8s
backends/mock-llm-auth-modes.spec.ts › auth mode: public gate › skips auth screen for returning user with valid stored key 834ms
backends/mock-llm-auth-modes.spec.ts › auth mode: public gate › re-prompts when the server rotates its key (stale localStorage) 1.5s
backends/mock-llm-cross-connect.spec.ts › cross-connect: frontend-only → backend-only › frontend-only connects to a separate backend-only instance 15.9s
backends/mock-llm-cross-connect.spec.ts › cross-connect: frontend-only → multiple backends › connects to two separate backends and switches between them 20.6s
backends/mock-llm-partial-stack.spec.ts › partial stack: --frontend-only › serves the frontend but returns 503 for backend routes 7.4s
backends/mock-llm-partial-stack.spec.ts › partial stack: --backend-only › serves backend APIs but returns 503 for the frontend root 14.1s
backends/mock-llm-partial-stack.spec.ts › partial stack: port conflict › fails with a clear error when the ingress port is occupied 108ms
backends/mock-llm-partial-stack.spec.ts › partial stack: port conflict › starts successfully on a free port after a conflict 6.0s
conversations/mock-llm-conversation.spec.ts › mock-LLM agent-server conversation › step 1: create an LLM profile pointing at the mock LLM server 6.3s
conversations/mock-llm-conversation.spec.ts › mock-LLM agent-server conversation › step 2: activate the mock-llm profile and verify settings API 6.2s
conversations/mock-llm-conversation.spec.ts › mock-LLM agent-server conversation › step 3: run a conversation with the mock LLM 6.6s
conversations/mock-llm-conversation.spec.ts › mock-LLM agent-server conversation › step 4: resume conversation from sidebar after navigating away 5.7s
conversations/mock-llm-image-upload.spec.ts › mock-LLM image upload › attaching an image embeds it as base64 in the LLM completion call 15.7s
files/mock-llm-files-and-git.spec.ts › files tab, git control bar, and browser tab › step 1: ensure mock LLM profile is configured 7.3s
files/mock-llm-files-and-git.spec.ts › files tab, git control bar, and browser tab › step 2: start conversation and attach workspace metadata 12.5s
files/mock-llm-files-and-git.spec.ts › files tab, git control bar, and browser tab › step 3: git control bar shows workspace pill and git actions 26.2s
files/mock-llm-files-and-git.spec.ts › files tab, git control bar, and browser tab › step 4: files tab defaults to diff view for attached workspace 5.9s
files/mock-llm-files-and-git.spec.ts › files tab, git control bar, and browser tab › step 5: browser tab shows empty state 6.3s
files/mock-llm-files-and-git.spec.ts › files tab, git control bar, and browser tab › step 6: files tab defaults to file-tree view without attached workspace 7.6s
home/mock-llm-folder-workspace.spec.ts › mock-LLM folder browser → workspace → conversation › step 1: browse to a folder, add it as a workspace, and launch a conversation with the correct working_dir 7.8s
mcp/mock-llm-mcp-github.spec.ts › MCP GitHub server install flow › step 1: GitHub card is visible on the MCP marketplace page 5.5s
mcp/mock-llm-mcp-github.spec.ts › MCP GitHub server install flow › step 2: clicking GitHub card opens the install modal with correct fields 5.7s
mcp/mock-llm-mcp-github.spec.ts › MCP GitHub server install flow › step 3: full install flow — fill PAT, submit, verify installed 13.0s
mcp/mock-llm-mcp-github.spec.ts › MCP GitHub server install flow › step 4: installed GitHub server can be deleted 5.8s
mcp/mock-llm-mcp-slack-credentials.spec.ts › MCP Test Connection credential verification (Slack) › install: invalid Slack credentials are blocked with a credential-check error 5.8s
mcp/mock-llm-mcp-slack-credentials.spec.ts › MCP Test Connection credential verification (Slack) › install: a valid token missing only a scope still installs (missing_scope is not a credential failure) 5.9s
mcp/mock-llm-mcp-slack-credentials.spec.ts › MCP Test Connection credential verification (Slack) › install: an older agent server that omits tool_result still installs (compat) 7.4s
mcp/mock-llm-mcp-slack-credentials.spec.ts › MCP Test Connection credential verification (Slack) › edit: Test Connection verifies the stored credentials and surfaces a credential failure 5.8s
mcp/mock-llm-mcp-slack-credentials.spec.ts › MCP Test Connection credential verification (Slack) › edit: Test Connection reports success for valid stored credentials 5.8s
mcp/mock-llm-mcp-slack-credentials.spec.ts › MCP Test Connection credential verification (Slack) › custom (non-catalog) server: Test Connection attaches no verification probe 5.9s
onboarding/mock-llm-onboarding-happy-path.spec.ts › onboarding happy path › completes the full onboarding flow and launches a conversation 4.4s
onboarding/mock-llm-onboarding-regressions.spec.ts › onboarding recent regressions › keeps the modal open on backdrop click and Escape 2.0s
onboarding/mock-llm-onboarding-regressions.spec.ts › onboarding recent regressions › defaults the LLM setup step to OpenAI GPT-5.5 2.1s
regressions/mock-llm-ui-regressions.spec.ts › UI regressions › scopes standalone styles to the agent-server-ui shell 1.4s
regressions/mock-llm-ui-regressions.spec.ts › UI regressions › renders critic results on agent messages and finish actions 1.5s
regressions/mock-llm-ui-regressions.spec.ts › UI regressions › loads older events when scrolling up 1.6s
regressions/mock-llm-ui-regressions.spec.ts › UI regressions › selected workspace persists after navigating away and returning 3.6s
regressions/mock-llm-ui-regressions.spec.ts › UI regressions › cleared sessionStorage yields empty workspace selection 1.3s
settings/mock-llm-acp-agent.spec.ts › mock-LLM ACP agent conversation › step 1: configure ACP agent via Settings → Agent UI 13.8s
settings/mock-llm-acp-agent.spec.ts › mock-LLM ACP agent conversation › step 2: reload and verify ACP settings are persisted in UI 5.5s
settings/mock-llm-acp-agent.spec.ts › mock-LLM ACP agent conversation › step 3: start ACP conversation and verify agent reply 6.8s
settings/mock-llm-acp-agent.spec.ts › mock-LLM ACP agent conversation › step 4: resume ACP conversation from sidebar after navigating away 5.7s
settings/mock-llm-model-switch.spec.ts › mock-LLM /model slash command › step 1: configure LLM, create switch-target profile, register trajectory 13.6s
settings/mock-llm-model-switch.spec.ts › mock-LLM /model slash command › step 2: start conversation, switch profile via /model, verify switch 6.9s
settings/mock-llm-profile-management.spec.ts › active profile deletion + reconciliation › active profile is deletable and reconciliation activates another profile 8.5s
settings/mock-llm-profile-management.spec.ts › same-model profile identity › chat header shows the correct profile when two profiles share the same model 15.3s
settings/mock-llm-profile-management.spec.ts › OpenHands provider hidden base_url preservation › re-saving an OpenHands profile from Basic view preserves hidden base_url 7.3s
skills/mock-llm-skills.spec.ts › skill loading: project, user, and deletion › project skill in workspace/.agents/skills/ triggers on matching keyword 13.6s
skills/mock-llm-skills.spec.ts › skill loading: project, user, and deletion › user skill in ~/.openhands/skills/ triggers on matching keyword 15.2s
skills/mock-llm-skills.spec.ts › skill loading: project, user, and deletion › deleting a user skill removes it from subsequent conversations 15.7s

Posted by the Mock-LLM E2E workflow · results are deterministic (scripted LLM responses)

@github-actions

Copy link
Copy Markdown
Contributor

✅ Mock-LLM Docker E2E Test Results

60/60 passed

Commit: 04958f3d · Workflow run · Test artifacts

Status Test Duration
automations/mock-llm-automation.spec.ts › mock-LLM automation lifecycle › step 1: setup LLM profile and register automation trajectory 7.3s
automations/mock-llm-automation.spec.ts › mock-LLM automation lifecycle › step 2: create automation and dispatch run via the UI 31.5s
automations/mock-llm-automation.spec.ts › mock-LLM automation lifecycle › step 3: verify automation and run on the automations page 6.1s
automations/mock-llm-preset-automation.spec.ts › preset automation → slash command conversation › automation card sends the correct slash command to a conversation 16.1s
automations/mock-llm-preset-automation.spec.ts › preset automation → slash command conversation › direct slash command from home page triggers skill activation 13.4s
backends/mock-llm-auth-modes.spec.ts › auth mode: fresh install with runtime-injected key › reaches the onboarding modal without pre-seeded localStorage 1.3s
backends/mock-llm-auth-modes.spec.ts › auth mode: non-public key rotation › recovers when localStorage has a stale session API key 5.3s
backends/mock-llm-auth-modes.spec.ts › auth mode: public gate › shows first-run onboarding before the auth screen when no key is configured 1.4s
backends/mock-llm-auth-modes.spec.ts › auth mode: public gate › rejects an incorrect key with an inline error 1.6s
backends/mock-llm-auth-modes.spec.ts › auth mode: public gate › allows access after pasting the correct key 1.9s
backends/mock-llm-auth-modes.spec.ts › auth mode: public gate › skips auth screen for returning user with valid stored key 865ms
backends/mock-llm-auth-modes.spec.ts › auth mode: public gate › re-prompts when the server rotates its key (stale localStorage) 1.5s
backends/mock-llm-cross-connect.spec.ts › cross-connect: frontend-only → backend-only › frontend-only connects to a separate backend-only instance 23.1s
backends/mock-llm-cross-connect.spec.ts › cross-connect: frontend-only → multiple backends › connects to two separate backends and switches between them 20.7s
backends/mock-llm-partial-stack.spec.ts › partial stack: --frontend-only › serves the frontend but returns 503 for backend routes 7.4s
backends/mock-llm-partial-stack.spec.ts › partial stack: --backend-only › serves backend APIs but returns 503 for the frontend root 14.1s
backends/mock-llm-partial-stack.spec.ts › partial stack: port conflict › fails with a clear error when the ingress port is occupied 123ms
backends/mock-llm-partial-stack.spec.ts › partial stack: port conflict › starts successfully on a free port after a conflict 6.0s
conversations/mock-llm-conversation.spec.ts › mock-LLM agent-server conversation › step 1: create an LLM profile pointing at the mock LLM server 6.3s
conversations/mock-llm-conversation.spec.ts › mock-LLM agent-server conversation › step 2: activate the mock-llm profile and verify settings API 7.8s
conversations/mock-llm-conversation.spec.ts › mock-LLM agent-server conversation › step 3: run a conversation with the mock LLM 6.3s
conversations/mock-llm-conversation.spec.ts › mock-LLM agent-server conversation › step 4: resume conversation from sidebar after navigating away 5.7s
conversations/mock-llm-image-upload.spec.ts › mock-LLM image upload › attaching an image embeds it as base64 in the LLM completion call 13.3s
files/mock-llm-files-and-git.spec.ts › files tab, git control bar, and browser tab › step 1: ensure mock LLM profile is configured 7.3s
files/mock-llm-files-and-git.spec.ts › files tab, git control bar, and browser tab › step 2: start conversation and attach workspace metadata 11.4s
files/mock-llm-files-and-git.spec.ts › files tab, git control bar, and browser tab › step 3: git control bar shows workspace pill and git actions 25.3s
files/mock-llm-files-and-git.spec.ts › files tab, git control bar, and browser tab › step 4: files tab defaults to diff view for attached workspace 5.9s
files/mock-llm-files-and-git.spec.ts › files tab, git control bar, and browser tab › step 5: browser tab shows empty state 6.3s
files/mock-llm-files-and-git.spec.ts › files tab, git control bar, and browser tab › step 6: files tab defaults to file-tree view without attached workspace 7.5s
home/mock-llm-folder-workspace.spec.ts › mock-LLM folder browser → workspace → conversation › step 1: browse to a folder, add it as a workspace, and launch a conversation with the correct working_dir 7.3s
mcp/mock-llm-mcp-github.spec.ts › MCP GitHub server install flow › step 1: GitHub card is visible on the MCP marketplace page 5.5s
mcp/mock-llm-mcp-github.spec.ts › MCP GitHub server install flow › step 2: clicking GitHub card opens the install modal with correct fields 5.7s
mcp/mock-llm-mcp-github.spec.ts › MCP GitHub server install flow › step 3: full install flow — fill PAT, submit, verify installed 13.1s
mcp/mock-llm-mcp-github.spec.ts › MCP GitHub server install flow › step 4: installed GitHub server can be deleted 5.8s
mcp/mock-llm-mcp-slack-credentials.spec.ts › MCP Test Connection credential verification (Slack) › install: invalid Slack credentials are blocked with a credential-check error 5.8s
mcp/mock-llm-mcp-slack-credentials.spec.ts › MCP Test Connection credential verification (Slack) › install: a valid token missing only a scope still installs (missing_scope is not a credential failure) 5.9s
mcp/mock-llm-mcp-slack-credentials.spec.ts › MCP Test Connection credential verification (Slack) › install: an older agent server that omits tool_result still installs (compat) 5.9s
mcp/mock-llm-mcp-slack-credentials.spec.ts › MCP Test Connection credential verification (Slack) › edit: Test Connection verifies the stored credentials and surfaces a credential failure 5.8s
mcp/mock-llm-mcp-slack-credentials.spec.ts › MCP Test Connection credential verification (Slack) › edit: Test Connection reports success for valid stored credentials 5.9s
mcp/mock-llm-mcp-slack-credentials.spec.ts › MCP Test Connection credential verification (Slack) › custom (non-catalog) server: Test Connection attaches no verification probe 5.8s
onboarding/mock-llm-onboarding-happy-path.spec.ts › onboarding happy path › completes the full onboarding flow and launches a conversation 3.4s
onboarding/mock-llm-onboarding-regressions.spec.ts › onboarding recent regressions › keeps the modal open on backdrop click and Escape 1.4s
onboarding/mock-llm-onboarding-regressions.spec.ts › onboarding recent regressions › defaults the LLM setup step to OpenAI GPT-5.5 1.5s
regressions/mock-llm-ui-regressions.spec.ts › UI regressions › scopes standalone styles to the agent-server-ui shell 1.0s
regressions/mock-llm-ui-regressions.spec.ts › UI regressions › renders critic results on agent messages and finish actions 1.5s
regressions/mock-llm-ui-regressions.spec.ts › UI regressions › loads older events when scrolling up 1.9s
regressions/mock-llm-ui-regressions.spec.ts › UI regressions › selected workspace persists after navigating away and returning 2.5s
regressions/mock-llm-ui-regressions.spec.ts › UI regressions › cleared sessionStorage yields empty workspace selection 1.4s
settings/mock-llm-acp-agent.spec.ts › mock-LLM ACP agent conversation › step 1: configure ACP agent via Settings → Agent UI 13.7s
settings/mock-llm-acp-agent.spec.ts › mock-LLM ACP agent conversation › step 2: reload and verify ACP settings are persisted in UI 5.6s
settings/mock-llm-acp-agent.spec.ts › mock-LLM ACP agent conversation › step 3: start ACP conversation and verify agent reply 6.7s
settings/mock-llm-acp-agent.spec.ts › mock-LLM ACP agent conversation › step 4: resume ACP conversation from sidebar after navigating away 5.7s
settings/mock-llm-model-switch.spec.ts › mock-LLM /model slash command › step 1: configure LLM, create switch-target profile, register trajectory 13.3s
settings/mock-llm-model-switch.spec.ts › mock-LLM /model slash command › step 2: start conversation, switch profile via /model, verify switch 6.7s
settings/mock-llm-profile-management.spec.ts › active profile deletion + reconciliation › active profile is deletable and reconciliation activates another profile 8.6s
settings/mock-llm-profile-management.spec.ts › same-model profile identity › chat header shows the correct profile when two profiles share the same model 15.0s
settings/mock-llm-profile-management.spec.ts › OpenHands provider hidden base_url preservation › re-saving an OpenHands profile from Basic view preserves hidden base_url 7.3s
skills/mock-llm-skills.spec.ts › skill loading: project, user, and deletion › project skill in workspace/.agents/skills/ triggers on matching keyword 13.6s
skills/mock-llm-skills.spec.ts › skill loading: project, user, and deletion › user skill in ~/.openhands/skills/ triggers on matching keyword 13.3s
skills/mock-llm-skills.spec.ts › skill loading: project, user, and deletion › deleting a user skill removes it from subsequent conversations 13.4s

Posted by the Mock-LLM E2E workflow · results are deterministic (scripted LLM responses)

@neubig neubig added the review-test experimental cloud code review bot label Jun 18, 2026
@github-actions

Copy link
Copy Markdown
Contributor

✅ Mock-LLM E2E Tests

60/60 passed

Commit: f874bff1 · Workflow run · Test artifacts

Status Test Duration
automations/mock-llm-automation.spec.ts › mock-LLM automation lifecycle › step 1: setup LLM profile and register automation trajectory 7.4s
automations/mock-llm-automation.spec.ts › mock-LLM automation lifecycle › step 2: create automation and dispatch run via the UI 27.4s
automations/mock-llm-automation.spec.ts › mock-LLM automation lifecycle › step 3: verify automation and run on the automations page 6.2s
automations/mock-llm-preset-automation.spec.ts › preset automation → slash command conversation › automation card sends the correct slash command to a conversation 16.7s
automations/mock-llm-preset-automation.spec.ts › preset automation → slash command conversation › direct slash command from home page triggers skill activation 14.6s
backends/mock-llm-auth-modes.spec.ts › auth mode: fresh install with runtime-injected key › reaches the onboarding modal without pre-seeded localStorage 1.4s
backends/mock-llm-auth-modes.spec.ts › auth mode: non-public key rotation › recovers when localStorage has a stale session API key 5.4s
backends/mock-llm-auth-modes.spec.ts › auth mode: public gate › shows first-run onboarding before the auth screen when no key is configured 1.5s
backends/mock-llm-auth-modes.spec.ts › auth mode: public gate › rejects an incorrect key with an inline error 1.7s
backends/mock-llm-auth-modes.spec.ts › auth mode: public gate › allows access after pasting the correct key 1.8s
backends/mock-llm-auth-modes.spec.ts › auth mode: public gate › skips auth screen for returning user with valid stored key 795ms
backends/mock-llm-auth-modes.spec.ts › auth mode: public gate › re-prompts when the server rotates its key (stale localStorage) 1.5s
backends/mock-llm-cross-connect.spec.ts › cross-connect: frontend-only → backend-only › frontend-only connects to a separate backend-only instance 15.9s
backends/mock-llm-cross-connect.spec.ts › cross-connect: frontend-only → multiple backends › connects to two separate backends and switches between them 20.8s
backends/mock-llm-partial-stack.spec.ts › partial stack: --frontend-only › serves the frontend but returns 503 for backend routes 7.4s
backends/mock-llm-partial-stack.spec.ts › partial stack: --backend-only › serves backend APIs but returns 503 for the frontend root 14.1s
backends/mock-llm-partial-stack.spec.ts › partial stack: port conflict › fails with a clear error when the ingress port is occupied 108ms
backends/mock-llm-partial-stack.spec.ts › partial stack: port conflict › starts successfully on a free port after a conflict 6.0s
conversations/mock-llm-conversation.spec.ts › mock-LLM agent-server conversation › step 1: create an LLM profile pointing at the mock LLM server 6.4s
conversations/mock-llm-conversation.spec.ts › mock-LLM agent-server conversation › step 2: activate the mock-llm profile and verify settings API 6.3s
conversations/mock-llm-conversation.spec.ts › mock-LLM agent-server conversation › step 3: run a conversation with the mock LLM 7.4s
conversations/mock-llm-conversation.spec.ts › mock-LLM agent-server conversation › step 4: resume conversation from sidebar after navigating away 5.9s
conversations/mock-llm-image-upload.spec.ts › mock-LLM image upload › attaching an image embeds it as base64 in the LLM completion call 13.7s
files/mock-llm-files-and-git.spec.ts › files tab, git control bar, and browser tab › step 1: ensure mock LLM profile is configured 7.3s
files/mock-llm-files-and-git.spec.ts › files tab, git control bar, and browser tab › step 2: start conversation and attach workspace metadata 11.9s
files/mock-llm-files-and-git.spec.ts › files tab, git control bar, and browser tab › step 3: git control bar shows workspace pill and git actions 25.3s
files/mock-llm-files-and-git.spec.ts › files tab, git control bar, and browser tab › step 4: files tab defaults to diff view for attached workspace 6.0s
files/mock-llm-files-and-git.spec.ts › files tab, git control bar, and browser tab › step 5: browser tab shows empty state 6.3s
files/mock-llm-files-and-git.spec.ts › files tab, git control bar, and browser tab › step 6: files tab defaults to file-tree view without attached workspace 7.5s
home/mock-llm-folder-workspace.spec.ts › mock-LLM folder browser → workspace → conversation › step 1: browse to a folder, add it as a workspace, and launch a conversation with the correct working_dir 7.7s
mcp/mock-llm-mcp-github.spec.ts › MCP GitHub server install flow › step 1: GitHub card is visible on the MCP marketplace page 5.6s
mcp/mock-llm-mcp-github.spec.ts › MCP GitHub server install flow › step 2: clicking GitHub card opens the install modal with correct fields 5.8s
mcp/mock-llm-mcp-github.spec.ts › MCP GitHub server install flow › step 3: full install flow — fill PAT, submit, verify installed 13.2s
mcp/mock-llm-mcp-github.spec.ts › MCP GitHub server install flow › step 4: installed GitHub server can be deleted 5.9s
mcp/mock-llm-mcp-slack-credentials.spec.ts › MCP Test Connection credential verification (Slack) › install: invalid Slack credentials are blocked with a credential-check error 5.8s
mcp/mock-llm-mcp-slack-credentials.spec.ts › MCP Test Connection credential verification (Slack) › install: a valid token missing only a scope still installs (missing_scope is not a credential failure) 5.9s
mcp/mock-llm-mcp-slack-credentials.spec.ts › MCP Test Connection credential verification (Slack) › install: an older agent server that omits tool_result still installs (compat) 6.0s
mcp/mock-llm-mcp-slack-credentials.spec.ts › MCP Test Connection credential verification (Slack) › edit: Test Connection verifies the stored credentials and surfaces a credential failure 5.8s
mcp/mock-llm-mcp-slack-credentials.spec.ts › MCP Test Connection credential verification (Slack) › edit: Test Connection reports success for valid stored credentials 5.8s
mcp/mock-llm-mcp-slack-credentials.spec.ts › MCP Test Connection credential verification (Slack) › custom (non-catalog) server: Test Connection attaches no verification probe 5.8s
onboarding/mock-llm-onboarding-happy-path.spec.ts › onboarding happy path › completes the full onboarding flow and launches a conversation 3.9s
onboarding/mock-llm-onboarding-regressions.spec.ts › onboarding recent regressions › keeps the modal open on backdrop click and Escape 1.4s
onboarding/mock-llm-onboarding-regressions.spec.ts › onboarding recent regressions › defaults the LLM setup step to OpenAI GPT-5.5 1.5s
regressions/mock-llm-ui-regressions.spec.ts › UI regressions › scopes standalone styles to the agent-server-ui shell 1.4s
regressions/mock-llm-ui-regressions.spec.ts › UI regressions › renders critic results on agent messages and finish actions 1.5s
regressions/mock-llm-ui-regressions.spec.ts › UI regressions › loads older events when scrolling up 1.7s
regressions/mock-llm-ui-regressions.spec.ts › UI regressions › selected workspace persists after navigating away and returning 2.6s
regressions/mock-llm-ui-regressions.spec.ts › UI regressions › cleared sessionStorage yields empty workspace selection 1.4s
settings/mock-llm-acp-agent.spec.ts › mock-LLM ACP agent conversation › step 1: configure ACP agent via Settings → Agent UI 14.2s
settings/mock-llm-acp-agent.spec.ts › mock-LLM ACP agent conversation › step 2: reload and verify ACP settings are persisted in UI 5.6s
settings/mock-llm-acp-agent.spec.ts › mock-LLM ACP agent conversation › step 3: start ACP conversation and verify agent reply 6.8s
settings/mock-llm-acp-agent.spec.ts › mock-LLM ACP agent conversation › step 4: resume ACP conversation from sidebar after navigating away 5.8s
settings/mock-llm-model-switch.spec.ts › mock-LLM /model slash command › step 1: configure LLM, create switch-target profile, register trajectory 13.3s
settings/mock-llm-model-switch.spec.ts › mock-LLM /model slash command › step 2: start conversation, switch profile via /model, verify switch 7.0s
settings/mock-llm-profile-management.spec.ts › active profile deletion + reconciliation › active profile is deletable and reconciliation activates another profile 8.6s
settings/mock-llm-profile-management.spec.ts › same-model profile identity › chat header shows the correct profile when two profiles share the same model 15.5s
settings/mock-llm-profile-management.spec.ts › OpenHands provider hidden base_url preservation › re-saving an OpenHands profile from Basic view preserves hidden base_url 7.5s
skills/mock-llm-skills.spec.ts › skill loading: project, user, and deletion › project skill in workspace/.agents/skills/ triggers on matching keyword 13.8s
skills/mock-llm-skills.spec.ts › skill loading: project, user, and deletion › user skill in ~/.openhands/skills/ triggers on matching keyword 14.4s
skills/mock-llm-skills.spec.ts › skill loading: project, user, and deletion › deleting a user skill removes it from subsequent conversations 14.5s

Posted by the Mock-LLM E2E workflow · results are deterministic (scripted LLM responses)

@github-actions

Copy link
Copy Markdown
Contributor

⚠️ Mock-LLM Docker E2E Test Results

0/0 passed

Commit: f874bff1 · Workflow run

Status Test Duration

Posted by the Mock-LLM E2E workflow · results are deterministic (scripted LLM responses)

@github-actions

Copy link
Copy Markdown
Contributor

✅ Mock-LLM Docker E2E Test Results

60/60 passed

Commit: f874bff1 · Workflow run · Test artifacts

Status Test Duration
automations/mock-llm-automation.spec.ts › mock-LLM automation lifecycle › step 1: setup LLM profile and register automation trajectory 7.0s
automations/mock-llm-automation.spec.ts › mock-LLM automation lifecycle › step 2: create automation and dispatch run via the UI 31.4s
automations/mock-llm-automation.spec.ts › mock-LLM automation lifecycle › step 3: verify automation and run on the automations page 6.0s
automations/mock-llm-preset-automation.spec.ts › preset automation → slash command conversation › automation card sends the correct slash command to a conversation 15.7s
automations/mock-llm-preset-automation.spec.ts › preset automation → slash command conversation › direct slash command from home page triggers skill activation 13.0s
backends/mock-llm-auth-modes.spec.ts › auth mode: fresh install with runtime-injected key › reaches the onboarding modal without pre-seeded localStorage 1.3s
backends/mock-llm-auth-modes.spec.ts › auth mode: non-public key rotation › recovers when localStorage has a stale session API key 5.3s
backends/mock-llm-auth-modes.spec.ts › auth mode: public gate › shows first-run onboarding before the auth screen when no key is configured 1.4s
backends/mock-llm-auth-modes.spec.ts › auth mode: public gate › rejects an incorrect key with an inline error 1.6s
backends/mock-llm-auth-modes.spec.ts › auth mode: public gate › allows access after pasting the correct key 1.8s
backends/mock-llm-auth-modes.spec.ts › auth mode: public gate › skips auth screen for returning user with valid stored key 1.3s
backends/mock-llm-auth-modes.spec.ts › auth mode: public gate › re-prompts when the server rotates its key (stale localStorage) 1.4s
backends/mock-llm-cross-connect.spec.ts › cross-connect: frontend-only → backend-only › frontend-only connects to a separate backend-only instance 19.8s
backends/mock-llm-cross-connect.spec.ts › cross-connect: frontend-only → multiple backends › connects to two separate backends and switches between them 18.5s
backends/mock-llm-partial-stack.spec.ts › partial stack: --frontend-only › serves the frontend but returns 503 for backend routes 7.3s
backends/mock-llm-partial-stack.spec.ts › partial stack: --backend-only › serves backend APIs but returns 503 for the frontend root 12.1s
backends/mock-llm-partial-stack.spec.ts › partial stack: port conflict › fails with a clear error when the ingress port is occupied 100ms
backends/mock-llm-partial-stack.spec.ts › partial stack: port conflict › starts successfully on a free port after a conflict 6.0s
conversations/mock-llm-conversation.spec.ts › mock-LLM agent-server conversation › step 1: create an LLM profile pointing at the mock LLM server 6.1s
conversations/mock-llm-conversation.spec.ts › mock-LLM agent-server conversation › step 2: activate the mock-llm profile and verify settings API 6.1s
conversations/mock-llm-conversation.spec.ts › mock-LLM agent-server conversation › step 3: run a conversation with the mock LLM 6.3s
conversations/mock-llm-conversation.spec.ts › mock-LLM agent-server conversation › step 4: resume conversation from sidebar after navigating away 5.6s
conversations/mock-llm-image-upload.spec.ts › mock-LLM image upload › attaching an image embeds it as base64 in the LLM completion call 13.1s
files/mock-llm-files-and-git.spec.ts › files tab, git control bar, and browser tab › step 1: ensure mock LLM profile is configured 7.0s
files/mock-llm-files-and-git.spec.ts › files tab, git control bar, and browser tab › step 2: start conversation and attach workspace metadata 11.4s
files/mock-llm-files-and-git.spec.ts › files tab, git control bar, and browser tab › step 3: git control bar shows workspace pill and git actions 25.4s
files/mock-llm-files-and-git.spec.ts › files tab, git control bar, and browser tab › step 4: files tab defaults to diff view for attached workspace 5.9s
files/mock-llm-files-and-git.spec.ts › files tab, git control bar, and browser tab › step 5: browser tab shows empty state 6.3s
files/mock-llm-files-and-git.spec.ts › files tab, git control bar, and browser tab › step 6: files tab defaults to file-tree view without attached workspace 9.4s
home/mock-llm-folder-workspace.spec.ts › mock-LLM folder browser → workspace → conversation › step 1: browse to a folder, add it as a workspace, and launch a conversation with the correct working_dir 7.3s
mcp/mock-llm-mcp-github.spec.ts › MCP GitHub server install flow › step 1: GitHub card is visible on the MCP marketplace page 5.5s
mcp/mock-llm-mcp-github.spec.ts › MCP GitHub server install flow › step 2: clicking GitHub card opens the install modal with correct fields 5.7s
mcp/mock-llm-mcp-github.spec.ts › MCP GitHub server install flow › step 3: full install flow — fill PAT, submit, verify installed 12.4s
mcp/mock-llm-mcp-github.spec.ts › MCP GitHub server install flow › step 4: installed GitHub server can be deleted 5.7s
mcp/mock-llm-mcp-slack-credentials.spec.ts › MCP Test Connection credential verification (Slack) › install: invalid Slack credentials are blocked with a credential-check error 5.6s
mcp/mock-llm-mcp-slack-credentials.spec.ts › MCP Test Connection credential verification (Slack) › install: a valid token missing only a scope still installs (missing_scope is not a credential failure) 5.9s
mcp/mock-llm-mcp-slack-credentials.spec.ts › MCP Test Connection credential verification (Slack) › install: an older agent server that omits tool_result still installs (compat) 5.7s
mcp/mock-llm-mcp-slack-credentials.spec.ts › MCP Test Connection credential verification (Slack) › edit: Test Connection verifies the stored credentials and surfaces a credential failure 5.7s
mcp/mock-llm-mcp-slack-credentials.spec.ts › MCP Test Connection credential verification (Slack) › edit: Test Connection reports success for valid stored credentials 5.7s
mcp/mock-llm-mcp-slack-credentials.spec.ts › MCP Test Connection credential verification (Slack) › custom (non-catalog) server: Test Connection attaches no verification probe 5.7s
onboarding/mock-llm-onboarding-happy-path.spec.ts › onboarding happy path › completes the full onboarding flow and launches a conversation 3.0s
onboarding/mock-llm-onboarding-regressions.spec.ts › onboarding recent regressions › keeps the modal open on backdrop click and Escape 1.4s
onboarding/mock-llm-onboarding-regressions.spec.ts › onboarding recent regressions › defaults the LLM setup step to OpenAI GPT-5.5 1.4s
regressions/mock-llm-ui-regressions.spec.ts › UI regressions › scopes standalone styles to the agent-server-ui shell 1.3s
regressions/mock-llm-ui-regressions.spec.ts › UI regressions › renders critic results on agent messages and finish actions 1.4s
regressions/mock-llm-ui-regressions.spec.ts › UI regressions › loads older events when scrolling up 1.5s
regressions/mock-llm-ui-regressions.spec.ts › UI regressions › selected workspace persists after navigating away and returning 2.3s
regressions/mock-llm-ui-regressions.spec.ts › UI regressions › cleared sessionStorage yields empty workspace selection 1.3s
settings/mock-llm-acp-agent.spec.ts › mock-LLM ACP agent conversation › step 1: configure ACP agent via Settings → Agent UI 13.5s
settings/mock-llm-acp-agent.spec.ts › mock-LLM ACP agent conversation › step 2: reload and verify ACP settings are persisted in UI 5.7s
settings/mock-llm-acp-agent.spec.ts › mock-LLM ACP agent conversation › step 3: start ACP conversation and verify agent reply 7.3s
settings/mock-llm-acp-agent.spec.ts › mock-LLM ACP agent conversation › step 4: resume ACP conversation from sidebar after navigating away 5.6s
settings/mock-llm-model-switch.spec.ts › mock-LLM /model slash command › step 1: configure LLM, create switch-target profile, register trajectory 13.2s
settings/mock-llm-model-switch.spec.ts › mock-LLM /model slash command › step 2: start conversation, switch profile via /model, verify switch 6.6s
settings/mock-llm-profile-management.spec.ts › active profile deletion + reconciliation › active profile is deletable and reconciliation activates another profile 8.3s
settings/mock-llm-profile-management.spec.ts › same-model profile identity › chat header shows the correct profile when two profiles share the same model 14.8s
settings/mock-llm-profile-management.spec.ts › OpenHands provider hidden base_url preservation › re-saving an OpenHands profile from Basic view preserves hidden base_url 7.3s
skills/mock-llm-skills.spec.ts › skill loading: project, user, and deletion › project skill in workspace/.agents/skills/ triggers on matching keyword 13.2s
skills/mock-llm-skills.spec.ts › skill loading: project, user, and deletion › user skill in ~/.openhands/skills/ triggers on matching keyword 13.2s
skills/mock-llm-skills.spec.ts › skill loading: project, user, and deletion › deleting a user skill removes it from subsequent conversations 13.0s

Posted by the Mock-LLM E2E workflow · results are deterministic (scripted LLM responses)

@github-actions

Copy link
Copy Markdown
Contributor

✅ Mock-LLM E2E Tests

60/60 passed

Commit: ad85ca03 · Workflow run · Test artifacts

Status Test Duration
automations/mock-llm-automation.spec.ts › mock-LLM automation lifecycle › step 1: setup LLM profile and register automation trajectory 7.4s
automations/mock-llm-automation.spec.ts › mock-LLM automation lifecycle › step 2: create automation and dispatch run via the UI 26.5s
automations/mock-llm-automation.spec.ts › mock-LLM automation lifecycle › step 3: verify automation and run on the automations page 6.1s
automations/mock-llm-preset-automation.spec.ts › preset automation → slash command conversation › automation card sends the correct slash command to a conversation 16.0s
automations/mock-llm-preset-automation.spec.ts › preset automation → slash command conversation › direct slash command from home page triggers skill activation 13.6s
backends/mock-llm-auth-modes.spec.ts › auth mode: fresh install with runtime-injected key › reaches the onboarding modal without pre-seeded localStorage 1.4s
backends/mock-llm-auth-modes.spec.ts › auth mode: non-public key rotation › recovers when localStorage has a stale session API key 5.3s
backends/mock-llm-auth-modes.spec.ts › auth mode: public gate › shows first-run onboarding before the auth screen when no key is configured 1.4s
backends/mock-llm-auth-modes.spec.ts › auth mode: public gate › rejects an incorrect key with an inline error 1.6s
backends/mock-llm-auth-modes.spec.ts › auth mode: public gate › allows access after pasting the correct key 1.9s
backends/mock-llm-auth-modes.spec.ts › auth mode: public gate › skips auth screen for returning user with valid stored key 774ms
backends/mock-llm-auth-modes.spec.ts › auth mode: public gate › re-prompts when the server rotates its key (stale localStorage) 1.5s
backends/mock-llm-cross-connect.spec.ts › cross-connect: frontend-only → backend-only › frontend-only connects to a separate backend-only instance 16.0s
backends/mock-llm-cross-connect.spec.ts › cross-connect: frontend-only → multiple backends › connects to two separate backends and switches between them 20.6s
backends/mock-llm-partial-stack.spec.ts › partial stack: --frontend-only › serves the frontend but returns 503 for backend routes 7.4s
backends/mock-llm-partial-stack.spec.ts › partial stack: --backend-only › serves backend APIs but returns 503 for the frontend root 14.1s
backends/mock-llm-partial-stack.spec.ts › partial stack: port conflict › fails with a clear error when the ingress port is occupied 101ms
backends/mock-llm-partial-stack.spec.ts › partial stack: port conflict › starts successfully on a free port after a conflict 6.0s
conversations/mock-llm-conversation.spec.ts › mock-LLM agent-server conversation › step 1: create an LLM profile pointing at the mock LLM server 6.4s
conversations/mock-llm-conversation.spec.ts › mock-LLM agent-server conversation › step 2: activate the mock-llm profile and verify settings API 6.2s
conversations/mock-llm-conversation.spec.ts › mock-LLM agent-server conversation › step 3: run a conversation with the mock LLM 6.6s
conversations/mock-llm-conversation.spec.ts › mock-LLM agent-server conversation › step 4: resume conversation from sidebar after navigating away 5.7s
conversations/mock-llm-image-upload.spec.ts › mock-LLM image upload › attaching an image embeds it as base64 in the LLM completion call 13.5s
files/mock-llm-files-and-git.spec.ts › files tab, git control bar, and browser tab › step 1: ensure mock LLM profile is configured 7.3s
files/mock-llm-files-and-git.spec.ts › files tab, git control bar, and browser tab › step 2: start conversation and attach workspace metadata 11.7s
files/mock-llm-files-and-git.spec.ts › files tab, git control bar, and browser tab › step 3: git control bar shows workspace pill and git actions 25.3s
files/mock-llm-files-and-git.spec.ts › files tab, git control bar, and browser tab › step 4: files tab defaults to diff view for attached workspace 5.8s
files/mock-llm-files-and-git.spec.ts › files tab, git control bar, and browser tab › step 5: browser tab shows empty state 6.3s
files/mock-llm-files-and-git.spec.ts › files tab, git control bar, and browser tab › step 6: files tab defaults to file-tree view without attached workspace 7.5s
home/mock-llm-folder-workspace.spec.ts › mock-LLM folder browser → workspace → conversation › step 1: browse to a folder, add it as a workspace, and launch a conversation with the correct working_dir 7.7s
mcp/mock-llm-mcp-github.spec.ts › MCP GitHub server install flow › step 1: GitHub card is visible on the MCP marketplace page 5.6s
mcp/mock-llm-mcp-github.spec.ts › MCP GitHub server install flow › step 2: clicking GitHub card opens the install modal with correct fields 5.7s
mcp/mock-llm-mcp-github.spec.ts › MCP GitHub server install flow › step 3: full install flow — fill PAT, submit, verify installed 12.9s
mcp/mock-llm-mcp-github.spec.ts › MCP GitHub server install flow › step 4: installed GitHub server can be deleted 5.8s
mcp/mock-llm-mcp-slack-credentials.spec.ts › MCP Test Connection credential verification (Slack) › install: invalid Slack credentials are blocked with a credential-check error 5.8s
mcp/mock-llm-mcp-slack-credentials.spec.ts › MCP Test Connection credential verification (Slack) › install: a valid token missing only a scope still installs (missing_scope is not a credential failure) 7.8s
mcp/mock-llm-mcp-slack-credentials.spec.ts › MCP Test Connection credential verification (Slack) › install: an older agent server that omits tool_result still installs (compat) 5.8s
mcp/mock-llm-mcp-slack-credentials.spec.ts › MCP Test Connection credential verification (Slack) › edit: Test Connection verifies the stored credentials and surfaces a credential failure 5.8s
mcp/mock-llm-mcp-slack-credentials.spec.ts › MCP Test Connection credential verification (Slack) › edit: Test Connection reports success for valid stored credentials 5.8s
mcp/mock-llm-mcp-slack-credentials.spec.ts › MCP Test Connection credential verification (Slack) › custom (non-catalog) server: Test Connection attaches no verification probe 5.7s
onboarding/mock-llm-onboarding-happy-path.spec.ts › onboarding happy path › completes the full onboarding flow and launches a conversation 4.4s
onboarding/mock-llm-onboarding-regressions.spec.ts › onboarding recent regressions › keeps the modal open on backdrop click and Escape 1.4s
onboarding/mock-llm-onboarding-regressions.spec.ts › onboarding recent regressions › defaults the LLM setup step to OpenAI GPT-5.5 1.5s
regressions/mock-llm-ui-regressions.spec.ts › UI regressions › scopes standalone styles to the agent-server-ui shell 1.4s
regressions/mock-llm-ui-regressions.spec.ts › UI regressions › renders critic results on agent messages and finish actions 1.6s
regressions/mock-llm-ui-regressions.spec.ts › UI regressions › loads older events when scrolling up 1.9s
regressions/mock-llm-ui-regressions.spec.ts › UI regressions › selected workspace persists after navigating away and returning 2.6s
regressions/mock-llm-ui-regressions.spec.ts › UI regressions › cleared sessionStorage yields empty workspace selection 1.4s
settings/mock-llm-acp-agent.spec.ts › mock-LLM ACP agent conversation › step 1: configure ACP agent via Settings → Agent UI 14.1s
settings/mock-llm-acp-agent.spec.ts › mock-LLM ACP agent conversation › step 2: reload and verify ACP settings are persisted in UI 5.6s
settings/mock-llm-acp-agent.spec.ts › mock-LLM ACP agent conversation › step 3: start ACP conversation and verify agent reply 6.7s
settings/mock-llm-acp-agent.spec.ts › mock-LLM ACP agent conversation › step 4: resume ACP conversation from sidebar after navigating away 5.8s
settings/mock-llm-model-switch.spec.ts › mock-LLM /model slash command › step 1: configure LLM, create switch-target profile, register trajectory 13.2s
settings/mock-llm-model-switch.spec.ts › mock-LLM /model slash command › step 2: start conversation, switch profile via /model, verify switch 6.8s
settings/mock-llm-profile-management.spec.ts › active profile deletion + reconciliation › active profile is deletable and reconciliation activates another profile 8.6s
settings/mock-llm-profile-management.spec.ts › same-model profile identity › chat header shows the correct profile when two profiles share the same model 15.3s
settings/mock-llm-profile-management.spec.ts › OpenHands provider hidden base_url preservation › re-saving an OpenHands profile from Basic view preserves hidden base_url 7.6s
skills/mock-llm-skills.spec.ts › skill loading: project, user, and deletion › project skill in workspace/.agents/skills/ triggers on matching keyword 13.7s
skills/mock-llm-skills.spec.ts › skill loading: project, user, and deletion › user skill in ~/.openhands/skills/ triggers on matching keyword 13.5s
skills/mock-llm-skills.spec.ts › skill loading: project, user, and deletion › deleting a user skill removes it from subsequent conversations 13.5s

Posted by the Mock-LLM E2E workflow · results are deterministic (scripted LLM responses)

@hieptl hieptl left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hello @neubig,

I followed the steps below and noticed a potential issue:

Step 1: Delete the ~/.openhands directory.
Step 2: Check out the lock-to-cloud-backend branch.
Step 3: Run npm i.
Step 4: Run the following command:

node /Users/hieple/Documents/openhands/agent-server-gui/scripts/static-server.mjs --lock-to-cloud https://app.all-hands.dev

Static-server + proxy listening on http://:::3001/
  Static dir: /Users/hieple/Documents/openhands/agent-server-gui/build
  Backend setup locked to Cloud: https://app.all-hands.dev
  * (default) -> static files + SPA fallback

Then, navigate to http://localhost:3001.

In my case, the Add Backend modal appears first. Is this the expected behavior?

Please see the attached image for additional context.

Image

Thank you very much! 🙏

@github-actions

Copy link
Copy Markdown
Contributor

⚠️ Mock-LLM Docker E2E Test Results

0/0 passed

Commit: ad85ca03 · Workflow run

Status Test Duration

Posted by the Mock-LLM E2E workflow · results are deterministic (scripted LLM responses)

neubig added 2 commits June 18, 2026 15:17
…-to-cloud-backend

# Conflicts:
#	__tests__/components/onboarding/onboarding-modal.test.tsx
@github-actions

Copy link
Copy Markdown
Contributor

🛑 Mock-LLM E2E Tests

1/1 passed · ⚠️ 59 not run (process killed at 1/60)

Commit: f88db645 · Workflow run · Test artifacts

Status Test Duration
chromium › automations/mock-llm-automation.spec.ts › mock-LLM automation lifecycle › step 1: setup LLM profile and register automation trajectory 7.8s

Posted by the Mock-LLM E2E workflow · results are deterministic (scripted LLM responses)

@github-actions

Copy link
Copy Markdown
Contributor

⚠️ Mock-LLM Docker E2E Test Results

0/0 passed

Commit: f88db645 · Workflow run

Status Test Duration

Posted by the Mock-LLM E2E workflow · results are deterministic (scripted LLM responses)

@github-actions

Copy link
Copy Markdown
Contributor

⚠️ Mock-LLM Docker E2E Test Results

0/0 passed

Commit: 5bb8049f · Workflow run

Status Test Duration

Posted by the Mock-LLM E2E workflow · results are deterministic (scripted LLM responses)

@github-actions

Copy link
Copy Markdown
Contributor

⚠️ Mock-LLM E2E Tests

0/0 passed

Commit: 5bb8049f · Workflow run

Status Test Duration

Posted by the Mock-LLM E2E workflow · results are deterministic (scripted LLM responses)

@github-actions

Copy link
Copy Markdown
Contributor

✅ Mock-LLM E2E Tests

60/60 passed

Commit: 6e7bac6f · Workflow run · Test artifacts

Status Test Duration
automations/mock-llm-automation.spec.ts › mock-LLM automation lifecycle › step 1: setup LLM profile and register automation trajectory 7.4s
automations/mock-llm-automation.spec.ts › mock-LLM automation lifecycle › step 2: create automation and dispatch run via the UI 27.5s
automations/mock-llm-automation.spec.ts › mock-LLM automation lifecycle › step 3: verify automation and run on the automations page 6.3s
automations/mock-llm-preset-automation.spec.ts › preset automation → slash command conversation › automation card sends the correct slash command to a conversation 16.7s
automations/mock-llm-preset-automation.spec.ts › preset automation → slash command conversation › direct slash command from home page triggers skill activation 14.5s
backends/mock-llm-auth-modes.spec.ts › auth mode: fresh install with runtime-injected key › reaches the onboarding modal without pre-seeded localStorage 1.4s
backends/mock-llm-auth-modes.spec.ts › auth mode: non-public key rotation › recovers when localStorage has a stale session API key 5.3s
backends/mock-llm-auth-modes.spec.ts › auth mode: public gate › shows first-run onboarding before the auth screen when no key is configured 1.5s
backends/mock-llm-auth-modes.spec.ts › auth mode: public gate › rejects an incorrect key with an inline error 1.7s
backends/mock-llm-auth-modes.spec.ts › auth mode: public gate › allows access after pasting the correct key 1.9s
backends/mock-llm-auth-modes.spec.ts › auth mode: public gate › skips auth screen for returning user with valid stored key 785ms
backends/mock-llm-auth-modes.spec.ts › auth mode: public gate › re-prompts when the server rotates its key (stale localStorage) 1.4s
backends/mock-llm-cross-connect.spec.ts › cross-connect: frontend-only → backend-only › frontend-only connects to a separate backend-only instance 18.0s
backends/mock-llm-cross-connect.spec.ts › cross-connect: frontend-only → multiple backends › connects to two separate backends and switches between them 20.9s
backends/mock-llm-partial-stack.spec.ts › partial stack: --frontend-only › serves the frontend but returns 503 for backend routes 7.4s
backends/mock-llm-partial-stack.spec.ts › partial stack: --backend-only › serves backend APIs but returns 503 for the frontend root 14.1s
backends/mock-llm-partial-stack.spec.ts › partial stack: port conflict › fails with a clear error when the ingress port is occupied 112ms
backends/mock-llm-partial-stack.spec.ts › partial stack: port conflict › starts successfully on a free port after a conflict 6.1s
conversations/mock-llm-conversation.spec.ts › mock-LLM agent-server conversation › step 1: create an LLM profile pointing at the mock LLM server 6.3s
conversations/mock-llm-conversation.spec.ts › mock-LLM agent-server conversation › step 2: activate the mock-llm profile and verify settings API 6.2s
conversations/mock-llm-conversation.spec.ts › mock-LLM agent-server conversation › step 3: run a conversation with the mock LLM 7.9s
conversations/mock-llm-conversation.spec.ts › mock-LLM agent-server conversation › step 4: resume conversation from sidebar after navigating away 5.8s
conversations/mock-llm-image-upload.spec.ts › mock-LLM image upload › attaching an image embeds it as base64 in the LLM completion call 14.4s
files/mock-llm-files-and-git.spec.ts › files tab, git control bar, and browser tab › step 1: ensure mock LLM profile is configured 7.3s
files/mock-llm-files-and-git.spec.ts › files tab, git control bar, and browser tab › step 2: start conversation and attach workspace metadata 12.5s
files/mock-llm-files-and-git.spec.ts › files tab, git control bar, and browser tab › step 3: git control bar shows workspace pill and git actions 25.3s
files/mock-llm-files-and-git.spec.ts › files tab, git control bar, and browser tab › step 4: files tab defaults to diff view for attached workspace 5.9s
files/mock-llm-files-and-git.spec.ts › files tab, git control bar, and browser tab › step 5: browser tab shows empty state 6.3s
files/mock-llm-files-and-git.spec.ts › files tab, git control bar, and browser tab › step 6: files tab defaults to file-tree view without attached workspace 8.2s
home/mock-llm-folder-workspace.spec.ts › mock-LLM folder browser → workspace → conversation › step 1: browse to a folder, add it as a workspace, and launch a conversation with the correct working_dir 8.6s
mcp/mock-llm-mcp-github.spec.ts › MCP GitHub server install flow › step 1: GitHub card is visible on the MCP marketplace page 5.6s
mcp/mock-llm-mcp-github.spec.ts › MCP GitHub server install flow › step 2: clicking GitHub card opens the install modal with correct fields 5.7s
mcp/mock-llm-mcp-github.spec.ts › MCP GitHub server install flow › step 3: full install flow — fill PAT, submit, verify installed 13.1s
mcp/mock-llm-mcp-github.spec.ts › MCP GitHub server install flow › step 4: installed GitHub server can be deleted 5.8s
mcp/mock-llm-mcp-slack-credentials.spec.ts › MCP Test Connection credential verification (Slack) › install: invalid Slack credentials are blocked with a credential-check error 5.8s
mcp/mock-llm-mcp-slack-credentials.spec.ts › MCP Test Connection credential verification (Slack) › install: a valid token missing only a scope still installs (missing_scope is not a credential failure) 6.0s
mcp/mock-llm-mcp-slack-credentials.spec.ts › MCP Test Connection credential verification (Slack) › install: an older agent server that omits tool_result still installs (compat) 6.0s
mcp/mock-llm-mcp-slack-credentials.spec.ts › MCP Test Connection credential verification (Slack) › edit: Test Connection verifies the stored credentials and surfaces a credential failure 5.8s
mcp/mock-llm-mcp-slack-credentials.spec.ts › MCP Test Connection credential verification (Slack) › edit: Test Connection reports success for valid stored credentials 5.8s
mcp/mock-llm-mcp-slack-credentials.spec.ts › MCP Test Connection credential verification (Slack) › custom (non-catalog) server: Test Connection attaches no verification probe 5.8s
onboarding/mock-llm-onboarding-happy-path.spec.ts › onboarding happy path › completes the full onboarding flow and launches a conversation 4.5s
onboarding/mock-llm-onboarding-regressions.spec.ts › onboarding recent regressions › keeps the modal open on backdrop click and Escape 1.4s
onboarding/mock-llm-onboarding-regressions.spec.ts › onboarding recent regressions › defaults the LLM setup step to OpenAI GPT-5.5 1.6s
regressions/mock-llm-ui-regressions.spec.ts › UI regressions › scopes standalone styles to the agent-server-ui shell 1.4s
regressions/mock-llm-ui-regressions.spec.ts › UI regressions › renders critic results on agent messages and finish actions 1.5s
regressions/mock-llm-ui-regressions.spec.ts › UI regressions › loads older events when scrolling up 1.7s
regressions/mock-llm-ui-regressions.spec.ts › UI regressions › selected workspace persists after navigating away and returning 2.6s
regressions/mock-llm-ui-regressions.spec.ts › UI regressions › cleared sessionStorage yields empty workspace selection 1.1s
settings/mock-llm-acp-agent.spec.ts › mock-LLM ACP agent conversation › step 1: configure ACP agent via Settings → Agent UI 14.0s
settings/mock-llm-acp-agent.spec.ts › mock-LLM ACP agent conversation › step 2: reload and verify ACP settings are persisted in UI 5.6s
settings/mock-llm-acp-agent.spec.ts › mock-LLM ACP agent conversation › step 3: start ACP conversation and verify agent reply 6.8s
settings/mock-llm-acp-agent.spec.ts › mock-LLM ACP agent conversation › step 4: resume ACP conversation from sidebar after navigating away 5.8s
settings/mock-llm-model-switch.spec.ts › mock-LLM /model slash command › step 1: configure LLM, create switch-target profile, register trajectory 13.4s
settings/mock-llm-model-switch.spec.ts › mock-LLM /model slash command › step 2: start conversation, switch profile via /model, verify switch 7.7s
settings/mock-llm-profile-management.spec.ts › active profile deletion + reconciliation › active profile is deletable and reconciliation activates another profile 8.8s
settings/mock-llm-profile-management.spec.ts › same-model profile identity › chat header shows the correct profile when two profiles share the same model 16.1s
settings/mock-llm-profile-management.spec.ts › OpenHands provider hidden base_url preservation › re-saving an OpenHands profile from Basic view preserves hidden base_url 7.5s
skills/mock-llm-skills.spec.ts › skill loading: project, user, and deletion › project skill in workspace/.agents/skills/ triggers on matching keyword 14.7s
skills/mock-llm-skills.spec.ts › skill loading: project, user, and deletion › user skill in ~/.openhands/skills/ triggers on matching keyword 13.8s
skills/mock-llm-skills.spec.ts › skill loading: project, user, and deletion › deleting a user skill removes it from subsequent conversations 13.8s

Posted by the Mock-LLM E2E workflow · results are deterministic (scripted LLM responses)

@github-actions

Copy link
Copy Markdown
Contributor

✅ Mock-LLM Docker E2E Test Results

60/60 passed

Commit: 6e7bac6f · Workflow run · Test artifacts

Status Test Duration
automations/mock-llm-automation.spec.ts › mock-LLM automation lifecycle › step 1: setup LLM profile and register automation trajectory 7.2s
automations/mock-llm-automation.spec.ts › mock-LLM automation lifecycle › step 2: create automation and dispatch run via the UI 31.5s
automations/mock-llm-automation.spec.ts › mock-LLM automation lifecycle › step 3: verify automation and run on the automations page 6.1s
automations/mock-llm-preset-automation.spec.ts › preset automation → slash command conversation › automation card sends the correct slash command to a conversation 15.9s
automations/mock-llm-preset-automation.spec.ts › preset automation → slash command conversation › direct slash command from home page triggers skill activation 13.3s
backends/mock-llm-auth-modes.spec.ts › auth mode: fresh install with runtime-injected key › reaches the onboarding modal without pre-seeded localStorage 1.4s
backends/mock-llm-auth-modes.spec.ts › auth mode: non-public key rotation › recovers when localStorage has a stale session API key 5.3s
backends/mock-llm-auth-modes.spec.ts › auth mode: public gate › shows first-run onboarding before the auth screen when no key is configured 1.4s
backends/mock-llm-auth-modes.spec.ts › auth mode: public gate › rejects an incorrect key with an inline error 1.6s
backends/mock-llm-auth-modes.spec.ts › auth mode: public gate › allows access after pasting the correct key 1.9s
backends/mock-llm-auth-modes.spec.ts › auth mode: public gate › skips auth screen for returning user with valid stored key 1.1s
backends/mock-llm-auth-modes.spec.ts › auth mode: public gate › re-prompts when the server rotates its key (stale localStorage) 1.5s
backends/mock-llm-cross-connect.spec.ts › cross-connect: frontend-only → backend-only › frontend-only connects to a separate backend-only instance 23.1s
backends/mock-llm-cross-connect.spec.ts › cross-connect: frontend-only → multiple backends › connects to two separate backends and switches between them 20.8s
backends/mock-llm-partial-stack.spec.ts › partial stack: --frontend-only › serves the frontend but returns 503 for backend routes 7.4s
backends/mock-llm-partial-stack.spec.ts › partial stack: --backend-only › serves backend APIs but returns 503 for the frontend root 14.1s
backends/mock-llm-partial-stack.spec.ts › partial stack: port conflict › fails with a clear error when the ingress port is occupied 105ms
backends/mock-llm-partial-stack.spec.ts › partial stack: port conflict › starts successfully on a free port after a conflict 6.0s
conversations/mock-llm-conversation.spec.ts › mock-LLM agent-server conversation › step 1: create an LLM profile pointing at the mock LLM server 6.3s
conversations/mock-llm-conversation.spec.ts › mock-LLM agent-server conversation › step 2: activate the mock-llm profile and verify settings API 7.9s
conversations/mock-llm-conversation.spec.ts › mock-LLM agent-server conversation › step 3: run a conversation with the mock LLM 6.4s
conversations/mock-llm-conversation.spec.ts › mock-LLM agent-server conversation › step 4: resume conversation from sidebar after navigating away 5.7s
conversations/mock-llm-image-upload.spec.ts › mock-LLM image upload › attaching an image embeds it as base64 in the LLM completion call 13.4s
files/mock-llm-files-and-git.spec.ts › files tab, git control bar, and browser tab › step 1: ensure mock LLM profile is configured 7.3s
files/mock-llm-files-and-git.spec.ts › files tab, git control bar, and browser tab › step 2: start conversation and attach workspace metadata 11.4s
files/mock-llm-files-and-git.spec.ts › files tab, git control bar, and browser tab › step 3: git control bar shows workspace pill and git actions 25.3s
files/mock-llm-files-and-git.spec.ts › files tab, git control bar, and browser tab › step 4: files tab defaults to diff view for attached workspace 5.9s
files/mock-llm-files-and-git.spec.ts › files tab, git control bar, and browser tab › step 5: browser tab shows empty state 6.3s
files/mock-llm-files-and-git.spec.ts › files tab, git control bar, and browser tab › step 6: files tab defaults to file-tree view without attached workspace 7.3s
home/mock-llm-folder-workspace.spec.ts › mock-LLM folder browser → workspace → conversation › step 1: browse to a folder, add it as a workspace, and launch a conversation with the correct working_dir 7.3s
mcp/mock-llm-mcp-github.spec.ts › MCP GitHub server install flow › step 1: GitHub card is visible on the MCP marketplace page 5.5s
mcp/mock-llm-mcp-github.spec.ts › MCP GitHub server install flow › step 2: clicking GitHub card opens the install modal with correct fields 5.7s
mcp/mock-llm-mcp-github.spec.ts › MCP GitHub server install flow › step 3: full install flow — fill PAT, submit, verify installed 12.9s
mcp/mock-llm-mcp-github.spec.ts › MCP GitHub server install flow › step 4: installed GitHub server can be deleted 5.8s
mcp/mock-llm-mcp-slack-credentials.spec.ts › MCP Test Connection credential verification (Slack) › install: invalid Slack credentials are blocked with a credential-check error 5.7s
mcp/mock-llm-mcp-slack-credentials.spec.ts › MCP Test Connection credential verification (Slack) › install: a valid token missing only a scope still installs (missing_scope is not a credential failure) 5.8s
mcp/mock-llm-mcp-slack-credentials.spec.ts › MCP Test Connection credential verification (Slack) › install: an older agent server that omits tool_result still installs (compat) 5.9s
mcp/mock-llm-mcp-slack-credentials.spec.ts › MCP Test Connection credential verification (Slack) › edit: Test Connection verifies the stored credentials and surfaces a credential failure 5.7s
mcp/mock-llm-mcp-slack-credentials.spec.ts › MCP Test Connection credential verification (Slack) › edit: Test Connection reports success for valid stored credentials 5.8s
mcp/mock-llm-mcp-slack-credentials.spec.ts › MCP Test Connection credential verification (Slack) › custom (non-catalog) server: Test Connection attaches no verification probe 5.7s
onboarding/mock-llm-onboarding-happy-path.spec.ts › onboarding happy path › completes the full onboarding flow and launches a conversation 3.5s
onboarding/mock-llm-onboarding-regressions.spec.ts › onboarding recent regressions › keeps the modal open on backdrop click and Escape 1.4s
onboarding/mock-llm-onboarding-regressions.spec.ts › onboarding recent regressions › defaults the LLM setup step to OpenAI GPT-5.5 1.5s
regressions/mock-llm-ui-regressions.spec.ts › UI regressions › scopes standalone styles to the agent-server-ui shell 1.3s
regressions/mock-llm-ui-regressions.spec.ts › UI regressions › renders critic results on agent messages and finish actions 1.4s
regressions/mock-llm-ui-regressions.spec.ts › UI regressions › loads older events when scrolling up 1.6s
regressions/mock-llm-ui-regressions.spec.ts › UI regressions › selected workspace persists after navigating away and returning 2.4s
regressions/mock-llm-ui-regressions.spec.ts › UI regressions › cleared sessionStorage yields empty workspace selection 1.3s
settings/mock-llm-acp-agent.spec.ts › mock-LLM ACP agent conversation › step 1: configure ACP agent via Settings → Agent UI 13.7s
settings/mock-llm-acp-agent.spec.ts › mock-LLM ACP agent conversation › step 2: reload and verify ACP settings are persisted in UI 5.5s
settings/mock-llm-acp-agent.spec.ts › mock-LLM ACP agent conversation › step 3: start ACP conversation and verify agent reply 6.7s
settings/mock-llm-acp-agent.spec.ts › mock-LLM ACP agent conversation › step 4: resume ACP conversation from sidebar after navigating away 5.7s
settings/mock-llm-model-switch.spec.ts › mock-LLM /model slash command › step 1: configure LLM, create switch-target profile, register trajectory 13.1s
settings/mock-llm-model-switch.spec.ts › mock-LLM /model slash command › step 2: start conversation, switch profile via /model, verify switch 6.7s
settings/mock-llm-profile-management.spec.ts › active profile deletion + reconciliation › active profile is deletable and reconciliation activates another profile 8.6s
settings/mock-llm-profile-management.spec.ts › same-model profile identity › chat header shows the correct profile when two profiles share the same model 15.0s
settings/mock-llm-profile-management.spec.ts › OpenHands provider hidden base_url preservation › re-saving an OpenHands profile from Basic view preserves hidden base_url 7.5s
skills/mock-llm-skills.spec.ts › skill loading: project, user, and deletion › project skill in workspace/.agents/skills/ triggers on matching keyword 13.6s
skills/mock-llm-skills.spec.ts › skill loading: project, user, and deletion › user skill in ~/.openhands/skills/ triggers on matching keyword 13.3s
skills/mock-llm-skills.spec.ts › skill loading: project, user, and deletion › deleting a user skill removes it from subsequent conversations 13.3s

Posted by the Mock-LLM E2E workflow · results are deterministic (scripted LLM responses)

@neubig neubig requested review from all-hands-bot and hieptl June 18, 2026 20:14

all-hands-bot commented Jun 18, 2026

Copy link
Copy Markdown
Contributor

Review complete.

This review was performed through OpenHands Cloud Automation. You can log in and view the conversation here.

@all-hands-bot all-hands-bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 Acceptable — Works well with good test coverage and manual QA evidence. One unused exported function to clean up.

The implementation is clean: the dual config path (VITE_LOCK_TO_CLOUD env var at compile time + window.__AGENT_CANVAS_LOCK_TO_CLOUD__ runtime injection) is the right design for pre-built/frontend-only deployments, and the normalizeCloudHost() helper correctly handles scheme normalization and trailing-slash stripping. The dynamic onboarding step indices are well-tracked and the test coverage across unit, component, and e2e layers is solid.

[IMPROVEMENT OPPORTUNITIES]

  • [src/api/agent-server-config.ts, Line 103] Dead export: isLockedToCloud() is exported but never imported anywhere in the codebase. Every consumer calls getLockedCloudHost() !== null directly (e.g. root.tsx line 182). Either use it consistently or remove it — an exported function that's unused is dead code that adds to the module's surface area without benefit.

[STYLE NOTES]

  • [src/components/features/onboarding/onboarding-modal.tsx, Line 130] Subtle coupling: hideSkip = currentStep === 0 && getLockedCloudHost() !== null assumes the backend step is always at index 0 when the modal is locked to cloud. This holds in the current code because skipBackendStep can only be true when a backend is already healthy, which prevents the locked onboarding from showing in the first place. But the invariant isn't obvious from the variable name alone — a comment like // step 0 is the backend step whenever this modal is shown in locked mode would help future readers avoid accidentally breaking it.

[RISK ASSESSMENT]

  • [Overall PR] ⚠️ Risk Assessment: 🟢 LOW
    Feature is purely additive and opt-in via a new config flag. No existing behavior is affected when VITE_LOCK_TO_CLOUD is unset (the default). Changes to the onboarding step-skipping logic are protected by the new unit and component tests. The parent-branch changes (fix-public-onboarding) are already reviewed separately.

VERDICT:
Worth merging — core logic is sound, single minor cleanup (unused isLockedToCloud export) suggested.

KEY INSIGHT:
The dual compile-time/runtime injection pattern is the right approach for pre-built static deployments where you can't bake env vars in — runtime injection via static-server.mjs is the escape hatch that keeps the Docker image generic.


Improve this review? If any feedback above seems incorrect or irrelevant to this repository, you can teach the reviewer to do better:

  1. Add a .agents/skills/custom-codereview-guide.md file to your branch (or edit it if one already exists) with the /codereview trigger and the context the reviewer is missing (e.g., "Security concerns about X do not apply here because Y"). See the customization docs for the required frontmatter format.
  2. Re-request a review - the reviewer reads guidelines from the PR branch, so your changes take effect immediately.
  3. When your PR is merged, the guideline file goes through normal code review by repository maintainers.

Resolve with AI? Install the iterate skill in your agent and run /iterate to automatically drive this PR through CI, review, and QA until it's merge-ready.

Was this review helpful? React with 👍 or 👎 to give feedback.

This review was generated by an AI agent (OpenHands) on behalf of the user through OpenHands Automation. View conversation

return null;
}

export function isLockedToCloud(): boolean {

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 Suggestion: isLockedToCloud() is exported but has zero imports in the codebase — every callsite uses getLockedCloudHost() !== null directly. Remove it or replace the direct calls with this helper to keep the API surface intentional.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

review-test experimental cloud code review bot

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants