Skip to content

Fix keyless active LLM profile recovery#1346

Open
malhotra5 wants to merge 2 commits into
mainfrom
fix-keyless-active-llm-profile
Open

Fix keyless active LLM profile recovery#1346
malhotra5 wants to merge 2 commits into
mainfrom
fix-keyless-active-llm-profile

Conversation

@malhotra5

@malhotra5 malhotra5 commented Jun 13, 2026

Copy link
Copy Markdown
Member

HUMAN:

  • A human has tested these changes.

AGENT:

This PR was created by an AI agent (OpenHands) on behalf of the user.

Verification performed:

npm run check-translation-completeness
npm run typecheck
npx vitest run __tests__/components/home/llm-not-configured-banner.test.tsx __tests__/components/settings/llm-profiles/llm-settings-local-view.test.tsx
npm run lint

Results: all commands above passed locally.

I also attempted the targeted mock-LLM E2E regression locally:

npm run test:e2e:mock-llm -- tests/e2e/mock-llm/mock-llm-profile-management.spec.ts -g "keyless active profile"

The first attempt needed the Python openhands-sdk dependency for the mock server, which I installed. Later Playwright webServer startup/teardown reset the terminal session, so I could not obtain a reliable local pass/fail result for that targeted E2E run. The mock-LLM E2E scenario is included in this PR for CI coverage.


Why

Fixes #1344. In local mode, users could end up with an active LLM profile whose API key was missing (api_key_set: false). The home/chat UI showed a generic blocked state even though the actionable fix was to re-enter the API key for the selected profile.

Summary

  • Detect the specific local active-profile-without-key state in useLlmConfigured.
  • Show profile-specific recovery copy on the home LLM warning and route directly to /settings/llm?profile=<name>.
  • Open the requested profile editor from the LLM settings query parameter.
  • Add unit coverage and mock-LLM E2E regression coverage for the keyless active profile recovery path.

Issue Number

Fixes #1344

How to Test

  1. Create or save a local LLM profile without an API key and make it active.
  2. Navigate to the home page.
  3. Confirm the LLM warning names the active profile and explains that its API key must be re-entered.
  4. Confirm the chat submit button is disabled while the key is missing.
  5. Click the warning action.
  6. Confirm the app navigates to /settings/llm?profile=<profile-name> and opens that profile in edit mode.

Automated checks run locally are listed in the AGENT section above.

Video/Screenshots

Not captured. A mock-LLM E2E regression was added to cover the browser scenario.

Type

  • Bug fix
  • Feature
  • Refactor
  • Breaking change
  • Docs / chore

Notes

No migrations or config changes required.

@malhotra5 can click here to continue refining the PR


🐳 Docker images for this PR

GHCR package: https://github.com/OpenHands/agent-canvas/pkgs/container/agent-canvas

Component Value
Image ghcr.io/openhands/agent-canvas
Architectures amd64, arm64
Agent Server ghcr.io/openhands/agent-server:1.28.1-python
Automation openhands-automation==1.0.0a9
Commit 37306d437adca7af3580d890d4906f39e25cf75c

Pull (multi-arch manifest)

# Multi-arch manifest — Docker automatically pulls the correct architecture
docker pull ghcr.io/openhands/agent-canvas:sha-37306d4

Run

docker run -it --rm \
  -p 8000:8000 \
  ghcr.io/openhands/agent-canvas:sha-37306d4

All tags pushed for this build

ghcr.io/openhands/agent-canvas:sha-37306d4-amd64
ghcr.io/openhands/agent-canvas:fix-keyless-active-llm-profile-amd64
ghcr.io/openhands/agent-canvas:pr-1346-amd64
ghcr.io/openhands/agent-canvas:sha-37306d4-arm64
ghcr.io/openhands/agent-canvas:fix-keyless-active-llm-profile-arm64
ghcr.io/openhands/agent-canvas:pr-1346-arm64
ghcr.io/openhands/agent-canvas:sha-37306d4
ghcr.io/openhands/agent-canvas:fix-keyless-active-llm-profile
ghcr.io/openhands/agent-canvas:pr-1346

About Multi-Architecture Support

  • Each tag (e.g., sha-37306d4) is a multi-arch manifest supporting both amd64 and arm64
  • Docker automatically pulls the correct architecture for your platform
  • Individual architecture tags (e.g., sha-37306d4-amd64) are also available if needed

Detect local active LLM profiles whose API key is missing and guide users directly to the affected profile editor. Add unit and mock-LLM E2E coverage for the recovery path.

Co-authored-by: openhands <openhands@all-hands.dev>
@vercel

vercel Bot commented Jun 13, 2026

Copy link
Copy Markdown

The latest updates on your projects. Learn more about Vercel for GitHub.

Project Deployment Actions Updated (UTC)
agent-canvas Ready Ready Preview, Comment Jun 13, 2026 2:14am

Request Review

Resolve mock-LLM profile management E2E conflicts by retaining main's OpenHands provider base_url coverage and the issue #1344 keyless active profile recovery test.

Co-authored-by: openhands <openhands@all-hands.dev>
@github-actions

Copy link
Copy Markdown
Contributor

✅ Mock-LLM E2E Tests

55/55 passed

Commit: 37306d43 · Workflow run · Test artifacts

Status Test Duration
mock-llm-acp-agent.spec.ts › mock-LLM ACP agent conversation › step 1: configure ACP agent via Settings → Agent UI 13.7s
mock-llm-acp-agent.spec.ts › mock-LLM ACP agent conversation › step 2: reload and verify ACP settings are persisted in UI 5.6s
mock-llm-acp-agent.spec.ts › mock-LLM ACP agent conversation › step 3: start ACP conversation and verify agent reply 6.3s
mock-llm-acp-agent.spec.ts › mock-LLM ACP agent conversation › step 4: resume ACP conversation from sidebar after navigating away 5.8s
mock-llm-auth-modes.spec.ts › auth mode: fresh install with runtime-injected key › reaches the onboarding modal without pre-seeded localStorage 1.3s
mock-llm-auth-modes.spec.ts › auth mode: non-public key rotation › recovers when localStorage has a stale session API key 5.3s
mock-llm-auth-modes.spec.ts › auth mode: public gate › shows the auth screen when no key is configured 1.2s
mock-llm-auth-modes.spec.ts › auth mode: public gate › rejects an incorrect key with an inline error 1.4s
mock-llm-auth-modes.spec.ts › auth mode: public gate › allows access after pasting the correct key 1.7s
mock-llm-auth-modes.spec.ts › auth mode: public gate › skips auth screen for returning user with valid stored key 778ms
mock-llm-auth-modes.spec.ts › auth mode: public gate › re-prompts when the server rotates its key (stale localStorage) 1.5s
mock-llm-automation.spec.ts › mock-LLM automation lifecycle › step 1: setup LLM profile and register automation trajectory 7.2s
mock-llm-automation.spec.ts › mock-LLM automation lifecycle › step 2: create automation and dispatch run via the UI 28.4s
mock-llm-automation.spec.ts › mock-LLM automation lifecycle › step 3: verify automation and run on the automations page 6.1s
mock-llm-conversation.spec.ts › mock-LLM agent-server conversation › step 1: create an LLM profile pointing at the mock LLM server 6.3s
mock-llm-conversation.spec.ts › mock-LLM agent-server conversation › step 2: activate the mock-llm profile and verify settings API 6.2s
mock-llm-conversation.spec.ts › mock-LLM agent-server conversation › step 3: run a conversation with the mock LLM 6.6s
mock-llm-conversation.spec.ts › mock-LLM agent-server conversation › step 4: resume conversation from sidebar after navigating away 5.7s
mock-llm-cross-connect.spec.ts › cross-connect: frontend-only → backend-only › frontend-only connects to a separate backend-only instance 15.9s
mock-llm-cross-connect.spec.ts › cross-connect: frontend-only → multiple backends › connects to two separate backends and switches between them 20.7s
mock-llm-files-and-git.spec.ts › files tab, git control bar, and browser tab › step 1: ensure mock LLM profile is configured 208ms
mock-llm-files-and-git.spec.ts › files tab, git control bar, and browser tab › step 2: start conversation and attach workspace metadata 11.7s
mock-llm-files-and-git.spec.ts › files tab, git control bar, and browser tab › step 3: git control bar shows workspace pill and git actions 25.3s
mock-llm-files-and-git.spec.ts › files tab, git control bar, and browser tab › step 4: files tab defaults to diff view for attached workspace 5.9s
mock-llm-files-and-git.spec.ts › files tab, git control bar, and browser tab › step 5: browser tab shows empty state 6.3s
mock-llm-files-and-git.spec.ts › files tab, git control bar, and browser tab › step 6: files tab defaults to file-tree view without attached workspace 8.2s
mock-llm-folder-workspace.spec.ts › mock-LLM folder browser → workspace → conversation › step 1: browse to a folder, add it as a workspace, and launch a conversation with the correct working_dir 8.6s
mock-llm-image-upload.spec.ts › mock-LLM image upload › attaching an image embeds it as base64 in the LLM completion call 13.9s
mock-llm-mcp-github.spec.ts › MCP GitHub server install flow › step 1: GitHub card is visible on the MCP marketplace page 5.6s
mock-llm-mcp-github.spec.ts › MCP GitHub server install flow › step 2: clicking GitHub card opens the install modal with correct fields 5.7s
mock-llm-mcp-github.spec.ts › MCP GitHub server install flow › step 3: full install flow — fill PAT, submit, verify installed 13.0s
mock-llm-mcp-github.spec.ts › MCP GitHub server install flow › step 4: installed GitHub server can be deleted 5.8s
mock-llm-model-switch.spec.ts › mock-LLM /model slash command › step 1: configure LLM, create switch-target profile, register trajectory 13.1s
mock-llm-model-switch.spec.ts › mock-LLM /model slash command › step 2: start conversation, switch profile via /model, verify switch 7.0s
mock-llm-onboarding-happy-path.spec.ts › onboarding happy path › completes the full onboarding flow and launches a conversation 4.5s
mock-llm-onboarding-regressions.spec.ts › onboarding recent regressions › keeps the modal open on backdrop click and Escape 1.4s
mock-llm-onboarding-regressions.spec.ts › onboarding recent regressions › defaults the LLM setup step to OpenAI GPT-5.5 1.6s
mock-llm-partial-stack.spec.ts › partial stack: --frontend-only › serves the frontend but returns 503 for backend routes 7.3s
mock-llm-partial-stack.spec.ts › partial stack: --backend-only › serves backend APIs but returns 503 for the frontend root 14.1s
mock-llm-partial-stack.spec.ts › partial stack: port conflict › fails with a clear error when the ingress port is occupied 120ms
mock-llm-partial-stack.spec.ts › partial stack: port conflict › starts successfully on a free port after a conflict 6.0s
mock-llm-preset-automation.spec.ts › preset automation → slash command conversation › automation card sends the correct slash command to a conversation 16.5s
mock-llm-preset-automation.spec.ts › preset automation → slash command conversation › direct slash command from home page triggers skill activation 14.4s
mock-llm-profile-management.spec.ts › active profile deletion + reconciliation › active profile is deletable and reconciliation activates another profile 8.7s
mock-llm-profile-management.spec.ts › same-model profile identity › chat header shows the correct profile when two profiles share the same model 16.0s
mock-llm-profile-management.spec.ts › OpenHands provider base_url normalization › re-saving an OpenHands profile from Basic view drops proxy base_url 8.1s
mock-llm-profile-management.spec.ts › active profile missing API key recovery › home page explains and links directly to a keyless active profile 5.6s
mock-llm-skills.spec.ts › skill loading: project, user, and deletion › project skill in workspace/.agents/skills/ triggers on matching keyword 14.5s
mock-llm-skills.spec.ts › skill loading: project, user, and deletion › user skill in ~/.openhands/skills/ triggers on matching keyword 14.3s
mock-llm-skills.spec.ts › skill loading: project, user, and deletion › deleting a user skill removes it from subsequent conversations 14.1s
mock-llm-ui-regressions.spec.ts › UI regressions › scopes standalone styles to the agent-server-ui shell 1.3s
mock-llm-ui-regressions.spec.ts › UI regressions › renders critic results on agent messages and finish actions 1.5s
mock-llm-ui-regressions.spec.ts › UI regressions › loads older events when scrolling up 1.7s
mock-llm-ui-regressions.spec.ts › UI regressions › selected workspace persists after navigating away and returning 2.5s
mock-llm-ui-regressions.spec.ts › UI regressions › cleared sessionStorage yields empty workspace selection 996ms

Posted by the Mock-LLM E2E workflow · results are deterministic (scripted LLM responses)

@github-actions

Copy link
Copy Markdown
Contributor

🔶 Mock-LLM Docker E2E Test Results

50/55 passed · 5 skipped

Commit: 37306d43 · Workflow run · Test artifacts

Status Test Duration
mock-llm-acp-agent.spec.ts › mock-LLM ACP agent conversation › step 1: configure ACP agent via Settings → Agent UI 13.8s
mock-llm-acp-agent.spec.ts › mock-LLM ACP agent conversation › step 2: reload and verify ACP settings are persisted in UI 5.6s
mock-llm-acp-agent.spec.ts › mock-LLM ACP agent conversation › step 3: start ACP conversation and verify agent reply 6.8s
mock-llm-acp-agent.spec.ts › mock-LLM ACP agent conversation › step 4: resume ACP conversation from sidebar after navigating away 5.7s
mock-llm-auth-modes.spec.ts › auth mode: fresh install with runtime-injected key › reaches the onboarding modal without pre-seeded localStorage 1.3s
mock-llm-auth-modes.spec.ts › auth mode: non-public key rotation › recovers when localStorage has a stale session API key 5.3s
mock-llm-auth-modes.spec.ts › auth mode: public gate › shows the auth screen when no key is configured 1.2s
mock-llm-auth-modes.spec.ts › auth mode: public gate › rejects an incorrect key with an inline error 1.4s
mock-llm-auth-modes.spec.ts › auth mode: public gate › allows access after pasting the correct key 1.8s
mock-llm-auth-modes.spec.ts › auth mode: public gate › skips auth screen for returning user with valid stored key 783ms
mock-llm-auth-modes.spec.ts › auth mode: public gate › re-prompts when the server rotates its key (stale localStorage) 1.5s
mock-llm-automation.spec.ts › mock-LLM automation lifecycle › step 1: setup LLM profile and register automation trajectory 7.1s
mock-llm-automation.spec.ts › mock-LLM automation lifecycle › step 2: create automation and dispatch run via the UI 35.4s
mock-llm-automation.spec.ts › mock-LLM automation lifecycle › step 3: verify automation and run on the automations page 6.1s
mock-llm-conversation.spec.ts › mock-LLM agent-server conversation › step 1: create an LLM profile pointing at the mock LLM server 6.3s
mock-llm-conversation.spec.ts › mock-LLM agent-server conversation › step 2: activate the mock-llm profile and verify settings API 6.2s
mock-llm-conversation.spec.ts › mock-LLM agent-server conversation › step 3: run a conversation with the mock LLM 6.3s
mock-llm-conversation.spec.ts › mock-LLM agent-server conversation › step 4: resume conversation from sidebar after navigating away 5.7s
⏭️ mock-llm-cross-connect.spec.ts › cross-connect: frontend-only → backend-only › frontend-only connects to a separate backend-only instance 201ms
⏭️ mock-llm-cross-connect.spec.ts › cross-connect: frontend-only → multiple backends › connects to two separate backends and switches between them 181ms
mock-llm-files-and-git.spec.ts › files tab, git control bar, and browser tab › step 1: ensure mock LLM profile is configured 182ms
mock-llm-files-and-git.spec.ts › files tab, git control bar, and browser tab › step 2: start conversation and attach workspace metadata 11.4s
mock-llm-files-and-git.spec.ts › files tab, git control bar, and browser tab › step 3: git control bar shows workspace pill and git actions 25.3s
mock-llm-files-and-git.spec.ts › files tab, git control bar, and browser tab › step 4: files tab defaults to diff view for attached workspace 5.9s
mock-llm-files-and-git.spec.ts › files tab, git control bar, and browser tab › step 5: browser tab shows empty state 6.3s
mock-llm-files-and-git.spec.ts › files tab, git control bar, and browser tab › step 6: files tab defaults to file-tree view without attached workspace 7.2s
mock-llm-folder-workspace.spec.ts › mock-LLM folder browser → workspace → conversation › step 1: browse to a folder, add it as a workspace, and launch a conversation with the correct working_dir 7.3s
mock-llm-image-upload.spec.ts › mock-LLM image upload › attaching an image embeds it as base64 in the LLM completion call 13.4s
mock-llm-mcp-github.spec.ts › MCP GitHub server install flow › step 1: GitHub card is visible on the MCP marketplace page 5.5s
mock-llm-mcp-github.spec.ts › MCP GitHub server install flow › step 2: clicking GitHub card opens the install modal with correct fields 5.7s
mock-llm-mcp-github.spec.ts › MCP GitHub server install flow › step 3: full install flow — fill PAT, submit, verify installed 13.0s
mock-llm-mcp-github.spec.ts › MCP GitHub server install flow › step 4: installed GitHub server can be deleted 5.8s
mock-llm-model-switch.spec.ts › mock-LLM /model slash command › step 1: configure LLM, create switch-target profile, register trajectory 13.2s
mock-llm-model-switch.spec.ts › mock-LLM /model slash command › step 2: start conversation, switch profile via /model, verify switch 6.6s
mock-llm-onboarding-happy-path.spec.ts › onboarding happy path › completes the full onboarding flow and launches a conversation 3.5s
mock-llm-onboarding-regressions.spec.ts › onboarding recent regressions › keeps the modal open on backdrop click and Escape 1.5s
mock-llm-onboarding-regressions.spec.ts › onboarding recent regressions › defaults the LLM setup step to OpenAI GPT-5.5 1.7s
⏭️ mock-llm-partial-stack.spec.ts › partial stack: --frontend-only › serves the frontend but returns 503 for backend routes 220ms
mock-llm-partial-stack.spec.ts › partial stack: --backend-only › serves backend APIs but returns 503 for the frontend root 25.2s
⏭️ mock-llm-partial-stack.spec.ts › partial stack: port conflict › fails with a clear error when the ingress port is occupied 1ms
⏭️ mock-llm-partial-stack.spec.ts › partial stack: port conflict › starts successfully on a free port after a conflict 3ms
mock-llm-preset-automation.spec.ts › preset automation → slash command conversation › automation card sends the correct slash command to a conversation 15.8s
mock-llm-preset-automation.spec.ts › preset automation → slash command conversation › direct slash command from home page triggers skill activation 13.4s
mock-llm-profile-management.spec.ts › active profile deletion + reconciliation › active profile is deletable and reconciliation activates another profile 8.6s
mock-llm-profile-management.spec.ts › same-model profile identity › chat header shows the correct profile when two profiles share the same model 15.1s
mock-llm-profile-management.spec.ts › OpenHands provider base_url normalization › re-saving an OpenHands profile from Basic view drops proxy base_url 8.7s
mock-llm-profile-management.spec.ts › active profile missing API key recovery › home page explains and links directly to a keyless active profile 5.6s
mock-llm-skills.spec.ts › skill loading: project, user, and deletion › project skill in workspace/.agents/skills/ triggers on matching keyword 13.5s
mock-llm-skills.spec.ts › skill loading: project, user, and deletion › user skill in ~/.openhands/skills/ triggers on matching keyword 13.3s
mock-llm-skills.spec.ts › skill loading: project, user, and deletion › deleting a user skill removes it from subsequent conversations 13.4s
mock-llm-ui-regressions.spec.ts › UI regressions › scopes standalone styles to the agent-server-ui shell 1.4s
mock-llm-ui-regressions.spec.ts › UI regressions › renders critic results on agent messages and finish actions 1.5s
mock-llm-ui-regressions.spec.ts › UI regressions › loads older events when scrolling up 1.6s
mock-llm-ui-regressions.spec.ts › UI regressions › selected workspace persists after navigating away and returning 2.1s
mock-llm-ui-regressions.spec.ts › UI regressions › cleared sessionStorage yields empty workspace selection 980ms

Posted by the Mock-LLM E2E workflow · results are deterministic (scripted LLM responses)

@malhotra5 malhotra5 added the review-test experimental cloud code review bot label Jun 17, 2026

all-hands-bot commented Jun 17, 2026

Copy link
Copy Markdown
Contributor

Review complete.

This review was performed through OpenHands Cloud Automation. You can log in and view the conversation here.

Outcome: APPROVE — see review comment.

@all-hands-bot all-hands-bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟢 Good taste — clean, well-scoped fix.

The change closes a real UX gap in local mode: when the active LLM profile exists but its API key is missing, the home banner now explains the actual problem and deep-links straight into the profile editor instead of dumping the user on a generic setup screen. Approving.

What works well

  • Right data structure. A small LLM_CONFIGURATION_ISSUES enum + discriminated hook return (issue, activeProfileName) keeps the banner dumb and the classification centralized in useLlmConfigured. Easy to extend if more recovery states show up.
  • Centralized route construction. buildLlmSettingsRoute(profileName) in src/constants/llm-settings.ts is the right place to own the URL shape and URL-encode the profile name — callers don't have to think about it.
  • Idempotent URL → editor effect. requestedEditProfileNameRef is the correct guard for "respond to a query param on mount" in React: it survives profile-list refetches without re-firing, and correctly handles a later ?profile=… change.
  • Local-only when it matters. hasActiveProfileMissingApiKey is gated on isLocal, matching the existing hasUsableLlm local-vs-cloud split — no spurious banners in cloud mode.
  • Test coverage. New unit test for the banner + a Playwright E2E that exercises the full home → settings editor flow against the real backend. CI confirms both pass (mock-LLM E2E: 55/55).

Minor notes (non-blocking)

  • The LLM_CONFIGURATION_ISSUES enum currently has only two values and one consumer. A plain boolean would suffice today, but keeping the enum is cheap and avoids a refactor if a third state appears. Keep as is.
  • The useEffect deps include handleEditProfile and profilesData?.profiles. The ref guard makes duplicate fires harmless, but profilesData?.profiles changes reference on every query refetch — the effect will re-run frequently. Functionally correct; if it ever shows up in a profile, narrow the dep to a derived boolean (e.g., profilesData ? "loaded" : "pending").

[RISK ASSESSMENT]

  • [Overall PR] ⚠️ Risk Assessment: 🟢 LOW
    Scoped to local-mode profile handling, no API or schema changes, opt-in ?profile= query parameter, backward-compatible with existing URLs and cloud backends.

VERDICT:
Worth merging. Core logic is sound, tests cover both unit and E2E paths, and the change follows the codebase's existing local/cloud split conventions.

This review was generated by an AI agent (OpenHands) on behalf of the user through OpenHands Automation. View conversation

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

review-test experimental cloud code review bot

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Clarify home-page error when active LLM profile is missing API key

3 participants