Skip to content

Show automation run error detail#1413

Draft
DevinVinson wants to merge 1 commit into
OpenHands:mainfrom
DevinVinson:dv/automation-error-data-output
Draft

Show automation run error detail#1413
DevinVinson wants to merge 1 commit into
OpenHands:mainfrom
DevinVinson:dv/automation-error-data-output

Conversation

@DevinVinson

Copy link
Copy Markdown
Member

HUMAN:

I stumbled into this error detail being available from the automations server and wanted to help get it surfaced.

  • A human has tested these changes.

AGENT:


Why

I want to surface the errror_detail content if available without getting rid of the other bits. I didn't dig into what is already showing and why.

Summary

  • Surface error_detail in automation run logs alongside fetched bash output
  • Keep the logs button available when a run has debug info from either a bash command or an error detail
  • Add coverage for rendering error details without replacing existing stdout
  • Add the localized label for the new error detail section

Issue Number

#1037 related although it doesn't "fix" this.

How to Test

Video/Screenshots

Type

  • Bug fix
  • Feature
  • Refactor
  • Breaking change
  • Docs / chore

Notes

@vercel

vercel Bot commented Jun 17, 2026

Copy link
Copy Markdown

@DevinVinson is attempting to deploy a commit to the openhands Team on Vercel.

A member of the Team first needs to authorize it.

@github-actions

Copy link
Copy Markdown
Contributor

✅ Mock-LLM E2E Tests

60/60 passed

Commit: beb85edf · Workflow run · Test artifacts

Status Test Duration
automations/mock-llm-automation.spec.ts › mock-LLM automation lifecycle › step 1: setup LLM profile and register automation trajectory 7.2s
automations/mock-llm-automation.spec.ts › mock-LLM automation lifecycle › step 2: create automation and dispatch run via the UI 27.4s
automations/mock-llm-automation.spec.ts › mock-LLM automation lifecycle › step 3: verify automation and run on the automations page 6.2s
automations/mock-llm-preset-automation.spec.ts › preset automation → slash command conversation › automation card sends the correct slash command to a conversation 15.8s
automations/mock-llm-preset-automation.spec.ts › preset automation → slash command conversation › direct slash command from home page triggers skill activation 13.5s
backends/mock-llm-auth-modes.spec.ts › auth mode: fresh install with runtime-injected key › reaches the onboarding modal without pre-seeded localStorage 1.3s
backends/mock-llm-auth-modes.spec.ts › auth mode: non-public key rotation › recovers when localStorage has a stale session API key 5.3s
backends/mock-llm-auth-modes.spec.ts › auth mode: public gate › shows the auth screen when no key is configured 1.2s
backends/mock-llm-auth-modes.spec.ts › auth mode: public gate › rejects an incorrect key with an inline error 1.4s
backends/mock-llm-auth-modes.spec.ts › auth mode: public gate › allows access after pasting the correct key 1.4s
backends/mock-llm-auth-modes.spec.ts › auth mode: public gate › skips auth screen for returning user with valid stored key 708ms
backends/mock-llm-auth-modes.spec.ts › auth mode: public gate › re-prompts when the server rotates its key (stale localStorage) 1.4s
backends/mock-llm-cross-connect.spec.ts › cross-connect: frontend-only → backend-only › frontend-only connects to a separate backend-only instance 15.8s
backends/mock-llm-cross-connect.spec.ts › cross-connect: frontend-only → multiple backends › connects to two separate backends and switches between them 20.7s
backends/mock-llm-partial-stack.spec.ts › partial stack: --frontend-only › serves the frontend but returns 503 for backend routes 7.3s
backends/mock-llm-partial-stack.spec.ts › partial stack: --backend-only › serves backend APIs but returns 503 for the frontend root 13.1s
backends/mock-llm-partial-stack.spec.ts › partial stack: port conflict › fails with a clear error when the ingress port is occupied 105ms
backends/mock-llm-partial-stack.spec.ts › partial stack: port conflict › starts successfully on a free port after a conflict 6.0s
conversations/mock-llm-conversation.spec.ts › mock-LLM agent-server conversation › step 1: create an LLM profile pointing at the mock LLM server 6.2s
conversations/mock-llm-conversation.spec.ts › mock-LLM agent-server conversation › step 2: activate the mock-llm profile and verify settings API 6.1s
conversations/mock-llm-conversation.spec.ts › mock-LLM agent-server conversation › step 3: run a conversation with the mock LLM 7.3s
conversations/mock-llm-conversation.spec.ts › mock-LLM agent-server conversation › step 4: resume conversation from sidebar after navigating away 5.7s
conversations/mock-llm-image-upload.spec.ts › mock-LLM image upload › attaching an image embeds it as base64 in the LLM completion call 13.3s
files/mock-llm-files-and-git.spec.ts › files tab, git control bar, and browser tab › step 1: ensure mock LLM profile is configured 7.2s
files/mock-llm-files-and-git.spec.ts › files tab, git control bar, and browser tab › step 2: start conversation and attach workspace metadata 11.8s
files/mock-llm-files-and-git.spec.ts › files tab, git control bar, and browser tab › step 3: git control bar shows workspace pill and git actions 25.3s
files/mock-llm-files-and-git.spec.ts › files tab, git control bar, and browser tab › step 4: files tab defaults to diff view for attached workspace 5.8s
files/mock-llm-files-and-git.spec.ts › files tab, git control bar, and browser tab › step 5: browser tab shows empty state 6.3s
files/mock-llm-files-and-git.spec.ts › files tab, git control bar, and browser tab › step 6: files tab defaults to file-tree view without attached workspace 7.4s
home/mock-llm-folder-workspace.spec.ts › mock-LLM folder browser → workspace → conversation › step 1: browse to a folder, add it as a workspace, and launch a conversation with the correct working_dir 7.7s
mcp/mock-llm-mcp-github.spec.ts › MCP GitHub server install flow › step 1: GitHub card is visible on the MCP marketplace page 5.5s
mcp/mock-llm-mcp-github.spec.ts › MCP GitHub server install flow › step 2: clicking GitHub card opens the install modal with correct fields 5.7s
mcp/mock-llm-mcp-github.spec.ts › MCP GitHub server install flow › step 3: full install flow — fill PAT, submit, verify installed 12.8s
mcp/mock-llm-mcp-github.spec.ts › MCP GitHub server install flow › step 4: installed GitHub server can be deleted 5.8s
mcp/mock-llm-mcp-slack-credentials.spec.ts › MCP Test Connection credential verification (Slack) › install: invalid Slack credentials are blocked with a credential-check error 5.8s
mcp/mock-llm-mcp-slack-credentials.spec.ts › MCP Test Connection credential verification (Slack) › install: a valid token missing only a scope still installs (missing_scope is not a credential failure) 5.9s
mcp/mock-llm-mcp-slack-credentials.spec.ts › MCP Test Connection credential verification (Slack) › install: an older agent server that omits tool_result still installs (compat) 5.9s
mcp/mock-llm-mcp-slack-credentials.spec.ts › MCP Test Connection credential verification (Slack) › edit: Test Connection verifies the stored credentials and surfaces a credential failure 5.7s
mcp/mock-llm-mcp-slack-credentials.spec.ts › MCP Test Connection credential verification (Slack) › edit: Test Connection reports success for valid stored credentials 5.8s
mcp/mock-llm-mcp-slack-credentials.spec.ts › MCP Test Connection credential verification (Slack) › custom (non-catalog) server: Test Connection attaches no verification probe 5.7s
onboarding/mock-llm-onboarding-happy-path.spec.ts › onboarding happy path › completes the full onboarding flow and launches a conversation 4.5s
onboarding/mock-llm-onboarding-regressions.spec.ts › onboarding recent regressions › keeps the modal open on backdrop click and Escape 1.3s
onboarding/mock-llm-onboarding-regressions.spec.ts › onboarding recent regressions › defaults the LLM setup step to OpenAI GPT-5.5 1.7s
regressions/mock-llm-ui-regressions.spec.ts › UI regressions › scopes standalone styles to the agent-server-ui shell 1.3s
regressions/mock-llm-ui-regressions.spec.ts › UI regressions › renders critic results on agent messages and finish actions 1.5s
regressions/mock-llm-ui-regressions.spec.ts › UI regressions › loads older events when scrolling up 1.5s
regressions/mock-llm-ui-regressions.spec.ts › UI regressions › selected workspace persists after navigating away and returning 2.0s
regressions/mock-llm-ui-regressions.spec.ts › UI regressions › cleared sessionStorage yields empty workspace selection 918ms
settings/mock-llm-acp-agent.spec.ts › mock-LLM ACP agent conversation › step 1: configure ACP agent via Settings → Agent UI 13.6s
settings/mock-llm-acp-agent.spec.ts › mock-LLM ACP agent conversation › step 2: reload and verify ACP settings are persisted in UI 5.5s
settings/mock-llm-acp-agent.spec.ts › mock-LLM ACP agent conversation › step 3: start ACP conversation and verify agent reply 6.2s
settings/mock-llm-acp-agent.spec.ts › mock-LLM ACP agent conversation › step 4: resume ACP conversation from sidebar after navigating away 5.7s
settings/mock-llm-model-switch.spec.ts › mock-LLM /model slash command › step 1: configure LLM, create switch-target profile, register trajectory 13.0s
settings/mock-llm-model-switch.spec.ts › mock-LLM /model slash command › step 2: start conversation, switch profile via /model, verify switch 7.0s
settings/mock-llm-profile-management.spec.ts › active profile deletion + reconciliation › active profile is deletable and reconciliation activates another profile 8.5s
settings/mock-llm-profile-management.spec.ts › same-model profile identity › chat header shows the correct profile when two profiles share the same model 15.2s
settings/mock-llm-profile-management.spec.ts › OpenHands provider hidden base_url preservation › re-saving an OpenHands profile from Basic view preserves hidden base_url 7.2s
skills/mock-llm-skills.spec.ts › skill loading: project, user, and deletion › project skill in workspace/.agents/skills/ triggers on matching keyword 13.5s
skills/mock-llm-skills.spec.ts › skill loading: project, user, and deletion › user skill in ~/.openhands/skills/ triggers on matching keyword 13.4s
skills/mock-llm-skills.spec.ts › skill loading: project, user, and deletion › deleting a user skill removes it from subsequent conversations 13.4s

Posted by the Mock-LLM E2E workflow · results are deterministic (scripted LLM responses)

@malhotra5 malhotra5 added the review-test experimental cloud code review bot label Jun 17, 2026

all-hands-bot commented Jun 17, 2026

Copy link
Copy Markdown
Contributor

Review complete.

This review was performed through OpenHands Cloud Automation. You can log in and view the conversation here.

@all-hands-bot all-hands-bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 Taste Rating: Acceptable — the implementation is small and the core UI path is sound, but the PR still lacks the required runtime evidence for a visible UI change.

Reviewer verification:

  • npm run test -- --run src/components/features/automations/detail/run-logs-modal.test.tsx
  • npm run typecheck
  • npm run build

[TESTING GAPS]

  • [PR description] No end-to-end evidence: The How to Test and Video/Screenshots sections are empty. This changes visible automation-run UI behavior, so please add concrete evidence from the real product path — for example, a screenshot/video of a failed automation run with error_detail shown in the logs modal, plus the exact steps or commands used to produce it. The unit test is useful, but by itself it does not prove the backend payload and automation detail page work end-to-end.

[RISK ASSESSMENT]

  • [Overall PR] ⚠️ Risk Assessment: 🟡 MEDIUM
    Localized frontend-only change with no dependency or backend API changes, and the code-level data flow is straightforward. Risk is medium because this surfaces backend-provided automation error detail in the UI and changes the debugging path users rely on; the missing runtime evidence should be addressed before approval.

VERDICT:
Needs more evidence before approval: I did not find a code correctness blocker, but the PR description needs concrete UI/runtime evidence before this should be approved.

KEY INSIGHT:
AutomationRun.error_detail is treated as supplemental debug information, preserving fetched stdout/stderr instead of replacing them — that is the right data-flow shape for this feature.


Improve this review? If any feedback above seems incorrect or irrelevant to this repository, you can teach the reviewer to do better:

  1. Add a .agents/skills/custom-codereview-guide.md file to your branch (or edit it if one already exists) with the /codereview trigger and the context the reviewer is missing (e.g., "Security concerns about X do not apply here because Y"). See the customization docs for the required frontmatter format.
  2. Re-request a review - the reviewer reads guidelines from the PR branch, so your changes take effect immediately.
  3. When your PR is merged, the guideline file goes through normal code review by repository maintainers.

Resolve with AI? Install the iterate skill in your agent and run /iterate to automatically drive this PR through CI, review, and QA until it's merge-ready.

Was this review helpful? React with 👍 or 👎 to give feedback.

This review was generated by an AI agent (OpenHands) on behalf of the user through OpenHands Automation. View conversation

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

review-test experimental cloud code review bot

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants