Skip to content

feat: add image generation MCP server and enhanced image rendering#1699

Draft
benjaminshafii wants to merge 2 commits intodevfrom
feat/image-gen-mcp
Draft

feat: add image generation MCP server and enhanced image rendering#1699
benjaminshafii wants to merge 2 commits intodevfrom
feat/image-gen-mcp

Conversation

@benjaminshafii
Copy link
Copy Markdown
Member

Summary

  • Add packages/image-gen-mcp — a standalone MCP server that exposes AI image generation as MCP tools
  • Enhance FileCard in session message list to render generated images as large inline previews

Architecture

The MCP server (openwork-image-gen-mcp) works with the existing OpenCode agent harness — no OpenCode changes required:

User prompt → Any agent (Claude, GPT, Gemini, etc.)
  → Agent calls generate_image MCP tool
  → MCP server calls OpenAI/xAI/Google API
  → Returns MCP image content { type: "image", data: base64, mimeType }
  → OpenCode converts to FilePart attachment (existing behavior)
  → OpenWork renders large inline preview (new FileCard enhancement)

MCP Server (packages/image-gen-mcp)

  • Single .mjs file, no build step, same pattern as openwork-ui-mcp
  • Supports three backends: OpenAI (gpt-image-1/dall-e-3), xAI/Grok, Google Imagen
  • Auto-discovers available providers from env vars (OPENAI_API_KEY, XAI_API_KEY, GOOGLE_API_KEY)
  • Two tools: generate_image and list_image_providers
  • Returns images as MCP image content blocks (not text), avoiding output truncation

FileCard Enhancement

  • Generated images (data URL + image MIME from assistant) now render as large inline previews (max 320px height, expandable to 800px)
  • Click to expand/collapse
  • Download button on hover
  • Existing small-thumbnail behavior preserved for user attachments and non-data-URL files

Usage

Add to opencode.json:

{
  "mcp": {
    "openwork-image-gen": {
      "type": "local",
      "command": ["node", "packages/image-gen-mcp/index.mjs"],
      "env": { "OPENAI_API_KEY": "sk-..." }
    }
  }
}

Or via the existing POST /workspace/:id/mcp endpoint.

Why MCP (not plugin or built-in tool)

  • Plugin tools return string only — base64 images get truncated by Truncate.output()
  • MCP image content bypasses text truncation entirely — OpenCode already converts it to FilePart attachments
  • No OpenCode PR needed
  • Cross-provider: works with any agent model (the image gen API call is separate from the LLM)

Testing

  • node --check packages/image-gen-mcp/index.mjs — syntax validation passes
  • Pre-existing type errors in settings-route.tsx and welcome-route.tsx — not related to this change
  • End-to-end test: configure MCP entry with OPENAI_API_KEY, ask agent to generate an image, verify FilePart arrives and renders as large preview

Future work

  • OpenWork settings UI for connecting image providers (auto-generates MCP config)
  • Capability indicator in composer when image gen is available
  • OpenCode Responses API provider-defined tool for Codex OAuth users (separate PR, ~20 lines in OpenCode)

Add @openwork/image-gen-mcp — a standalone MCP server that exposes
AI image generation (OpenAI, xAI/Grok, Google Imagen) as MCP tools.
The server reads API keys from environment variables and returns
generated images as MCP image content blocks, which OpenCode already
converts to FilePart attachments.

Also enhance FileCard in the session message list to render generated
images (data URL + image MIME from assistant) as large inline previews
with click-to-expand and download, instead of tiny 44x44 thumbnails.

No OpenCode changes required — the entire pipeline from MCP subprocess
to FilePart to SSE stream to UI already works.
@vercel
Copy link
Copy Markdown
Contributor

vercel Bot commented May 7, 2026

The latest updates on your projects. Learn more about Vercel for GitHub.

Project Deployment Actions Updated (UTC)
openwork-app Ready Ready Preview, Comment May 7, 2026 5:45pm
openwork-den Ready Ready Preview, Comment May 7, 2026 5:45pm
openwork-den-worker-proxy Ready Ready Preview, Comment May 7, 2026 5:45pm
openwork-landing Ready Ready Preview, Comment, Open in v0 May 7, 2026 5:45pm
openwork-share Ready Ready Preview, Comment May 7, 2026 5:45pm

Tool results with attachments (e.g. MCP image content) were stored
correctly by OpenCode but never extracted by the OpenWork adapter
layer. The attachments array on completed tool state was silently
dropped in toUIPart(), mapToolPart(), handleToolPart(), and
snapshotToUIMessages().

Fix all three paths:
- session-sync.ts: emit file parts alongside tool part on update
- usechat-adapter.ts handleToolPart: emit file chunks on tool output
- usechat-adapter.ts snapshotToUIMessages: flatMap tool attachments

This completes the pipeline from MCP image content through to the
enhanced FileCard renderer.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant