visual-test #5
Workflow file for this run
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| name: visual-test | |
| on: | |
| workflow_dispatch: | |
| jobs: | |
| visual-test: | |
| runs-on: blacksmith-2vcpu-ubuntu-2404 | |
| timeout-minutes: 30 | |
| permissions: | |
| id-token: write | |
| contents: read | |
| issues: write | |
| steps: | |
| - uses: actions/checkout@v4 | |
| - name: Install Nix (primary) | |
| id: nix_install_primary | |
| continue-on-error: true | |
| uses: DeterminateSystems/nix-installer-action@v16 | |
| - name: Install Nix (fallback) | |
| if: steps.nix_install_primary.outcome == 'failure' | |
| uses: cachix/install-nix-action@v31 | |
| with: | |
| extra_nix_config: | | |
| experimental-features = nix-command flakes | |
| - name: Verify Nix installation | |
| run: nix --version | |
| - name: Cache Nix Store | |
| continue-on-error: true | |
| uses: nix-community/cache-nix-action@v7 | |
| with: | |
| primary-key: nix-${{ runner.os }}-${{ hashFiles('flake.nix', 'flake.lock') }} | |
| restore-prefixes-first-match: nix-${{ runner.os }}- | |
| paths: ~/.cache/nix | |
| - name: Start headless Wayland compositor | |
| run: | | |
| mkdir -p /tmp/runtime-runner | |
| chmod 700 /tmp/runtime-runner | |
| mkdir -p /tmp/opencode-cache | |
| export XDG_RUNTIME_DIR=/tmp/runtime-runner | |
| nix develop --command weston --socket=headless --backend=headless-backend.so --width=1280 --height=720 & | |
| echo "WAYLAND_DISPLAY=headless" >> $GITHUB_ENV | |
| echo "XDG_RUNTIME_DIR=/tmp/runtime-runner" >> $GITHUB_ENV | |
| echo "XDG_CACHE_HOME=/tmp/opencode-cache" >> $GITHUB_ENV | |
| sleep 5 | |
| - name: Ensure visual-test label exists | |
| run: | | |
| if ! gh label list --json name --jq '.[].name' | grep -q '^visual-test$'; then | |
| gh label create "visual-test" \ | |
| --description "Issues from automated visual regression tests" \ | |
| --color "E06C75" | |
| fi | |
| env: | |
| GH_TOKEN: ${{ secrets.GITHUB_TOKEN }} | |
| - name: Run menu screenshot capture | |
| id: screenshot | |
| continue-on-error: true | |
| env: | |
| ZIG_GLOBAL_CACHE_DIR: /tmp/zig-cache-global | |
| XDG_RUNTIME_DIR: /tmp/runtime-runner | |
| WAYLAND_DISPLAY: headless | |
| ZIGCRAFT_SMOKE_FRAMES: "5" | |
| ZIGCRAFT_SAFE_RENDER: "1" | |
| run: | | |
| LVP_PATH=$(nix build --no-link --print-out-paths nixpkgs#mesa.drivers)/share/vulkan/icd.d/lvp_icd.x86_64.json | |
| LAYER_PATH=$(nix build --no-link --print-out-paths nixpkgs#vulkan-validation-layers)/share/vulkan/explicit_layer.d | |
| export VK_ICD_FILENAMES=$LVP_PATH | |
| export VK_LAYER_PATH=$LAYER_PATH | |
| nix develop --command zig build run -Dscreenshot-path=screenshot.ppm -Dskip-present=true 2>&1 | tee build-output.log | |
| - name: Convert PPM to PNG | |
| id: convert | |
| if: always() | |
| run: | | |
| if [ -f screenshot.ppm ]; then | |
| nix run nixpkgs#imagemagick -- screenshot.ppm screenshot.png | |
| echo "screenshot_exists=true" >> $GITHUB_OUTPUT | |
| else | |
| echo "screenshot_exists=false" >> $GITHUB_OUTPUT | |
| fi | |
| - name: Upload screenshot artifact | |
| if: steps.convert.outputs.screenshot_exists == 'true' | |
| uses: actions/upload-artifact@v7 | |
| with: | |
| name: menu-screenshot | |
| path: | | |
| screenshot.png | |
| screenshot.ppm | |
| retention-days: 30 | |
| - name: Check build log exists | |
| id: check_log | |
| if: always() | |
| run: 'test -f build-output.log && echo "exists=true" >> $GITHUB_OUTPUT || echo "exists=false" >> $GITHUB_OUTPUT' | |
| - name: Upload build log artifact | |
| if: always() && steps.check_log.outputs.exists == 'true' | |
| uses: actions/upload-artifact@v7 | |
| with: | |
| name: build-output-log | |
| path: build-output.log | |
| retention-days: 7 | |
| - name: Run opencode visual verification | |
| if: steps.convert.outputs.screenshot_exists == 'true' | |
| uses: anomalyco/opencode/github@v1.3.3 | |
| env: | |
| GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }} | |
| KIMI_API_KEY: ${{ secrets.KIMI_API_KEY }} | |
| with: | |
| model: kimi-for-coding/k2p5 | |
| prompt: | | |
| You are a visual QA tester for a game engine. A screenshot of the main menu has been captured during a headless CI run using Lavapipe (software Vulkan). | |
| ## YOUR TASK | |
| 1. Read the file `screenshot.png` in the workspace root. This is a screenshot of the ZigCraft main menu. | |
| 2. Analyze the screenshot and verify ALL of the following: | |
| - The screen is NOT blank, black, or empty | |
| - The title "ZIG VOXEL ENGINE" is visible at the top | |
| - At least 2 menu buttons are visible (e.g., SINGLEPLAYER, SETTINGS, QUIT) | |
| - The UI layout looks reasonable (buttons centered, not overlapping, not off-screen) | |
| - There are no obvious rendering artifacts (complete blackness, garbled pixels, missing geometry) | |
| 3. If the screenshot passes ALL checks: | |
| - Do nothing. A passing test means silence. | |
| - Run: `echo "VISUAL TEST PASSED: Menu screenshot looks correct"` | |
| 4. If the screenshot FAILS any check: | |
| - Create a GitHub issue with label `visual-test` describing exactly what is wrong. | |
| - Use this title format: `[Visual Test] Menu screenshot shows {problem}` | |
| - Include the check that failed and a description of what you see vs what you expected. | |
| ## CRITICAL CONSTRAINTS | |
| - You may ONLY create a single GitHub issue. Do NOT create branches, PRs, or modify files. | |
| - If the screenshot looks correct, file NO issue. Silence is better than noise. | |
| - Be lenient with software rendering artifacts (Lavapipe may not render identically to a real GPU). | |
| - Font rendering may be slightly different in software mode - that is acceptable. | |
| - The important thing is that the menu is FUNCTIONAL (visible text, clickable buttons, proper layout). | |
| - name: Run opencode failure diagnosis | |
| if: failure() || steps.screenshot.outcome == 'failure' || steps.convert.outputs.screenshot_exists != 'true' | |
| uses: anomalyco/opencode/github@v1.3.3 | |
| env: | |
| GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }} | |
| KIMI_API_KEY: ${{ secrets.KIMI_API_KEY }} | |
| with: | |
| model: kimi-for-coding/k2p5 | |
| prompt: | | |
| You are a senior systems programmer debugging a CI failure in a Zig voxel engine project. | |
| ## SITUATION | |
| The automated visual test workflow failed. This test: | |
| 1. Starts a headless Weston compositor (1280x720) | |
| 2. Builds and runs the game with Lavapipe (software Vulkan): `zig build run -Dscreenshot-path=screenshot.ppm -Dskip-present=true` | |
| 3. The game should render the HomeScreen menu for 5 frames, capture a screenshot as PPM, and exit | |
| 4. The PPM is then converted to PNG | |
| Something went wrong. Your job is to diagnose the failure and file a detailed GitHub issue. | |
| ## YOUR TASK | |
| 1. **Read the build log**: Read `build-output.log` in the workspace root. This contains the full stdout/stderr from the game run. | |
| 2. **Check for common failure patterns** in the log: | |
| - Vulkan instance/device creation failures | |
| - Swapchain creation errors (especially in headless mode) | |
| - Shader compilation failures | |
| - Screenshot capture errors (staging buffer, fence timeout, PPM write) | |
| - Application panics or segfaults | |
| - Missing files or environment issues | |
| 3. **Read relevant source code** to understand the failure: | |
| - `src/engine/graphics/vulkan/screenshot.zig` — screenshot capture implementation | |
| - `src/engine/graphics/vulkan_swapchain.zig` — headless swapchain setup (look at `headless_mode` path) | |
| - `src/game/app.zig` — screenshot mode initialization and frame counting | |
| - `src/game/screens/home.zig` — the HomeScreen that should be rendered | |
| - `src/engine/graphics/rhi_vulkan.zig` — RHI initialization | |
| 4. **Diagnose root cause**: Based on the log output and code, determine what went wrong and why. | |
| 5. **File a GitHub issue** with label `visual-test` containing: | |
| - Title: `[Visual Test] {concise description of the failure}` | |
| - The exact error output from the log (relevant lines only, not the entire log) | |
| - Your diagnosis of the root cause | |
| - The specific file and function where the failure originates | |
| - A suggested fix (code snippet if applicable) | |
| ## IMPORTANT CONTEXT | |
| - The game uses `-Dscreenshot-path=screenshot.ppm` which sets `build_options.screenshot_path` | |
| - This enables screenshot mode: loads HomeScreen instead of WorldScreen, counts frames, captures PPM via Vulkan, then exits | |
| - The environment has `ZIGCRAFT_SAFE_RENDER=1` and `ZIGCRAFT_SMOKE_FRAMES=5` | |
| - Lavapipe is the software Vulkan driver (VK_ICD_FILENAMES points to lvp_icd.x86_64.json) | |
| - The headless swapchain creates an offscreen VkImage with `VK_IMAGE_USAGE_COLOR_ATTACHMENT_BIT | VK_IMAGE_USAGE_TRANSFER_SRC_BIT` | |
| ## CRITICAL CONSTRAINTS | |
| - You may ONLY create a single GitHub issue. Do NOT create branches, PRs, or modify files. | |
| - Read the actual log and code — do not guess. Reference specific log lines and source file locations. | |
| - If you cannot determine the root cause, file the issue with what you found and note that further investigation is needed. | |
| - Include the workflow run link in the issue: ${{ github.server_url }}/${{ github.repository }}/actions/runs/${{ github.run_id }} |