feat: added windows build on windows to go wrapper#16
Merged
Conversation
* getFreeVncPort * printCurrentWorkingDirectory
* readAndPrintStdoutStderr
* startVncScreenCaptureOnMacosDarwin * openVncViewerOnMacosDarwin
* runFfmpegOnMacosDarwin
10af6cd to
b4a89e7
Compare
* nested virtualization not supported on public gh runners
d19ea88 to
eafcc48
Compare
…nload/upload-build-cache
c666503 to
d17b843
Compare
07b85a6 to
3484f7f
Compare
There was a problem hiding this comment.
Pull request overview
This PR extends the build tooling to support building Windows 11 VMs on Windows hosts (Hyper‑V + VirtualBox) via the Go wrapper, while also introducing an Azure Function “runner broker” + Terraform/Terragrunt scaffolding and standardizing large artifact storage under ./cache/ (plus GH Actions composite actions to download/upload that cache).
Changes:
- Add Windows-on-Windows Packer build path (Hyper‑V retry logic, VirtualBox build) and extend VM config to include host OS / virtualization engine + auto CPU/memory sizing.
- Introduce Azure Function app + scripts to provision ephemeral self-hosted Windows runner VMs, plus Terraform/Terragrunt modules/env scaffolding to deploy it.
- Replace various
vendor/references withcache/paths and add build-cache download/upload composite actions.
Reviewed changes
Copilot reviewed 91 out of 95 changed files in this pull request and generated 12 comments.
Show a summary per file
| File | Description |
|---|---|
| scripts/windows/install_oscdimg.ps1 | Adds helper to install Windows ADK Deployment Tools (oscdimg). |
| scripts/windows/download_win_11.ps1 | Updates Fido version and moves downloads from vendor/ to cache/. |
| scripts/windows/dev-alchemy-self-setup.ps1 | Adds Chocolatey-based VirtualBox installation to Windows bootstrap. |
| scripts/macos/playwright_win11_iso.sh | Removes old shell wrapper for Playwright ISO fetch. |
| scripts/macos/playwright_win11_iso.py | Adds stealth + cookies + optional download support; updates outputs to cache/. |
| scripts/macos/download-virtio-win-iso.sh | Moves virtio ISO download to cache/ and ensures directory exists. |
| scripts/macos/download-utm-guest-tools.sh | Moves UTM guest tools output to cache/utm. |
| scripts/macos/download-arm64-uefi.sh | Switches to dynamic Debian URL resolution and moves artifacts to cache/. |
| scripts/macos/create-win11-autounattend-iso.sh | Updates ISO paths to cache/ and adds disk space check. |
| scripts/macos/README.md | Documents playwright-stealth install and updated cache paths. |
| scripts/gh_actions/cloud-init.ps1 | Adds Windows cloud-init script for GH Actions runner provisioning. |
| scripts/gh-runner-func/test-endpoints.sh | Adds CLI script to test Function App endpoints with Entra auth. |
| scripts/gh-runner-func/runner-setup.ps1 | Adds VM bootstrap script for Hyper‑V networking + GH runner service setup. |
| scripts/gh-runner-func/requirements.txt | Defines Python deps for Function App (azure SDKs + playwright). |
| scripts/gh-runner-func/host.json | Adds Azure Functions host configuration. |
| scripts/gh-runner-func/function_app.py | Implements request_runner / delete_resource_group endpoints and auth guard. |
| scripts/gh-runner-func/README.md | Documents architecture, auth model, deployment, and endpoint usage. |
| scripts/gh-runner-func/.vscode/extensions.json | Recommends Azure Functions VS Code extension. |
| scripts/gh-runner-func/.gitignore | Adds Function App-local ignores (venv, local.settings.json, etc.). |
| scripts/gh-runner-func/.funcignore | Excludes venv and test script from function packaging. |
| pkg/build/windows-build_test.go | Adds Windows-only build tests and packer-init error propagation tests. |
| pkg/build/windows-build.go | Implements Windows host Packer builds for Hyper‑V and VirtualBox. |
| pkg/build/virtual-machine.go | Extends VM config with Host OS + virtualization engine + CPU/memory fields. |
| pkg/build/memory_windows.go | Adds Windows total-memory detection for VM memory auto-sizing. |
| pkg/build/memory_other.go | Adds non-Windows stub for memory detection. |
| pkg/build/macos-silicon-build_test.go | Adds build tags and integrates new CPU/memory fields into macOS build tests. |
| pkg/build/macos-silicon-build.go | Refactors build runner to use generic build path + skips VNC/ffmpeg on Windows. |
| pkg/build/macos-silicon-build-helper_test.go | Updates helper tests to use cache/ paths. |
| pkg/build/helper.go | Adds CPU/memory selection helpers + Hyper‑V temp-dir creation. |
| pkg/build/generic_build.go | Introduces shared build runner logic (ports, timeouts, VNC/ffmpeg integration). |
| pkg/build/fixtures_test.go | Removes old TestMain skip logic for macOS silicon tests. |
| pkg/build/external_process_handler.go | Adds reusable external process runner (with retries) and CLI command runner. |
| pkg/build/dependencies_test.go | Adds tests for python venv bootstrap + download/URL resolution helpers. |
| deployments/terraform/root.hcl | Adds Terragrunt root config + hooks + version constraints. |
| deployments/terraform/modules/azure_state_backend/versions.tf | Adds azurerm provider constraints for state backend module. |
| deployments/terraform/modules/azure_state_backend/variables.tf | Adds inputs for env/location/storage/container. |
| deployments/terraform/modules/azure_state_backend/outputs.tf | Exposes storage account/container outputs. |
| deployments/terraform/modules/azure_state_backend/main.tf | Creates state RG and resolves location. |
| deployments/terraform/modules/azure_state_backend/blob_storage.tf | Provisions storage account/container and role assignment. |
| deployments/terraform/modules/azure_gh_runner/versions.tf | Adds provider constraints for runner broker module. |
| deployments/terraform/modules/azure_gh_runner/variables.tf | Adds module inputs (env, runner_location, allowed_user_object_ids). |
| deployments/terraform/modules/azure_gh_runner/runner.tf | Creates main RG for runner resources. |
| deployments/terraform/modules/azure_gh_runner/outputs.tf | Adds module outputs for tenant/subscription/app and function app name/host. |
| deployments/terraform/modules/azure_gh_runner/manager.tf | Deploys Key Vault, Function App, MI roles, App Insights, auth settings, etc. |
| deployments/terraform/modules/azure_gh_runner/main.tf | Adds location data source. |
| deployments/terraform/modules/azure_gh_runner/github_oidc.tf | Configures Entra app registration + OIDC federation + pre-auth Azure CLI. |
| deployments/terraform/modules/azure_gh_runner/cache.tf | Adds build-cache storage account/container and RBAC for GH Actions SP. |
| deployments/terraform/env/azure_dev/env_local.example.hcl | Adds template for local env config. |
| deployments/terraform/env/azure_dev/env.hcl | Adds env definition for azure_dev. |
| deployments/terraform/env/azure_dev/azure_state_backend/terragrunt.hcl | Adds env wiring for state backend module. |
| deployments/terraform/env/azure_dev/azure_state_backend/env_override.hcl | Adds local override for backend selection. |
| deployments/terraform/env/azure_dev/azure_state_backend/.terraform.lock.hcl | Locks azurerm provider for env. |
| deployments/terraform/env/azure_dev/azure_gh_runner/terragrunt.hcl | Adds env wiring for runner broker module. |
| deployments/terraform/env/azure_dev/azure_gh_runner/.terraform.lock.hcl | Locks azurerm/azuread/random for env. |
| deployments/terraform/env/_env/provider_azure_config.hcl | Generates azurerm provider file using env locals. |
| deployments/terraform/env/_env/azure_state_backend.hcl | Defines module source for backend module. |
| deployments/terraform/env/_env/azure_gh_runner.hcl | Defines module source + dependency on backend. |
| deployments/terraform/backend_local.hcl | Adds local backend config. |
| deployments/terraform/backend_azure.hcl | Adds azurerm backend config for Terragrunt remote_state. |
| deployments/terraform/.tflint.hcl | Adds tflint configuration/plugins. |
| cmd/cmd/build.go | Tweaks build output formatting. |
| build/packer/windows/windows11-on-windows-virtualbox.pkr.hcl | Adds VirtualBox Windows 11 build template on Windows host. |
| build/packer/windows/windows11-on-windows-hyperv.pkr.hcl | Adds Hyper‑V Windows 11 build template on Windows host. |
| build/packer/windows/windows11-on-macos.sh | Adds cpu/memory flags and migrates Windows ISO/cache paths to cache/. |
| build/packer/windows/windows11-on-macos.pkr.hcl | Adds cpu/memory variables and migrates dependency paths to cache/. |
| build/packer/windows/windows.pkr.hcl | Removes old Windows Packer template. |
| build/packer/windows/README.md | Updates Windows packer docs (but still references removed template). |
| build/packer/linux/ubuntu/linux-ubuntu-on-macos.sh | Adds cpu/memory flags and migrates ISO path to cache/. |
| build/packer/linux/ubuntu/linux-ubuntu-on-macos.pkr.hcl | Adds cpu/memory variables and updates qemu args + cache paths. |
| build/packer/linux/ubuntu/linux-ubuntu-hyperv.pkr.hcl | Updates output path to cache/ (currently introduces a parse error). |
| build/packer/linux/mint/linux-mint-hyperv.pkr.hcl | Updates output path to cache/ (currently introduces a parse error). |
| build/gh_actions/windows-azure-gh-runner.pkr.hcl | Adds Azure ARM Packer template for runner image with flavor support. |
| build/gh_actions/windows-azure-gh-runner-packer-build.sh | Adds helper to build both Hyper‑V and VirtualBox runner images. |
| Makefile | Adds build target and adds GH runner function test/deploy targets. |
| CHANGELOG.md | Adds consolidated changelog entry for the feature branch. |
| .vscode/tasks.json | Adds VS Code tasks for Functions local run + pip install. |
| .vscode/settings.json | Adds Azure Functions VS Code settings and schema mapping (currently invalid JSON). |
| .vscode/launch.json | Adds Go test debug config + Python Functions attach config. |
| .vscode/extensions.json | Recommends Azure Functions + Python extensions. |
| .gitignore | Expands ignores for cache, terraform, terragrunt, cookies, secrets, etc. |
| .github/workflows/test-packer-build-win11-on-macos.yml | Removes old macOS packer workflow. |
| .github/workflows/test-build-azure-self-hosted-runner-test.md | Adds manual notes for testing Azure runner provisioning. |
| .github/workflows/gitleaks.yaml | Adds gitleaks workflow. |
| .github/runners/prepare-tart-base.sh | Adds golden-image provisioning for tart-based macOS runners. |
| .github/runners/create-macos-tart-runner.sh | Adds pool manager for ephemeral tart-based macOS runners + cache sharing. |
| .github/runners/README.md | Documents tart runner usage + local build-cache mechanism. |
| .github/actions/upload-build-cache/action.yml | Adds composite action to upload cached artifacts to local cache + Azure blobs. |
| .github/actions/upload-build-cache/scripts/upload-build-cache.sh | Bash implementation of upload build-cache action. |
| .github/actions/upload-build-cache/scripts/upload-build-cache.ps1 | PowerShell implementation of upload build-cache action. |
| .github/actions/download-build-cache/action.yml | Adds composite action to restore cached artifacts from local cache/Azure blobs. |
| .github/actions/download-build-cache/scripts/download-build-cache.sh | Bash implementation of download build-cache action. |
| .github/actions/download-build-cache/scripts/download-build-cache.ps1 | PowerShell implementation of download build-cache action. |
Files not reviewed (2)
- deployments/terraform/env/azure_dev/azure_gh_runner/.terraform.lock.hcl: Language not supported
- deployments/terraform/env/azure_dev/azure_state_backend/.terraform.lock.hcl: Language not supported
Comments suppressed due to low confidence (1)
build/packer/windows/README.md:32
- This README still instructs users to edit/build
windows.pkr.hcl, but that template was removed in this PR and replaced by the Hyper-V / VirtualBox Windows 11 templates. Please update the doc to reference the newwindows11-on-windows-*.pkr.hclfiles and the correct ISO cache path (cache/windows11/iso).
Set the iso_url variable in [windows.pkr.hcl](windows.pkr.hcl) to point to your Windows ISO file.
```powershell
# Example for Windows 11 ISO
$isoPath = "C:\path\to\your\Win11_*.iso"
# Find newest iso file in cache/windows directory
$isoPath = Get-ChildItem -Path ".\cache\windows" -Filter "Win11_*.iso" | Sort-Object LastWriteTime -Descending | Select-Object -First 1 | Select-Object -ExpandProperty FullName
Write-Host "Using ISO: $isoPath"
To build the Windows image, run:
# with default iso_url from windows.pkr.hcl
packer build build/packer/windows/windows.pkr.hcl
# or override iso_url
packer build -var "iso_url=$isoPath" build/packer/windows/windows.pkr.hcl</details>
---
💡 <a href="/csautter/dev-alchemy/new/main?filename=.github/instructions/*.instructions.md" class="Link--inTextBlock" target="_blank" rel="noopener noreferrer">Add Copilot custom instructions</a> for smarter, more guided reviews. <a href="https://docs.github.com/en/copilot/customizing-copilot/adding-repository-custom-instructions-for-github-copilot" class="Link--inTextBlock" target="_blank" rel="noopener noreferrer">Learn how to get started</a>.
…ilable vm configs for current OS
… constructing manually
- reduced cpu count fixed hang up on arm64 qemu boot
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Uh oh!
There was an error while loading. Please reload this page.