Skip to content

ci: Nightly runtime tests across the Debug/Release × BundleMode/LegacyEval matrix#9932

Open
tjzel wants to merge 1 commit into
@tjzel/runtime-tests-fixesfrom
@tjzel/runtime-tests-nightly-ci
Open

ci: Nightly runtime tests across the Debug/Release × BundleMode/LegacyEval matrix#9932
tjzel wants to merge 1 commit into
@tjzel/runtime-tests-fixesfrom
@tjzel/runtime-tests-nightly-ci

Conversation

@tjzel

@tjzel tjzel commented Jul 10, 2026

Copy link
Copy Markdown
Collaborator

Note

This PR description was AI-generated.

Stacked on #9931. Part 6 (final) of the runtime-tests spec.

Summary

I added a nightly CI workflow that runs the runtime tests in all four configurations.

  • runtime-tests-ios.yml is a reusable workflow (workflow_call with configuration and bundleMode inputs) modeled on example-ios-build-check.yml: macos-26 runner, bundler-cached Ruby, yarn install --immutable, optional yarn toggle-bundle-mode (which switches the repo default — Bundle Mode — to Legacy Eval Mode), Reanimated package build, cached Pods and DerivedData. It then runs scripts/runtime-tests-server.js twice: Reanimated first (builds the app via the DebugRuntimeTests scheme with the requested configuration), then Worklets with --skip-build. In Bundle Mode the Worklets run adds --include "bundle mode core" so the bundle-mode-only suite is exercised. The server boots the simulator itself and falls back to the first available iPhone if the default device name doesn't exist on the runner image; for Debug configurations it also starts Metro (always with --reset-cache), while Release configurations run from the embedded per-library bundles with no Metro at all.
  • runtime-tests-nightly.yml runs a fail-fast: false matrix of {DebugRuntimeTests, ReleaseRuntimeTests} × {bundleMode on, off} on a cron (53 3 * * *, offset from the other nightlies), and can be triggered manually via workflow_dispatch. Each job fails (exit 1) if any test fails, times out at 90 minutes, and the two workflows are also runnable on PRs that touch the runtime tests or the workflows themselves.

Test plan

  • The exact command sequence the workflow runs was executed locally for all four matrix cells with the results recorded in fix: Make runtime tests pass in the whole Debug/Release × BundleMode/LegacyEval matrix #9931.
  • The reusable workflow triggers on pull_request for this PR's paths, so this PR itself exercises the DebugRuntimeTests + Bundle Mode cell once the stack lands on main.
  • After merging, trigger workflow_dispatch once per configuration to confirm the runner-image simulator fallback before relying on the cron.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant