ci: Nightly runtime tests across the Debug/Release × BundleMode/LegacyEval matrix#9932
Open
tjzel wants to merge 1 commit into
Open
ci: Nightly runtime tests across the Debug/Release × BundleMode/LegacyEval matrix#9932tjzel wants to merge 1 commit into
tjzel wants to merge 1 commit into
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Note
This PR description was AI-generated.
Stacked on #9931. Part 6 (final) of the runtime-tests spec.
Summary
I added a nightly CI workflow that runs the runtime tests in all four configurations.
runtime-tests-ios.ymlis a reusable workflow (workflow_callwithconfigurationandbundleModeinputs) modeled onexample-ios-build-check.yml:macos-26runner, bundler-cached Ruby,yarn install --immutable, optionalyarn toggle-bundle-mode(which switches the repo default — Bundle Mode — to Legacy Eval Mode), Reanimated package build, cached Pods and DerivedData. It then runsscripts/runtime-tests-server.jstwice: Reanimated first (builds the app via theDebugRuntimeTestsscheme with the requested configuration), then Worklets with--skip-build. In Bundle Mode the Worklets run adds--include "bundle mode core"so the bundle-mode-only suite is exercised. The server boots the simulator itself and falls back to the first available iPhone if the default device name doesn't exist on the runner image; for Debug configurations it also starts Metro (always with--reset-cache), while Release configurations run from the embedded per-library bundles with no Metro at all.runtime-tests-nightly.ymlruns afail-fast: falsematrix of{DebugRuntimeTests, ReleaseRuntimeTests} × {bundleMode on, off}on a cron (53 3 * * *, offset from the other nightlies), and can be triggered manually viaworkflow_dispatch. Each job fails (exit 1) if any test fails, times out at 90 minutes, and the two workflows are also runnable on PRs that touch the runtime tests or the workflows themselves.Test plan
pull_requestfor this PR's paths, so this PR itself exercises theDebugRuntimeTests+ Bundle Mode cell once the stack lands on main.workflow_dispatchonce per configuration to confirm the runner-image simulator fallback before relying on the cron.