Add Appium parallel execution support

[omnarayan]

Summary

• Enable --parallel N for the Appium driver — all N sessions hit the same --appium-url with identical capabilities, server (local or cloud) allocates devices
• Per-worker cloud provider detection and result reporting (each session = separate cloud job)
• Reuses existing ParallelRunner work-queue infrastructure — no changes to the executor layer

Usage

# 3 parallel sessions against same Appium URL
maestro-runner --driver appium --appium-url "http://localhost:4723" \
  --caps caps.json test --parallel 3 flows/

# Cloud — same thing, provider allocates devices
maestro-runner --driver appium --appium-url "https://ondemand.saucelabs.com/wd/hub" \
  --caps caps.json test --parallel 3 flows/

Test plan

• go build ./... passes
• go test ./pkg/cli/... ./pkg/executor/... passes
• E2E: Android real device + emulator with --parallel 2 — flows distributed across both devices
• E2E: Single session (--parallel 1 / no flag) — unchanged behavior
• E2E: Cloud provider (Sauce Labs) with --parallel 2

Notes

• iOS local Appium parallel requires appium:udid per session — Appium XCUITest driver doesn't auto-distribute across booted simulators with identical caps
• Android local Appium parallel works automatically — UiAutomator2 distributes across available devices

[eyaly]

Hi @omnarayan . Thanks for this PR. It’s an important improvement when using maestro-runner with Appium.

I tested this PR on the Sauce Labs platform.
I have a folder flows/tests/ios with two YAML files, and a caps file configured to run on any available iPhone device.

Here is the command I ran:

maestro-runner --driver appium
--appium-url "https://$SAUCE_USERNAME:$SAUCE_ACCESS_KEY@ondemand.eu-central-1.saucelabs.com:443/wd/hub"
--caps config/caps-ios-RDC-sauce.json
--parallel 2 flows/tests/ios

My main expectation is that one YAML file should run on one device.
The --parallel parameter should control how many YAML files run in parallel.

1. When I have 2 YAML files and use --parallel 2, it works as expected - each YAML file runs on a separate iPhone device, and both executions run in parallel.

2. When using --parallel 3, my expectation is the same as before: since I only have 2 YAML files, they should run on 2 devices. However, 3 iOS devices are used, and one of the YAML files runs twice.

3. When using --parallel 1, my expectation is that one YAML file runs on one iPhone device, and once it finishes, the second YAML file runs on another iPhone device. However, both YAML files run on the same device with the same Appium job ID.

I hope this makes sense,
Eyal

[omnarayan]

@eyaly, really appreciate you taking the time to test on Sauce Labs and putting together such detailed feedback — it made tracking this down much easier!

Issue 2 (--parallel 3 with 2 flows) — Fixed! We now cap sessions to min(parallel, flows), so --parallel 3 with 2 flows will only create 2 sessions. A warning is printed to make the behavior explicit:

⚠ --parallel 3 but only 2 flow(s), using 2 session(s)

Issue 3 (--parallel 1) — This is actually working as designed. --parallel 1 means 1 session, with all flows running sequentially on it (same Appium session). If you want a fresh session per flow, add newSession: true to launchApp in your flow YAML.

The fix is pushed to this PR. Would love it if you could re-test when you get a chance — your feedback has been invaluable! 🙏

[eyaly]

Thanks.
For Issue 2 (--parallel 3 with 2 flows), where the number of YAML files is less than the parallel executions:
I tested it again, and it is now working as expected. Thanks.

For Issue 3 (--parallel 1), where the number of YAML files is greater than the parallel executions:
I believe there is no right or wrong here, just two different ways of interpreting the behavior.

From my perspective:
Each YAML file represents an Appium job ID.

So even without defining the parallel parameter, all YAML files should run sequentially on the same device (or different devices) but with different Appium job IDs. This would result in multiple executions on Sauce Labs.

The parallel parameter defines how many executions run at the same time.

For example:
If I have 3 YAML files and use "--parallel 2", then 2 YAML files should run in parallel on 2 devices. Once one finishes, the remaining YAML file should be picked up and executed.

Currently, in this scenario (3 YAML files with --parallel 2):

2 YAML files run on the same device with the same Appium job ID

The third YAML file runs on a second device in parallel

I hope this makes sense.

[omnarayan]

For Issue 3 (--parallel 1), where the number of YAML files is greater than the parallel executions:
I believe there is no right or wrong here, just two different ways of interpreting the behavior.

@eyaly , I respect your view on this, and honestly I think you're right — there's no wrong answer here, just two different ways of looking at it.

The reason I went with this design (and I did add a newSession: true option for those who want a fresh session each time) came down to a couple of things:

1. Appium session creation is slow — it adds up, especially at scale.
2. Creating a new session per test breaks the workflow for teams who want to pick up from where the last test left off. Yes, I know that's not "textbook" QA — every test should ideally be independent and self-contained — and parallel runs make that even harder to guarantee. But honestly, in practice, a lot of teams run it this way intentionally. I'm sure you've seen that too.

That said, I'll admit point 1 isn't exactly airtight, and I can see it being argued the other way. At the end of the day, we probably need to support both behaviors — the real question is just which one becomes the default.

And to be clear: I'm not trying to make a final call here or act like I have the authority to close this out. I don't. This is still very much on the table — just wanted to share where my head was at when I built it this way.

[eyaly]

Hi @omnarayan - I can see your points.

With Appium, both approaches are actually used:

1. Java (TestNG/JUnit) + Appium – tests are typically split across devices, and each test runs separately with a unique Appium session (job ID).

2. WebdriverIO – tests in the same file are usually executed on the same device within a single Appium session (same job ID).

With Android Espresso, you define the number of shards, and the tests are split and executed across that number of Android devices (so it is your approach) :
https://developer.android.com/training/testing/instrumented-tests/androidx-test-libraries/runner#shard-tests

From what I’ve observed with maestro-runner, if I have 10 YAML files and use "--parallel 2", the files are evenly distributed between the two devices (5 YAML files per device). This is great :-)

One improvement I can add latter for Sauce Labs executions - adding the YAML file name to the logs when execution starts. Currently if 5 YAML files are executed with the same Appium Job , it’s difficult to tell when one YAML file ends and another begins.

If both approaches are technically possible to implement on your side, you could keep the current behavior as the default. And, It would be great to have an optional parameter that allows running each YAML file in a new Appium session.

Thanks,
Eyal

[omnarayan]

@eyaly Thanks for the quality-of-life improvement suggestions! Here's what we've implemented — please review.

---

Changes Implemented

1. YAML filename and flow name in log

Added across all relevant log output locations:

Parallel execution log:

[2/6] login_invalid_credentials (login_invalid_credentials.yaml) - ⚡ Started on 11171JEC200939 (session: 9c4bc5e9-0fe7-4a80-8a9e-69c53e901c08)

Per-flow output:

[4/6] login_manual_entry (/Users/omnarayan/work/test/performance_cmp/mda/auth/login_manual_entry.yaml) - Device: 11171JEC200939 (android 13) [session: 63048efb-6a2b-43f7-a2a3-f0ad24483f78]
────────────────────────────────────────────────────────────
  ✓ launchApp (2.9s)
  ✓ tapOn (1.5s)
  ✓ tapOn (915ms)
  ✓ assertVisible (427ms)
  ✓ tapOn (384ms)
  ✓ inputText (849ms)
  ✓ tapOn (415ms)
  ✓ inputText (414ms)
  ✓ tapOn (380ms)
  ✓ assertVisible (756ms)
✓ login_manual_entry 9.0s

JSON report now includes device & session details:

"device": {
  "id": "11171JEC200939",
  "name": "11171JEC200939",
  "platform": "android",
  "osVersion": "13",
  "sessionId": "3a9c2fd1-d35e-4cbd-8f54-61a0203a177b",
  "isSimulator": true
}

2. Per-YAML Appium session (optional parameter)

The option to run each YAML file in a new Appium session already exists in the YAML config. Would you also like it exposed as a CLI argument?

---

On Scheduling: Pull-Based vs. Pre-Assignment

Since you raised how --parallel 2 distributes work, wanted to clarify how maestro-runner actually handles this — it's likely different from what you're used to.

maestro-runner uses a pull-based scheduler, not static pre-assignment:

• Each device picks up one YAML file at a time from a shared queue
• When it finishes, it pulls the next available file
• No test cases are pre-assigned to a specific device or Appium session

This avoids the classic pitfalls of static sharding:

• If one device gets slower or longer test cases, other devices aren't left idle
• If a device disconnects, only the currently running test is affected — the remaining queue is still served by healthy devices

So while it may look like files are evenly distributed (e.g. 5 per device with --parallel 2), that's an emergent outcome of roughly equal test durations — not a scheduling guarantee.

---

On Android Espresso Sharding

You're right that Espresso's --num-shards approach pre-assigns tests to devices by shard index. It can face similar hotspot issues when test durations vary significantly across shards. That said, this is separate from maestro-runner's scope.

For context: at DeviceLab, we've implemented pull-based scheduling for Espresso, Appium, and other frameworks for exactly this reason — it consistently yields better device utilisation regardless of test duration variance.