Skip to content

Flaky tests under full-suite load + Docker-dependent tests need skip guard #582

@diberry

Description

@diberry

Problem

When running the full test suite (~177 test files, ~4989 tests), approximately 24 tests fail intermittently due to two unrelated categories:

1. Docker-dependent tests (~15 failures)

Tests in \ est/cli/init.test.ts\ and \ est/init-prompt.test.ts\ require Docker Desktop to be running. When Docker is not available, they fail with:
\
failed to connect to the docker API at npipe:////./pipe/dockerDesktopLinuxEngine
\\

Suggested fix: Add a skip guard that checks for Docker availability before running these tests (e.g., \describe.skipIf(!dockerAvailable)).

2. Timing/flaky tests under resource contention (~9 failures)

Tests in \ est/speed-gates.test.ts, \ est/streams.test.ts, \ est/template-sync.test.ts, \ est/repl-ux-fixes.test.ts, and \ est/human-journeys.test.ts\ fail when the full suite runs with 22 parallel workers but pass individually when run in isolation.

Symptoms:

  • Vitest worker timeout errors (\Timeout calling onTaskUpdate)
  • \ENOTEMPTY: directory not empty, rmdir\ during test cleanup
  • Timing assertions fail under load (speed gates)

Suggested fix: Increase timeout margins for speed-gate tests, add retry for cleanup operations, or reduce parallelism for resource-sensitive test files.

Impact

These are pre-existing on dev/main — not introduced by any specific PR. They affect CI reliability and cause false negatives in PR checks.

Environment

  • Windows 11, Node.js 20+
  • Vitest with 22 parallel workers
  • Docker Desktop not always running

squad obo dina

Metadata

Metadata

Assignees

No one assigned

    Labels

    bugSomething isn't workinggo:needs-researchNeeds investigationsquadSquad triage inbox — Lead will assign to a membersquad:fidoAssigned to FIDO (Quality Owner)

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions