diff --git a/.github/workflows/php.yml b/.github/workflows/php.yml index 254c91f..8eee0f5 100644 --- a/.github/workflows/php.yml +++ b/.github/workflows/php.yml @@ -84,6 +84,6 @@ jobs: # Add a test script to composer.json, for instance: "test": "vendor/bin/phpunit" # Docs: https://getcomposer.org/doc/articles/scripts.md - name: Run PHPUnit - run: ./vendor/bin/phpunit --fail-on-warning + run: composer phpunit - name: Run Behat - run: ./vendor/bin/behat --colors --strict --stop-on-failure + run: composer behat diff --git a/.gitignore b/.gitignore index 65201dc..7112056 100644 --- a/.gitignore +++ b/.gitignore @@ -2,4 +2,5 @@ /.phpcs-cache /.phpunit.result.cache /test.json -/composer.lock \ No newline at end of file +/tests/fixtures/*/chunk_test.json +/composer.lock diff --git a/README.md b/README.md index 373baa2..34d0f49 100644 --- a/README.md +++ b/README.md @@ -18,6 +18,7 @@ If you Properly integrate your app with this extension then it can be dramatical * Extension can cancel your tests when you hit CTRL+C. * When you have failed tests in **Parallel scenario** mode then can you rerun this test with Behat option `--rerun`. * For each worker you can set environment variables. +* You can group scenarios or features into a single job to improve test execution time by using the same kernel bootstrapped once. ### Main modes @@ -25,8 +26,7 @@ Behat Parallel Extension can work in two main modes: * **Parallel scenario** witch can be enabled by option `--parallel` or `-l`. * **Parallel feature** to enable this you need use behat option `--parallel-feature`. - - Parallel feature option does not support's `--rerun` option. +- **Parallel chunk size** to group multiple scenarios or features into a single job, use the --parallel-chunk-size option (default is 1). Only available with Behat >= 3.23 ## Requirements @@ -159,7 +159,7 @@ Use `--parallel` or `-l` option for start in parallel scenario mode. ``` $ vendor/bin/behat -l 8 --colors - Starting parallel scenario tests with 8 workers + Starting parallel tests with 8 workers Feature: Parallel Scenario: Test behat tests with failed result 3/3 [============================] 100% 12 secs/12 secs @@ -167,17 +167,36 @@ Use `--parallel` or `-l` option for start in parallel scenario mode. ``` $ vendor/bin/behat --parallel-feature 8 --colors - Starting parallel scenario tests with 8 workers + Starting parallel tests with 8 workers Feature: Parallel - Scenario: Test behat tests with failed result + 3/3 [============================] 100% 12 secs/12 secs ``` -## Test with the oldest supported PHP version + ``` + $ vendor/bin/behat --parallel 8 --colors --parallel-chunk-size 2 + Starting parallel tests with 8 workers + Feature: Parallel, Any other feature + Scenario: Test behat tests with failed result, Any other scenario + 3/3 [============================] 100% 12 secs/12 secs + ``` + +## Test with the oldest supported PHP version and oldest vendors + +With docker, without Dockerfile: ```bash -composer update --prefer-lowest --prefer-stable -docker run --rm -ti -v $(pwd):/app -w /app -u $(id -u):$(id -g) php:7.4-cli bash -c "vendor/bin/phpunit; vendor/bin/behat" +# Create the docker container +docker run --rm -d -v $(pwd):/app -w /app --name behat-parallel-old-php php:7.4-cli bash -c "tail -f /dev/null" +# Install deps +docker exec -ti behat-parallel-old-php bash -c "apt-get update && apt-get install -y libzip-dev && docker-php-ext-install zip" +# Install composer for the current user +docker exec -ti -u $(id -u):$(id -g) -e HOME=/tmp behat-parallel-old-php bash -c "curl -s https://getcomposer.org/installer | php -- --install-dir=/tmp; php /tmp/composer.phar update --prefer-lowest --prefer-dist;" +# Run tests +docker exec -ti -u $(id -u):$(id -g) behat-parallel-old-php php /tmp/composer.phar phpunit +docker exec -ti -u $(id -u):$(id -g) behat-parallel-old-php php /tmp/composer.phar behat +# Stop and remove the docker container +docker kill behat-parallel-old-php ``` ## Tools and Coding standards diff --git a/behat.yml.dist b/behat.yml.dist index 9bb939a..1854dc5 100644 --- a/behat.yml.dist +++ b/behat.yml.dist @@ -14,4 +14,4 @@ default: - DMarynicz\Tests\Behat\Context\SimulateTestContext extensions: DMarynicz\BehatParallelExtension\Extension: ~ - + DMarynicz\Tests\Behat\Extension\TestsSkipperExtension: ~ diff --git a/features/chunk-size.feature b/features/chunk-size.feature new file mode 100644 index 0000000..d138fb8 --- /dev/null +++ b/features/chunk-size.feature @@ -0,0 +1,132 @@ +@skip_without_multiple_paths +Feature: Parallel chunk size parameter + As a programmer, I want to able to group my tests in chunks to reduce overhead. + + Scenario: Test behat tests with --parallel-chunk-size and --parallel-feature option + Given I delete file "tests/fixtures/chunked/chunk_test.json" + When I run "behat --config tests/fixtures/chunked/behat.yml.dist --parallel-feature 20 --parallel-chunk-size 2" + Then it should pass + And I should see progress bar + And the output should contain: + """ + Starting parallel tests with 20 workers and 2 tests per worker + """ + # suite01 has 2 features -> should be 1 chunk containing both + # suite02 has 1 feature -> should be 1 chunk + # Total 2 commands expected + And I should have 2 behat commands in "tests/fixtures/chunked/chunk_test.json" + Then behat commands in "tests/fixtures/chunked/chunk_test.json" should match: + """ + {BEHAT_BIN} --no-interaction --fail-on-undefined-step --config tests/fixtures/chunked/behat.yml.dist tests/fixtures/chunked/suite01/f1.feature tests/fixtures/chunked/suite01/f2.feature + {BEHAT_BIN} --no-interaction --fail-on-undefined-step --config tests/fixtures/chunked/behat.yml.dist tests/fixtures/chunked/suite02/f3.feature + """ + + Scenario: Test behat tests with --parallel-chunk-size and --parallel option + Given I delete file "tests/fixtures/chunked/chunk_test.json" + When I run "behat --config tests/fixtures/chunked/behat.yml.dist --parallel 20 --parallel-chunk-size 2" + Then it should pass + And I should see progress bar + And the output should contain: + """ + Starting parallel tests with 20 workers and 2 tests per worker + """ + # suite01: f1 (1 scenario), f2 (1 scenario) -> 2 scenarios total -> 1 chunk + # suite02: f3 (2 scenarios) -> 2 scenarios -> 1 chunk + # Total 2 commands expected + And I should have 2 behat commands in "tests/fixtures/chunked/chunk_test.json" + Then behat commands in "tests/fixtures/chunked/chunk_test.json" should match: + """ + {BEHAT_BIN} --no-interaction --fail-on-undefined-step --config tests/fixtures/chunked/behat.yml.dist tests/fixtures/chunked/suite01/f1.feature:2 tests/fixtures/chunked/suite01/f2.feature:2 + {BEHAT_BIN} --no-interaction --fail-on-undefined-step --config tests/fixtures/chunked/behat.yml.dist tests/fixtures/chunked/suite02/f3.feature:2 tests/fixtures/chunked/suite02/f3.feature:4 + """ + + Scenario: Test behat tests with --parallel-chunk-size 1 (default) + Given I delete file "tests/fixtures/chunked/chunk_test.json" + When I run "behat --config tests/fixtures/chunked/behat.yml.dist --parallel 20" + Then it should pass + And I should see progress bar + And the output should contain: + """ + Starting parallel tests with 20 workers + """ + # 3 features total, 4 scenarios total + # suite01: f1 (1 scenario), f2 (1 scenario) + # suite02: f3 (2 scenarios) + # Total 4 scenarios -> 4 commands + And I should have 4 behat commands in "tests/fixtures/chunked/chunk_test.json" + Then behat commands in "tests/fixtures/chunked/chunk_test.json" should match: + """ + {BEHAT_BIN} --no-interaction --fail-on-undefined-step --config tests/fixtures/chunked/behat.yml.dist tests/fixtures/chunked/suite01/f1.feature:2 + {BEHAT_BIN} --no-interaction --fail-on-undefined-step --config tests/fixtures/chunked/behat.yml.dist tests/fixtures/chunked/suite01/f2.feature:2 + {BEHAT_BIN} --no-interaction --fail-on-undefined-step --config tests/fixtures/chunked/behat.yml.dist tests/fixtures/chunked/suite02/f3.feature:2 + {BEHAT_BIN} --no-interaction --fail-on-undefined-step --config tests/fixtures/chunked/behat.yml.dist tests/fixtures/chunked/suite02/f3.feature:4 + """ + + Scenario: Test behat tests with --rerun and --parallel-chunk-size 2 + Given I delete file "tests/fixtures/chunked/chunk_test.json" + # Rerun should still execute chunked tasks when no previous failures exist. + When I run "behat --config tests/fixtures/chunked/behat.yml.dist --parallel 2 --parallel-chunk-size 2 --rerun" + Then it should pass + And I should see progress bar + And the output should contain: + """ + Starting parallel tests with 2 workers and 2 tests per worker + """ + And I should have 2 behat commands in "tests/fixtures/chunked/chunk_test.json" + Then behat commands in "tests/fixtures/chunked/chunk_test.json" should match: + """ + {BEHAT_BIN} --no-interaction --fail-on-undefined-step --rerun --config tests/fixtures/chunked/behat.yml.dist tests/fixtures/chunked/suite01/f1.feature:2 tests/fixtures/chunked/suite01/f2.feature:2 + {BEHAT_BIN} --no-interaction --fail-on-undefined-step --rerun --config tests/fixtures/chunked/behat.yml.dist tests/fixtures/chunked/suite02/f3.feature:2 tests/fixtures/chunked/suite02/f3.feature:4 + """ + + Scenario: Test behat tests with --rerun and one failing job with --parallel option + Given I delete file "tests/fixtures/chunked_rerun/chunk_test.json" + # First run executes at least two chunks from suite01. + And I run "behat --config tests/fixtures/chunked_rerun/behat.yml.dist --parallel 2 --parallel-chunk-size 2" + Then it should fail + And I should have 2 behat commands in "tests/fixtures/chunked_rerun/chunk_test.json" + And behat commands in "tests/fixtures/chunked_rerun/chunk_test.json" should match: + """ + {BEHAT_BIN} --no-interaction --fail-on-undefined-step --config tests/fixtures/chunked_rerun/behat.yml.dist tests/fixtures/chunked_rerun/suite01/f1.feature:2 tests/fixtures/chunked_rerun/suite01/f2.feature:2 + {BEHAT_BIN} --no-interaction --fail-on-undefined-step --config tests/fixtures/chunked_rerun/behat.yml.dist tests/fixtures/chunked_rerun/suite01/f3.feature:2 + """ + And I delete file "tests/fixtures/chunked_rerun/chunk_test.json" + # Rerun only the previously failed command from suite01. + When I run "behat --config tests/fixtures/chunked_rerun/behat.yml.dist --parallel 2 --parallel-chunk-size 2 --rerun" + Then it should fail + And I should see progress bar + And the output should contain: + """ + Starting parallel tests with 2 workers and 2 tests per worker + """ + And I should have 1 behat commands in "tests/fixtures/chunked_rerun/chunk_test.json" + And behat commands in "tests/fixtures/chunked_rerun/chunk_test.json" should match: + """ + {BEHAT_BIN} --no-interaction --fail-on-undefined-step --rerun --config tests/fixtures/chunked_rerun/behat.yml.dist tests/fixtures/chunked_rerun/suite01/f1.feature:2 tests/fixtures/chunked_rerun/suite01/f2.feature:2 + """ + + Scenario: Test behat tests with --rerun and one failing job with --parallel-feature option + Given I delete file "tests/fixtures/chunked_rerun/chunk_test.json" + # First run executes at least two chunks from suite01. + And I run "behat --config tests/fixtures/chunked_rerun/behat.yml.dist --parallel-feature 2 --parallel-chunk-size 2" + Then it should fail + And I should have 2 behat commands in "tests/fixtures/chunked_rerun/chunk_test.json" + And behat commands in "tests/fixtures/chunked_rerun/chunk_test.json" should match: + """ + {BEHAT_BIN} --no-interaction --fail-on-undefined-step --config tests/fixtures/chunked_rerun/behat.yml.dist tests/fixtures/chunked_rerun/suite01/f1.feature tests/fixtures/chunked_rerun/suite01/f2.feature + {BEHAT_BIN} --no-interaction --fail-on-undefined-step --config tests/fixtures/chunked_rerun/behat.yml.dist tests/fixtures/chunked_rerun/suite01/f3.feature + """ + And I delete file "tests/fixtures/chunked_rerun/chunk_test.json" + # Rerun only the previously failed command from suite01. + When I run "behat --config tests/fixtures/chunked_rerun/behat.yml.dist --parallel-feature 2 --parallel-chunk-size 2 --rerun" + Then it should fail + And I should see progress bar + And the output should contain: + """ + Starting parallel tests with 2 workers and 2 tests per worker + """ + And I should have 1 behat commands in "tests/fixtures/chunked_rerun/chunk_test.json" + And behat commands in "tests/fixtures/chunked_rerun/chunk_test.json" should match: + """ + {BEHAT_BIN} --no-interaction --fail-on-undefined-step --rerun --config tests/fixtures/chunked_rerun/behat.yml.dist tests/fixtures/chunked_rerun/suite01/f1.feature tests/fixtures/chunked_rerun/suite01/f2.feature + """ diff --git a/features/environment.feature b/features/environment.feature index d781e33..f85df08 100644 --- a/features/environment.feature +++ b/features/environment.feature @@ -7,7 +7,7 @@ Feature: Environment Then it should pass And the output should contain: """ - Starting parallel scenario tests with 8 workers + Starting parallel tests with 8 workers """ And the output should contain: """ diff --git a/features/parallel-feature-environment.feature b/features/parallel-feature-environment.feature index a9fb20a..5edf913 100644 --- a/features/parallel-feature-environment.feature +++ b/features/parallel-feature-environment.feature @@ -7,7 +7,7 @@ Feature: Environment Then it should pass And the output should contain: """ - Starting parallel scenario tests with 8 workers + Starting parallel tests with 8 workers """ And the output should contain: """ diff --git a/features/sigint.feature b/features/sigint.feature index d0fba07..8c482b8 100644 --- a/features/sigint.feature +++ b/features/sigint.feature @@ -7,4 +7,4 @@ Feature: Sigint Given I start "behat --config tests/fixtures/fail/behat.yml.dist --parallel 2" And I wait for 3 seconds And I send a SIGINT signal to behat process - Then it should fail \ No newline at end of file + Then it should fail diff --git a/src/Cli/ParallelController.php b/src/Cli/ParallelController.php index 8d45672..b579144 100644 --- a/src/Cli/ParallelController.php +++ b/src/Cli/ParallelController.php @@ -2,7 +2,6 @@ namespace DMarynicz\BehatParallelExtension\Cli; -use Behat\Gherkin\Node\ScenarioLikeInterface; use Behat\Testwork\Cli\Controller; use Behat\Testwork\Tester\Cli\ExerciseController; use DMarynicz\BehatParallelExtension\Event\AfterTaskTested; @@ -52,6 +51,9 @@ abstract class ParallelController /** @var CanDetermineNumberOfProcessingUnits */ protected $numberOfCores; + /** @var TitleListFormatter */ + protected $titleListFormatter; + public function __construct( Controller $decoratedController, TaskFactory $taskFactory, @@ -86,20 +88,20 @@ public function execute(InputInterface $input, OutputInterface $output) $this->output = $output; $this->input = $input; + // Unable to get the COLUMNS environment variable to know the width of the terminal + // because Behat overrides it to 9999. Let's use the default TitleListFormatter's maxLength value. + $this->titleListFormatter = new TitleListFormatter(); + return $this->parallelExecute(); } public function beforeTaskTested(BeforeTaskTested $beforeTaskTested): void { - $task = $beforeTaskTested->getTask(); - $featureTitle = sprintf('Feature: %s', $task->getFeature()->getTitle()); - $scenarioTitle = ''; - if ($task->getScenario() instanceof ScenarioLikeInterface) { - $scenarioTitle = sprintf('Scenario: %s', $task->getScenario()->getTitle()); - } + $task = $beforeTaskTested->getTask(); + $units = $task->getUnits(); - $this->progressBar->setMessage($featureTitle, 'feature'); - $this->progressBar->setMessage($scenarioTitle, 'scenario'); + $this->progressBar->setMessage($this->titleListFormatter->formatFeatures($units), 'feature'); + $this->progressBar->setMessage($this->titleListFormatter->formatScenarios($units), 'scenario'); } public function afterTaskTested(AfterTaskTested $taskTested): void @@ -168,8 +170,17 @@ private function setupTasksWithProgressBar(): void private function startPoll(): void { - $poolSize = $this->getMaxPoolSize(); - $this->output->writeln(sprintf('Starting parallel scenario tests with %d workers', $poolSize)); + $poolSize = $this->getMaxPoolSize(); + $chunkSize = (int) $this->input->getOption('parallel-chunk-size'); + $message = sprintf( + 'Starting parallel tests with %d workers', + $poolSize + ); + if ($chunkSize > 1) { + $message .= sprintf(' and %d tests per worker', $chunkSize); + } + + $this->output->writeln($message); $this->poll->setMaxWorkers($poolSize); $this->poll->start(); $maxSize = $this->getMaxSizeFromParallelOption(); @@ -204,12 +215,12 @@ private function createTasks() return $this->taskFactory->createTasks($this->input, null); } - $tasks = []; + $tasksPerPath = []; foreach ($paths as $path) { - $tasks = array_merge($tasks, $this->taskFactory->createTasks($this->input, $path)); + $tasksPerPath[] = $this->taskFactory->createTasks($this->input, $path); } - return $tasks; + return array_merge(...$tasksPerPath); } /** diff --git a/src/Cli/ParallelFeatureController.php b/src/Cli/ParallelFeatureController.php index dd32472..bc342da 100644 --- a/src/Cli/ParallelFeatureController.php +++ b/src/Cli/ParallelFeatureController.php @@ -20,6 +20,13 @@ public function configure(SymfonyCommand $command): void 'How many scenario jobs run in parallel feature mode? Available values empty or integer', false ); + $command->addOption( + 'parallel-chunk-size', + null, + InputOption::VALUE_OPTIONAL, + 'How many features run in one chunk? (Behat >= 3.23)', + 1 + ); } /** diff --git a/src/Cli/ParallelScenarioController.php b/src/Cli/ParallelScenarioController.php index f912dc3..3376224 100644 --- a/src/Cli/ParallelScenarioController.php +++ b/src/Cli/ParallelScenarioController.php @@ -22,6 +22,13 @@ public function configure(SymfonyCommand $command): void ) ->addUsage('--parallel 8') ->addUsage('--parallel'); + $command->addOption( + 'parallel-chunk-size', + null, + InputOption::VALUE_OPTIONAL, + 'How many scenarios run in one chunk? (Behat >= 3.23)', + 1 + ); } /** diff --git a/src/Cli/RerunController.php b/src/Cli/RerunController.php index 8fd2150..7ef9440 100644 --- a/src/Cli/RerunController.php +++ b/src/Cli/RerunController.php @@ -3,7 +3,6 @@ namespace DMarynicz\BehatParallelExtension\Cli; use Behat\Behat\Tester\Cli\RerunController as BehatRerunController; -use Behat\Gherkin\Node\ScenarioLikeInterface; use Behat\Testwork\Cli\Controller; use DMarynicz\BehatParallelExtension\Event\AfterTaskTested; use DMarynicz\BehatParallelExtension\Event\EventDispatcherDecorator; @@ -59,20 +58,18 @@ public function collectFailedTask(AfterTaskTested $taskTested): void return; } - $suiteName = $taskTested->getTask()->getSuite()->getName(); - $featureFile = $taskTested->getTask()->getFeature()->getFile(); - $scenarioLine = null; - $scenario = $taskTested->getTask()->getScenario(); - if ($scenario instanceof ScenarioLikeInterface) { - $scenarioLine = $scenario->getLine(); - } + $task = $taskTested->getTask(); + $suiteName = $task->getSuite()->getName(); + $units = $task->getUnits(); - $line = $featureFile; - if ($scenarioLine !== null) { - $line .= ':' . $scenarioLine; - } + foreach ($units as $unit) { + $path = $unit->getPath(); + if (in_array($path, $this->lines[$suiteName] ?? [], true)) { + continue; + } - $this->lines[$suiteName][] = $line; + $this->lines[$suiteName][] = $path; + } } /** diff --git a/src/Cli/TitleListFormatter.php b/src/Cli/TitleListFormatter.php new file mode 100644 index 0000000..a89ed11 --- /dev/null +++ b/src/Cli/TitleListFormatter.php @@ -0,0 +1,110 @@ +maxLength = $maxLength; + } + + /** + * @param TaskUnit[] $units + */ + public function formatFeatures(array $units): string + { + $features = []; + foreach ($units as $unit) { + $feature = $unit->getFeature(); + if (in_array($feature, $features, true)) { + continue; + } + + $features[] = $feature; + } + + return $this->format($features, 'Feature'); + } + + /** + * @param TaskUnit[] $units + */ + public function formatScenarios(array $units): string + { + $scenarios = []; + foreach ($units as $unit) { + if ($unit->getScenario() === null) { + continue; + } + + $scenarios[] = $unit->getScenario(); + } + + return $this->format($scenarios, 'Scenario'); + } + + /** + * Formats the list of items into a string, truncating if necessary. + * + * @param array $items List of features or scenarios + * @param string $prefix Prefix for the first item (e.g., 'Feature' or 'Scenario') + * + * @return string Formatted string + */ + public function format(array $items, string $prefix): string + { + $titles = []; + foreach ($items as $index => $item) { + $titles[] = $index === 0 + ? sprintf('%s: %s', $prefix, $item->getTitle()) + : sprintf('%s', $item->getTitle()); + } + + $message = implode(', ', $titles); + if (mb_strlen(strip_tags($message)) <= $this->maxLength) { + return $message; + } + + $limitedTitles = []; + $currentLength = 0; + foreach ($titles as $index => $title) { + $titleLength = mb_strlen(strip_tags($title)); + if ($index > 0) { + $titleLength += 2; // for ", " + } + + $suffix = sprintf(' (and %d more)', count($titles) - $index); + $suffixLength = mb_strlen($suffix); + + if ($currentLength + $titleLength + $suffixLength > $this->maxLength) { + break; + } + + $limitedTitles[] = $title; + $currentLength += $titleLength; + } + + if (empty($limitedTitles)) { + $firstTitle = reset($titles); + $suffix = sprintf(' (and %d more)', count($titles) - 1); + + return $firstTitle . $suffix; + } + + return implode(', ', $limitedTitles) . sprintf(' (and %d more)', count($titles) - count($limitedTitles)); + } +} diff --git a/src/Task/ArgumentsBuilder.php b/src/Task/ArgumentsBuilder.php index b2baf0c..a42f70a 100644 --- a/src/Task/ArgumentsBuilder.php +++ b/src/Task/ArgumentsBuilder.php @@ -7,9 +7,9 @@ interface ArgumentsBuilder { /** - * @param string $path + * @param string[] $paths * * @return string[] */ - public function buildArguments(InputInterface $input, $path); + public function buildArguments(InputInterface $input, array $paths): array; } diff --git a/src/Task/ChunkBuilder.php b/src/Task/ChunkBuilder.php new file mode 100644 index 0000000..5376683 --- /dev/null +++ b/src/Task/ChunkBuilder.php @@ -0,0 +1,39 @@ + $items Items to group + * @param int $chunkSize Maximum number of items per chunk + * @param callable $taskCreator Function to create a task from a chunk + * + * @return TaskEntity[] List of created tasks + */ + public static function buildChunks($items, $chunkSize, $taskCreator): array + { + $tasks = []; + $chunk = []; + foreach ($items as $item) { + $chunk[] = $item; + if (count($chunk) < $chunkSize) { + continue; + } + + $tasks[] = $taskCreator($chunk); + $chunk = []; + } + + if ($chunk) { + $tasks[] = $taskCreator($chunk); + } + + return $tasks; + } +} diff --git a/src/Task/FeatureTaskFactory.php b/src/Task/FeatureTaskFactory.php index 6e6b4e1..68c6432 100644 --- a/src/Task/FeatureTaskFactory.php +++ b/src/Task/FeatureTaskFactory.php @@ -2,6 +2,8 @@ namespace DMarynicz\BehatParallelExtension\Task; +use Behat\Gherkin\Node\FeatureNode; +use Behat\Testwork\Suite\Suite; use ReflectionException; use Symfony\Component\Console\Input\InputInterface; @@ -26,22 +28,41 @@ public function __construct(TestworkSpecificationsFinder $finder, ArgumentsBuild * * @throws ReflectionException */ - public function createTasks(InputInterface $input, $path = null) + public function createTasks(InputInterface $input, $path = null): array { - $suites = $this->finder->findGroupedSpecifications($path); - $tasks = []; + $suites = $this->finder->findGroupedSpecifications($path); + $tasks = []; + $chunkSize = max((int) $input->getOption('parallel-chunk-size'), 1); foreach ($suites as $suite) { - foreach ($suite as $feature) { - $testPath = $feature->getFile(); - $tasks[] = new Task( - $suite->getSuite(), - $feature, - $testPath, - $this->argumentsBuilder->buildArguments($input, $testPath) - ); - } + $tasks = array_merge($tasks, ChunkBuilder::buildChunks( + $suite, + $chunkSize, + function (array $chunk) use ($input, $suite) { + return $this->createTaskFromChunk($input, $suite->getSuite(), $chunk); + } + )); } return $tasks; } + + /** + * @param FeatureNode[] $chunk + */ + private function createTaskFromChunk(InputInterface $input, Suite $suite, array $chunk): Task + { + $units = array_map(static function (FeatureNode $feature) { + return new TaskUnit($feature); + }, $chunk); + + $paths = array_map(static function (TaskUnit $unit) { + return $unit->getPath(); + }, $units); + + return new Task( + $suite, + $units, + $this->argumentsBuilder->buildArguments($input, $paths) + ); + } } diff --git a/src/Task/ScenarioTaskFactory.php b/src/Task/ScenarioTaskFactory.php index dc25b77..bfddee8 100644 --- a/src/Task/ScenarioTaskFactory.php +++ b/src/Task/ScenarioTaskFactory.php @@ -2,6 +2,9 @@ namespace DMarynicz\BehatParallelExtension\Task; +use Behat\Gherkin\Node\FeatureNode; +use Behat\Gherkin\Node\ScenarioLikeInterface; +use Behat\Testwork\Suite\Suite; use ReflectionException; use Symfony\Component\Console\Input\InputInterface; @@ -26,25 +29,52 @@ public function __construct(TestworkSpecificationsFinder $finder, ArgumentsBuild * * @throws ReflectionException */ - public function createTasks(InputInterface $input, $path = null) + public function createTasks(InputInterface $input, $path = null): array { $specifications = $this->finder->findGroupedSpecifications($path); $tasks = []; + $chunkSize = max((int) $input->getOption('parallel-chunk-size'), 1); + foreach ($specifications as $spec) { + $scenarios = []; foreach ($spec as $feature) { foreach ($feature->getScenarios() as $scenario) { - $testPath = sprintf('%s:%s', $feature->getFile(), $scenario->getLine()); - $tasks[] = new Task( - $spec->getSuite(), - $feature, - $testPath, - $this->argumentsBuilder->buildArguments($input, $testPath), - $scenario - ); + $scenarios[] = [ + 'feature' => $feature, + 'scenario' => $scenario, + ]; } } + + $tasks = array_merge($tasks, ChunkBuilder::buildChunks( + $scenarios, + $chunkSize, + function (array $chunk) use ($input, $spec) { + return $this->createTaskFromChunk($input, $spec->getSuite(), $chunk); + } + )); } return $tasks; } + + /** + * @param array $chunk + */ + private function createTaskFromChunk(InputInterface $input, Suite $suite, array $chunk): Task + { + $units = array_map(static function (array $item) { + return new TaskUnit($item['feature'], $item['scenario']); + }, $chunk); + + $paths = array_map(static function (TaskUnit $unit) { + return $unit->getPath(); + }, $units); + + return new Task( + $suite, + $units, + $this->argumentsBuilder->buildArguments($input, $paths) + ); + } } diff --git a/src/Task/Task.php b/src/Task/Task.php index b765cc2..e623508 100644 --- a/src/Task/Task.php +++ b/src/Task/Task.php @@ -2,8 +2,6 @@ namespace DMarynicz\BehatParallelExtension\Task; -use Behat\Gherkin\Node\FeatureNode; -use Behat\Gherkin\Node\ScenarioLikeInterface as Scenario; use Behat\Testwork\Suite\Suite; final class Task implements TaskEntity @@ -11,61 +9,50 @@ final class Task implements TaskEntity /** @var Suite */ private $suite; - /** @var FeatureNode */ - private $feature; - - /** @var string */ - private $path; - - /** @var Scenario|null */ - private $scenario; + /** @var TaskUnit[] */ + private $units; /** @var string[] */ private $command; /** - * @param string $path - * @param string[] $command + * @param TaskUnit[] $units + * @param string[] $command */ - public function __construct(Suite $suite, FeatureNode $feature, $path, $command = [], ?Scenario $scenario = null) - { - $this->suite = $suite; - $this->feature = $feature; - $this->path = $path; - $this->command = $command; - $this->scenario = $scenario; + public function __construct( + Suite $suite, + array $units, + array $command = [] + ) { + $this->suite = $suite; + $this->units = $units; + $this->command = $command; } /** - * @return Suite + * @return TaskUnit[] */ - public function getSuite() + public function getUnits(): array { - return $this->suite; + return $this->units; } /** - * @return FeatureNode - */ - public function getFeature() - { - return $this->feature; - } - - /** - * @return Scenario|null + * @return Suite */ - public function getScenario() + public function getSuite() { - return $this->scenario; + return $this->suite; } /** - * @return string + * @return string[] */ - public function getPath() + public function getPaths(): array { - return $this->path; + return array_map(static function (TaskUnit $unit) { + return $unit->getPath(); + }, $this->units); } /** diff --git a/src/Task/TaskArgumentsBuilder.php b/src/Task/TaskArgumentsBuilder.php index 9f2f496..2b3b317 100644 --- a/src/Task/TaskArgumentsBuilder.php +++ b/src/Task/TaskArgumentsBuilder.php @@ -18,22 +18,19 @@ public function __construct(PhpExecutableFinder $phpFinder) } /** - * @param string $path + * @param string[] $paths * * @return string[] * * @throws ReflectionException */ - public function buildArguments(InputInterface $input, $path): array + public function buildArguments(InputInterface $input, array $paths): array { $arguments = $this->buildFirstArguments(); - $arguments = array_merge($arguments, $this->buildOptionArguments($input)); $arguments = array_merge($arguments, $this->buildRemainingArguments($input)); - $arguments[] = $path; - - return $arguments; + return array_merge($arguments, $paths); } /** @@ -61,7 +58,7 @@ private function buildOptionArguments(InputInterface $input): array $argsFromInputValue = []; foreach ($input->getOptions() as $name => $value) { - if (in_array($name, ['parallel', 'parallel-feature'])) { + if (in_array($name, ['parallel', 'parallel-feature', 'parallel-chunk-size'])) { continue; } diff --git a/src/Task/TaskEntity.php b/src/Task/TaskEntity.php index 98ef329..147e657 100644 --- a/src/Task/TaskEntity.php +++ b/src/Task/TaskEntity.php @@ -2,8 +2,6 @@ namespace DMarynicz\BehatParallelExtension\Task; -use Behat\Gherkin\Node\FeatureNode; -use Behat\Gherkin\Node\ScenarioLikeInterface as Scenario; use Behat\Testwork\Suite\Suite; interface TaskEntity @@ -14,22 +12,17 @@ interface TaskEntity public function getSuite(); /** - * @return FeatureNode - */ - public function getFeature(); - - /** - * @return Scenario|null + * @return string[] */ - public function getScenario(); + public function getPaths(); /** - * @return string + * @return string[] */ - public function getPath(); + public function getCommand(); /** - * @return string[] + * @return TaskUnit[] */ - public function getCommand(); + public function getUnits(): array; } diff --git a/src/Task/TaskUnit.php b/src/Task/TaskUnit.php new file mode 100644 index 0000000..5078fdd --- /dev/null +++ b/src/Task/TaskUnit.php @@ -0,0 +1,51 @@ +feature = $feature; + $this->scenario = $scenario; + } + + /** + * @return FeatureNode + */ + public function getFeature() + { + return $this->feature; + } + + /** + * @return Scenario|null + */ + public function getScenario() + { + return $this->scenario; + } + + /** + * @return string + */ + public function getPath() + { + $path = (string) $this->feature->getFile(); + + if ($this->scenario !== null) { + $path .= ':' . $this->scenario->getLine(); + } + + return $path; + } +} diff --git a/tests/Behat/Context/ChunkTestContext.php b/tests/Behat/Context/ChunkTestContext.php new file mode 100644 index 0000000..8d657a8 --- /dev/null +++ b/tests/Behat/Context/ChunkTestContext.php @@ -0,0 +1,94 @@ +filesPath = $filesPath; + } + + /** + * @param string $filename + * + * @Given I log behat command to :filename + */ + public function iLogBehatCommandTo($filename): void + { + $path = $this->getRealPath($filename); + if (! file_exists($path)) { + file_put_contents($path, '[]'); + } + + $handle = new ReadWriteDataToFileWithLocking($path); + + $data = $handle->read(); + + $array = json_decode($data, true); + if (! is_array($array)) { + $array = []; + } + + $argv = $_SERVER['argv']; + $root = realpath(__DIR__ . '/../../../') . DIRECTORY_SEPARATOR; + foreach ($argv as &$arg) { + if (! is_string($arg) || strpos($arg, $root) !== 0) { + continue; + } + + $arg = substr($arg, strlen($root)); + } + + if ( + isset($argv[0]) + && is_string($argv[0]) + && strpos($argv[0], 'php') !== false + && count($argv) > 1 + && isset($argv[1]) + && is_string($argv[1]) + && strpos($argv[1], 'behat') !== false + ) { + array_shift($argv); + } + + $command = implode(' ', $argv); + if (! in_array($command, $array)) { + $array[] = $command; + } + + $data = json_encode($array); + + if (! is_string($data)) { + throw new Logic('Expected string'); + } + + $handle->truncateAndWrite($data); + } + + /** + * @param string $filename + */ + private function getRealPath($filename): string + { + return $this->getRealFilesPath() . DIRECTORY_SEPARATOR . $filename; + } + + private function getRealFilesPath(): string + { + $path = $this->filesPath ?: ''; + $path = (string) realpath($path); + + return rtrim($path, DIRECTORY_SEPARATOR); + } +} diff --git a/tests/Behat/Context/ParallelBehatContext.php b/tests/Behat/Context/ParallelBehatContext.php index a567337..d215134 100644 --- a/tests/Behat/Context/ParallelBehatContext.php +++ b/tests/Behat/Context/ParallelBehatContext.php @@ -165,6 +165,82 @@ public function thenPrintLastOutput(): void echo $this->getOutput(); } + /** + * @param string $filename + * + * @Given I delete file :filename + */ + public function iDeleteFile($filename): void + { + $path = realpath(__DIR__ . '/../../../') . DIRECTORY_SEPARATOR . $filename; + if (! file_exists($path)) { + return; + } + + unlink($path); + } + + /** + * @param int $count + * @param string $filename + * + * @Then I should have :count behat commands in :filename + */ + public function iShouldHaveBehatCommandsIn($count, $filename): void + { + $array = $this->fetchJsonArrayFile($filename); + Assert::assertCount((int) $count, $array); + } + + /** + * @param string $filename + * + * @Then behat commands in :filename should match: + */ + public function behatCommandsInShouldMatch($filename, PyStringNode $expectedCommands): void + { + $commands = $this->fetchJsonArrayFile($filename); + $expected = array_filter(array_map('trim', $expectedCommands->getStrings())); + + // Replace placeholders in expected commands + $phpBin = $this->phpBin; + $behatBin = defined('BEHAT_BIN_PATH') ? BEHAT_BIN_PATH : 'behat'; + // If behatBin is absolute, make it relative to root + $root = realpath(__DIR__ . '/../../../') . DIRECTORY_SEPARATOR; + if (strpos($behatBin, $root) === 0) { + $behatBin = substr($behatBin, strlen($root)); + } + + if (strpos($phpBin, $root) === 0) { + $phpBin = substr($phpBin, strlen($root)); + } + + foreach ($expected as &$cmd) { + $cmd = str_replace(['{PHP_BIN}', '{BEHAT_BIN}'], [$phpBin, $behatBin], $cmd); + } + + sort($commands); + sort($expected); + + Assert::assertEquals($expected, $commands); + } + + /** + * @param string $filename + * + * @return array + */ + private function fetchJsonArrayFile($filename): array + { + $path = realpath(__DIR__ . '/../../../') . DIRECTORY_SEPARATOR . $filename; + Assert::assertFileExists($path); + $data = file_get_contents($path); + $array = json_decode((string) $data, true); + Assert::assertIsArray($array); + + return $array; + } + /** * @param string $cmd */ diff --git a/tests/Behat/Context/SimulateTestContext.php b/tests/Behat/Context/SimulateTestContext.php index b049b81..737ecd0 100644 --- a/tests/Behat/Context/SimulateTestContext.php +++ b/tests/Behat/Context/SimulateTestContext.php @@ -7,6 +7,12 @@ class SimulateTestContext implements Context { + /* + * Any value you want to multiply the wait time by. + * Tips: when running tests many times, you can temporarily set this value to 0.0001 to speed up the tests. + */ + private const WAIT_TIME_MULTIPLIER = 1; + /** * @Given /^(?:|I )am on pretending to be on (?:|the )homepage$/ * @When /^(?:|I )am pretending to go to (?:|the )homepage$/ @@ -30,7 +36,7 @@ public function iAmPretendingOnPage(): void */ public function iWaitForSeconds($seconds): void { - sleep($seconds); + usleep((int) ($seconds * 1000 * 1000 * self::WAIT_TIME_MULTIPLIER)); } /** diff --git a/tests/Behat/Extension/TestsSkipperExtension.php b/tests/Behat/Extension/TestsSkipperExtension.php new file mode 100644 index 0000000..6bc058e --- /dev/null +++ b/tests/Behat/Extension/TestsSkipperExtension.php @@ -0,0 +1,128 @@ + '3.23.0', 'tag' => 'skip_without_multiple_paths'], + ]; + + public function getConfigKey(): string + { + return 'skip_without_multiple_paths'; + } + + public function initialize(ExtensionManager $extensionManager): void + { + } + + public function configure(ArrayNodeDefinition $builder): void + { + } + + /** + * {@inheritdoc} + */ + public function load(ContainerBuilder $container, array $config): void + { + } + + public function process(ContainerBuilder $container): void + { + if (! InstalledVersions::isInstalled('behat/behat')) { + return; + } + + $installedVersion = InstalledVersions::getVersion('behat/behat'); + if ($installedVersion === null) { + return; + } + + if (! $container->hasParameter('suite.configurations')) { + return; + } + + $tagExpressions = $this->collectTagExpressions($installedVersion); + if (! $tagExpressions) { + return; + } + + $suites = (array) $container->getParameter('suite.configurations'); + + foreach ($suites as $name => $suite) { + $settings = $suite['settings'] ?? []; + $filters = $settings['filters'] ?? []; + $existing = $filters['tags'] ?? null; + + $filters['tags'] = $this->mergeTagExpressions($existing, $tagExpressions); + + $settings['filters'] = $filters; + $suite['settings'] = $settings; + $suites[$name] = $suite; + } + + $container->setParameter('suite.configurations', $suites); + } + + /** + * @return string[] + */ + private function collectTagExpressions(string $installedVersion): array + { + $expressions = []; + + foreach (self::VERSION_TAG_RULES as $rule) { + $minVersion = $rule['min_version']; + if (! version_compare($installedVersion, $minVersion, '<')) { + continue; + } + + $tag = ltrim($rule['tag'], '@'); + $expressions[] = '~@' . $tag; + } + + return $expressions; + } + + /** + * @param string[] $expressions + */ + private function mergeTagExpressions(?string $existing, array $expressions): string + { + $unique = []; + + if ($existing) { + $unique[] = $existing; + } + + foreach ($expressions as $expression) { + $alreadyPresent = false; + foreach ($unique as $current) { + if (strpos($current, $expression) !== false) { + $alreadyPresent = true; + break; + } + } + + if ($alreadyPresent) { + continue; + } + + $unique[] = $expression; + } + + return implode('&&', $unique); + } +} diff --git a/tests/Cli/ParallelControllerTest.php b/tests/Cli/ParallelControllerTest.php index ee5ba31..b13a1d7 100644 --- a/tests/Cli/ParallelControllerTest.php +++ b/tests/Cli/ParallelControllerTest.php @@ -6,15 +6,18 @@ use Behat\Gherkin\Node\ScenarioLikeInterface; use Behat\Testwork\Cli\Controller; use DMarynicz\BehatParallelExtension\Cli\ParallelController; +use DMarynicz\BehatParallelExtension\Cli\TitleListFormatter; use DMarynicz\BehatParallelExtension\Event\AfterTaskTested; use DMarynicz\BehatParallelExtension\Event\BeforeTaskTested; use DMarynicz\BehatParallelExtension\Event\EventDispatcherDecorator; use DMarynicz\BehatParallelExtension\Task\Queue; use DMarynicz\BehatParallelExtension\Task\TaskEntity; use DMarynicz\BehatParallelExtension\Task\TaskFactory; +use DMarynicz\BehatParallelExtension\Task\TaskUnit; use DMarynicz\BehatParallelExtension\Util\CanDetermineNumberOfProcessingUnits; use DMarynicz\BehatParallelExtension\Worker\Poll; use PHPUnit\Framework\MockObject\MockObject; +use ReflectionClass; use ReflectionException; use ReflectionProperty; use Symfony\Component\Console\Formatter\OutputFormatterInterface; @@ -66,10 +69,10 @@ public function testConfigure(): void } /** - * @param string $featureTitle - * @param string|null $scenarioTitle - * @param string $expectedFeatureTitle - * @param string|null $expectedScenarioTitle + * @param string|string[] $featureTitle + * @param string|string[]|null $scenarioTitle + * @param string $expectedFeatureTitle + * @param string|null $expectedScenarioTitle * * @dataProvider beforeTaskTestedProvider */ @@ -83,19 +86,36 @@ public function testBeforeTaskTested( $task = $this->createMock(TaskEntity::class); $beforeTested->method('getTask')->willReturn($task); - $feature = $this->createMock(FeatureNode::class); - $feature->method('getTitle')->willReturn($featureTitle); - - $task->method('getFeature')->willReturn($feature); + $featureTitles = is_array($featureTitle) ? $featureTitle : [$featureTitle]; + $units = []; + foreach ($featureTitles as $title) { + $feature = $this->createMock(FeatureNode::class); + $feature->method('getTitle')->willReturn($title); + $units[] = new TaskUnit($feature); + } if ($scenarioTitle) { - $scenario = $this->createMock(ScenarioLikeInterface::class); - $scenario->method('getTitle')->willReturn($scenarioTitle); - $task->method('getScenario')->willReturn($scenario); + $scenarioTitles = is_array($scenarioTitle) ? $scenarioTitle : [$scenarioTitle]; + foreach ($scenarioTitles as $index => $title) { + $scenario = $this->createMock(ScenarioLikeInterface::class); + $scenario->method('getTitle')->willReturn($title); + if (isset($units[$index])) { + // Update existing unit to add scenario + $ref = new ReflectionClass($units[$index]); + $property = $ref->getProperty('scenario'); + $property->setAccessible(true); + $property->setValue($units[$index], $scenario); + } else { + $units[] = new TaskUnit($this->createMock(FeatureNode::class), $scenario); + } + } } + $task->method('getUnits')->willReturn($units); + $progressBar = new ProgressBar($this->output); $this->setNonAccessibleValue($this->controller, 'progressBar', $progressBar); + $this->setNonAccessibleValue($this->controller, 'titleListFormatter', new TitleListFormatter()); $this->controller->beforeTaskTested($beforeTested); @@ -110,7 +130,7 @@ public function testParallelTestsAborted(): void } /** - * @return array> + * @return array */ public function beforeTaskTestedProvider(): array { @@ -127,6 +147,21 @@ public function beforeTaskTestedProvider(): array 'Feature: feature title', null, ], + [ + array_fill(0, 10, 'Long Feature Title'), + null, + 'Feature: Long Feature Title, Long Feature Title, ' + . 'Long Feature Title, Long Feature Title, ' + . 'Long Feature Title (and 5 more)', + null, + ], + [ + ['Feature 1', 'Feature 2', 'Feature 3', 'Feature 4', 'Feature 5'], + null, + 'Feature: Feature 1, Feature 2, Feature 3, ' + . 'Feature 4, Feature 5', + null, + ], ]; } diff --git a/tests/Cli/ParallelFeatureControllerTest.php b/tests/Cli/ParallelFeatureControllerTest.php index ffb494d..9a40c65 100644 --- a/tests/Cli/ParallelFeatureControllerTest.php +++ b/tests/Cli/ParallelFeatureControllerTest.php @@ -24,7 +24,17 @@ public function testExecute(): void { $taskEntity = $this->createMock(TaskEntity::class); $this->taskFactory->method('createTasks')->willReturn([$taskEntity]); - $this->input->method('getOption')->with('parallel-feature')->willReturn(32); + $this->input->method('getOption')->willReturnCallback(static function ($name) { + if ($name === 'parallel-feature') { + return 32; + } + + if ($name === 'parallel-chunk-size') { + return 1; + } + + return null; + }); $result = $this->controller->execute($this->input, $this->output); $this->assertEquals(0, $result); } diff --git a/tests/Cli/ParallelScenarioControllerTest.php b/tests/Cli/ParallelScenarioControllerTest.php index eeabb7a..538ff5a 100644 --- a/tests/Cli/ParallelScenarioControllerTest.php +++ b/tests/Cli/ParallelScenarioControllerTest.php @@ -24,7 +24,17 @@ public function testExecute(): void { $taskEntity = $this->createMock(TaskEntity::class); $this->taskFactory->method('createTasks')->willReturn([$taskEntity]); - $this->input->method('getOption')->with('parallel')->willReturn(32); + $this->input->method('getOption')->willReturnCallback(static function ($name) { + if ($name === 'parallel') { + return 32; + } + + if ($name === 'parallel-chunk-size') { + return 1; + } + + return null; + }); $result = $this->controller->execute($this->input, $this->output); $this->assertEquals(0, $result); } diff --git a/tests/Cli/RerunControllerTest.php b/tests/Cli/RerunControllerTest.php index 822c0f6..d911a88 100644 --- a/tests/Cli/RerunControllerTest.php +++ b/tests/Cli/RerunControllerTest.php @@ -3,12 +3,13 @@ namespace DMarynicz\Tests\Cli; use Behat\Gherkin\Node\FeatureNode; -use Behat\Gherkin\Node\ScenarioInterface; +use Behat\Gherkin\Node\ScenarioLikeInterface; use Behat\Testwork\Suite\Suite; use DMarynicz\BehatParallelExtension\Cli\RerunController; use DMarynicz\BehatParallelExtension\Event\AfterTaskTested; use DMarynicz\BehatParallelExtension\Event\EventDispatcherDecorator; use DMarynicz\BehatParallelExtension\Task\TaskEntity; +use DMarynicz\BehatParallelExtension\Task\TaskUnit; use PHPUnit\Framework\MockObject\MockObject; use ReflectionClass; use ReflectionException; @@ -47,6 +48,12 @@ public function testExecute(): void $this->controller->execute($this->input, $this->output); } + public function testExecuteDoesNotOverrideParallelChunkSizeOnRerun(): void + { + $this->input->expects($this->never())->method('setOption'); + $this->controller->execute($this->input, $this->output); + } + public function testCollectFailedTask(): void { $afterTested = $this->createMock(AfterTaskTested::class); @@ -83,14 +90,17 @@ public function testCollectFailedTask2($suiteName, $featureFile, $expectedLines $feature->method('getFile')->willReturn($featureFile); $task->method('getSuite')->willReturn($suite); - $task->method('getFeature')->willReturn($feature); - $scenario = $this->createMock(ScenarioInterface::class); if ($scenarioLine) { - $task->method('getScenario')->willReturn($scenario); + $scenario = $this->createMock(ScenarioLikeInterface::class); $scenario->method('getLine')->willReturn($scenarioLine); + $unit = new TaskUnit($feature, $scenario); + } else { + $unit = new TaskUnit($feature); } + $task->method('getUnits')->willReturn([$unit]); + $this->controller->collectFailedTask($afterTested); $this->assertCount(1, $this->getLinesPropperty()); diff --git a/tests/Cli/TitleListFormatterTest.php b/tests/Cli/TitleListFormatterTest.php new file mode 100644 index 0000000..687f375 --- /dev/null +++ b/tests/Cli/TitleListFormatterTest.php @@ -0,0 +1,61 @@ +createMock(FeatureNode::class); + $item->method('getTitle')->willReturn($title); + $items[] = $item; + } + + $formatter = new TitleListFormatter(120); + $result = $formatter->format($items, 'Scenario'); + + $plainResult = strip_tags($result); + $this->assertLessThanOrEqual(120, mb_strlen($plainResult), 'The formatted string should not exceed 120 chars'); + } + + public function testFormatTruncation(): void + { + $items = []; + for ($i = 1; $i <= 5; $i++) { + $item = $this->createMock(FeatureNode::class); + $item->method('getTitle')->willReturn('Title ' . $i); + $items[] = $item; + } + + // "Scenario: Title 1" (17 chars) + // ", Title 2" (9 chars) + // ", Title 3" (9 chars) + // " (and 2 more)" (13 chars) + + $formatter = new TitleListFormatter(30); + $result = $formatter->format($items, 'Scenario'); + + $plainResult = strip_tags($result); + $this->assertLessThanOrEqual(30, mb_strlen($plainResult)); + $this->assertStringContainsString('(and', $plainResult); + } +} diff --git a/tests/Task/ChunkedFeatureTaskFactoryTest.php b/tests/Task/ChunkedFeatureTaskFactoryTest.php new file mode 100644 index 0000000..c71230a --- /dev/null +++ b/tests/Task/ChunkedFeatureTaskFactoryTest.php @@ -0,0 +1,54 @@ + 'suite01', + 'features' => [ + ['name' => 'f1', 'file' => 'f1.feature', 'scenarios' => [['name' => 's1', 'line' => 1]]], + ['name' => 'f2', 'file' => 'f2.feature', 'scenarios' => [['name' => 's2', 'line' => 2]]], + ['name' => 'f3', 'file' => 'f3.feature', 'scenarios' => [['name' => 's3', 'line' => 3]]], + ], + ], + ]; + + $finder = $this->createSpecificationsFinderMock($specsToMock); + $argumentsBuilder = $this->createArgumentsBuilderMock(); + $argumentsBuilder->method('buildArguments')->willReturnCallback(static function ($input, $paths) { + return array_merge(['behat'], $paths); + }); + + $input = $this->createInputInterfaceMock(); + $input->method('getOption')->willReturnMap([ + ['parallel-chunk-size', 2], + ['rerun', false], + ]); + + $factory = new FeatureTaskFactory($finder, $argumentsBuilder); + $tasks = $factory->createTasks($input); + + $this->assertCount(2, $tasks); + + $this->assertInstanceOf(Task::class, $tasks[0]); + $this->assertEquals(['f1.feature', 'f2.feature'], $tasks[0]->getPaths()); + $this->assertEquals(['behat', 'f1.feature', 'f2.feature'], $tasks[0]->getCommand()); + $this->assertCount(2, $tasks[0]->getUnits()); + + $this->assertInstanceOf(Task::class, $tasks[1]); + $this->assertEquals(['f3.feature'], $tasks[1]->getPaths()); + $this->assertEquals(['behat', 'f3.feature'], $tasks[1]->getCommand()); + $this->assertCount(1, $tasks[1]->getUnits()); + } +} diff --git a/tests/Task/ChunkedScenarioTaskFactoryTest.php b/tests/Task/ChunkedScenarioTaskFactoryTest.php new file mode 100644 index 0000000..5c8d741 --- /dev/null +++ b/tests/Task/ChunkedScenarioTaskFactoryTest.php @@ -0,0 +1,60 @@ + 'suite01', + 'features' => [ + [ + 'name' => 'f1', + 'file' => 'f1.feature', + 'scenarios' => [ + ['name' => 's1', 'line' => 1], + ['name' => 's2', 'line' => 2], + ['name' => 's3', 'line' => 3], + ], + ], + ], + ], + ]; + + $finder = $this->createSpecificationsFinderMock($specsToMock); + $argumentsBuilder = $this->createArgumentsBuilderMock(); + $argumentsBuilder->method('buildArguments')->willReturnCallback(static function ($input, $paths) { + return array_merge(['behat'], $paths); + }); + + $input = $this->createInputInterfaceMock(); + $input->method('getOption')->willReturnMap([ + ['parallel-chunk-size', 2], + ['rerun', false], + ]); + + $factory = new ScenarioTaskFactory($finder, $argumentsBuilder); + $tasks = $factory->createTasks($input); + + $this->assertCount(2, $tasks); + + $this->assertInstanceOf(Task::class, $tasks[0]); + $this->assertEquals(['f1.feature:1', 'f1.feature:2'], $tasks[0]->getPaths()); + $this->assertEquals(['behat', 'f1.feature:1', 'f1.feature:2'], $tasks[0]->getCommand()); + $this->assertCount(2, $tasks[0]->getUnits()); + + $this->assertInstanceOf(Task::class, $tasks[1]); + $this->assertEquals(['f1.feature:3'], $tasks[1]->getPaths()); + $this->assertEquals(['behat', 'f1.feature:3'], $tasks[1]->getCommand()); + $this->assertCount(1, $tasks[1]->getUnits()); + } +} diff --git a/tests/Task/FeatureTaskFactoryTest.php b/tests/Task/FeatureTaskFactoryTest.php index c1a0fca..e1a2d20 100644 --- a/tests/Task/FeatureTaskFactoryTest.php +++ b/tests/Task/FeatureTaskFactoryTest.php @@ -4,6 +4,7 @@ use DMarynicz\BehatParallelExtension\Task\FeatureTaskFactory; use DMarynicz\BehatParallelExtension\Task\Task; +use DMarynicz\BehatParallelExtension\Task\TaskUnit; use ReflectionException; class FeatureTaskFactoryTest extends TaskFactoryTest @@ -21,21 +22,21 @@ public function testCreateTasks($specsToMock): void $argumentsBuilder = $this->createArgumentsBuilderMock(); $argumentsBuilder->method('buildArguments')->willReturn(['some', 'args']); $input = $this->createInputInterfaceMock(); + $input->method('getOption')->willReturnMap([ + ['parallel-chunk-size', 1], + ['rerun', false], + ]); $expected = []; foreach ($specsToMock as $suiteToMock) { $suite = $this->createSuiteMock($suiteToMock); foreach ($suiteToMock['features'] as $featureToMock) { - $feature = $this->createFeatureMock($featureToMock); - foreach ($featureToMock['scenarios'] as $scenarioToMock) { - $expected[] = new Task( - $suite, - $feature, - $featureToMock['file'], - ['some', 'args'], - null - ); - } + $feature = $this->createFeatureMock($featureToMock); + $expected[] = new Task( + $suite, + [new TaskUnit($feature)], + ['some', 'args'] + ); } } diff --git a/tests/Task/ScenarioTaskFactoryTest.php b/tests/Task/ScenarioTaskFactoryTest.php index d8b310e..a93c45b 100644 --- a/tests/Task/ScenarioTaskFactoryTest.php +++ b/tests/Task/ScenarioTaskFactoryTest.php @@ -4,6 +4,7 @@ use DMarynicz\BehatParallelExtension\Task\ScenarioTaskFactory; use DMarynicz\BehatParallelExtension\Task\Task; +use DMarynicz\BehatParallelExtension\Task\TaskUnit; use ReflectionException; class ScenarioTaskFactoryTest extends TaskFactoryTest @@ -21,6 +22,10 @@ public function testCreateTasks($specsToMock): void $argumentsBuilder = $this->createArgumentsBuilderMock(); $argumentsBuilder->method('buildArguments')->willReturn(['some', 'args']); $input = $this->createInputInterfaceMock(); + $input->method('getOption')->willReturnMap([ + ['parallel-chunk-size', 1], + ['rerun', false], + ]); $expected = []; foreach ($specsToMock as $suiteToMock) { @@ -29,13 +34,10 @@ public function testCreateTasks($specsToMock): void $feature = $this->createFeatureMock($featureToMock); foreach ($featureToMock['scenarios'] as $scenarioToMock) { $scenario = $this->createScenarioMock($scenarioToMock); - $testPath = sprintf('%s:%s', $featureToMock['file'], $scenarioToMock['line']); $expected[] = new Task( $suite, - $feature, - $testPath, - ['some', 'args'], - $scenario + [new TaskUnit($feature, $scenario)], + ['some', 'args'] ); } } diff --git a/tests/Task/TaskArgumentsBuilderTest.php b/tests/Task/TaskArgumentsBuilderTest.php index d1c00d3..95dec79 100644 --- a/tests/Task/TaskArgumentsBuilderTest.php +++ b/tests/Task/TaskArgumentsBuilderTest.php @@ -24,20 +24,20 @@ protected function setUp(): void * @param array $options * @param array $arguments * @param string $phpPath - * @param string $path + * @param string[] $paths * @param string[] $expected * * @throws ReflectionException * * @dataProvider buildArgumentsProvider */ - public function testBuildArguments($options, $arguments, $phpPath, $path, $expected): void + public function testBuildArguments($options, $arguments, $phpPath, $paths, $expected): void { $input = $this->createInputInterfaceMock($options, $arguments); $finder = $this->createPhpExecutableFinder($phpPath); $builder = new TaskArgumentsBuilder($finder); - $actual = $builder->buildArguments($input, $path); + $actual = $builder->buildArguments($input, $paths); $this->assertEquals($expected, $actual); } @@ -64,7 +64,7 @@ public function buildArgumentsProvider(): array ], ['paths' => 'path'], 'php-binary', - 'path-some-test.feature:21', + ['path-some-test.feature:21'], [ 'php-binary', 'behat', @@ -90,6 +90,7 @@ public function buildArgumentsProvider(): array 'some-false-option' => false, 'parallel' => true, 'parallel-feature' => true, + 'parallel-chunk-size' => 1, 'string-option' => 'string', 'integer-option' => 123, 'float-option' => 1.23, @@ -102,7 +103,7 @@ public function buildArgumentsProvider(): array ], ['paths' => 'path'], 'php-binary', - 'path-some-test.feature:21', + ['path-some-test.feature:21', 'path-some-test.feature:22'], [ 'php-binary', 'behat', @@ -120,6 +121,7 @@ public function buildArgumentsProvider(): array '--no-interaction', '--fail-on-undefined-step', 'path-some-test.feature:21', + 'path-some-test.feature:22', ], ], ]; diff --git a/tests/Task/TaskTest.php b/tests/Task/TaskTest.php index c9bb4a5..a0b7380 100644 --- a/tests/Task/TaskTest.php +++ b/tests/Task/TaskTest.php @@ -6,6 +6,7 @@ use Behat\Gherkin\Node\ScenarioInterface; use Behat\Testwork\Suite\Suite; use DMarynicz\BehatParallelExtension\Task\Task; +use DMarynicz\BehatParallelExtension\Task\TaskUnit; use PHPUnit\Framework\TestCase; class TaskTest extends TestCase @@ -17,41 +18,31 @@ class TaskTest extends TestCase */ public function testTask($testWithScenario): void { - //Suite $suite, FeatureNode $feature, $path, $command = [], Scenario $scenario = null $suite = $this->createMock(Suite::class); $feature = $this->createMock(FeatureNode::class); - $path = 'some-path'; - $command = ['php', 'ls']; + $feature->method('getFile')->willReturn('some-path'); - $task = new Task($suite, $feature, $path, $command); if ($testWithScenario) { $scenario = $this->createMock(ScenarioInterface::class); - $task = new Task($suite, $feature, $path, $command, $scenario); + $scenario->method('getLine')->willReturn(123); + $units = [new TaskUnit($feature, $scenario)]; + } else { + $units = [new TaskUnit($feature)]; } - $this->assertEquals( - $task->getSuite(), - $this->createMock(Suite::class) - ); + $command = ['php', 'ls']; + $task = new Task($suite, $units, $command); - $this->assertEquals( - $task->getFeature(), - $this->createMock(FeatureNode::class) - ); + $this->assertSame($suite, $task->getSuite()); $this->assertEquals( - 'some-path', - $task->getPath() + $testWithScenario ? ['some-path:123'] : ['some-path'], + $task->getPaths() ); - $this->assertEquals( - ['php', 'ls'], - $task->getCommand() - ); - $this->assertEquals( - $testWithScenario ? $this->createMock(ScenarioInterface::class) : null, - $task->getScenario() - ); + $this->assertEquals(['php', 'ls'], $task->getCommand()); + + $this->assertSame($units, $task->getUnits()); } /** diff --git a/tests/fixtures/chunked/behat.yml.dist b/tests/fixtures/chunked/behat.yml.dist new file mode 100644 index 0000000..d930b82 --- /dev/null +++ b/tests/fixtures/chunked/behat.yml.dist @@ -0,0 +1,16 @@ +default: + suites: + suite01: + paths: + - '%paths.base%/suite01' + contexts: + - DMarynicz\Tests\Behat\Context\ChunkTestContext: + filesPath: '%paths.base%' + suite02: + paths: + - '%paths.base%/suite02' + contexts: + - DMarynicz\Tests\Behat\Context\ChunkTestContext: + filesPath: '%paths.base%' + extensions: + DMarynicz\BehatParallelExtension\Extension: ~ diff --git a/tests/fixtures/chunked/suite01/f1.feature b/tests/fixtures/chunked/suite01/f1.feature new file mode 100644 index 0000000..3ef215a --- /dev/null +++ b/tests/fixtures/chunked/suite01/f1.feature @@ -0,0 +1,3 @@ +Feature: Feature 1 + Scenario: Scenario 1 + Given I log behat command to "chunk_test.json" diff --git a/tests/fixtures/chunked/suite01/f2.feature b/tests/fixtures/chunked/suite01/f2.feature new file mode 100644 index 0000000..2c681ce --- /dev/null +++ b/tests/fixtures/chunked/suite01/f2.feature @@ -0,0 +1,3 @@ +Feature: Feature 2 + Scenario: Scenario 2 + Given I log behat command to "chunk_test.json" diff --git a/tests/fixtures/chunked/suite02/f3.feature b/tests/fixtures/chunked/suite02/f3.feature new file mode 100644 index 0000000..a5db310 --- /dev/null +++ b/tests/fixtures/chunked/suite02/f3.feature @@ -0,0 +1,5 @@ +Feature: Feature 3 + Scenario: Scenario 3 + Given I log behat command to "chunk_test.json" + Scenario: Scenario 4 + Given I log behat command to "chunk_test.json" diff --git a/tests/fixtures/chunked_rerun/behat.yml.dist b/tests/fixtures/chunked_rerun/behat.yml.dist new file mode 100644 index 0000000..3a17711 --- /dev/null +++ b/tests/fixtures/chunked_rerun/behat.yml.dist @@ -0,0 +1,11 @@ +default: + suites: + suite01: + paths: + - '%paths.base%/suite01' + contexts: + - DMarynicz\Tests\Behat\Context\ChunkTestContext: + filesPath: '%paths.base%' + - DMarynicz\Tests\Behat\Context\SimulateTestContext + extensions: + DMarynicz\BehatParallelExtension\Extension: ~ diff --git a/tests/fixtures/chunked_rerun/suite01/f1.feature b/tests/fixtures/chunked_rerun/suite01/f1.feature new file mode 100644 index 0000000..8a95d1b --- /dev/null +++ b/tests/fixtures/chunked_rerun/suite01/f1.feature @@ -0,0 +1,4 @@ +Feature: Feature 1 + Scenario: Scenario 1 + Given I log behat command to "chunk_test.json" + Then this test will be successful diff --git a/tests/fixtures/chunked_rerun/suite01/f2.feature b/tests/fixtures/chunked_rerun/suite01/f2.feature new file mode 100644 index 0000000..eddec62 --- /dev/null +++ b/tests/fixtures/chunked_rerun/suite01/f2.feature @@ -0,0 +1,4 @@ +Feature: Feature 2 + Scenario: Scenario 2 + Given I log behat command to "chunk_test.json" + Then this test will fail diff --git a/tests/fixtures/chunked_rerun/suite01/f3.feature b/tests/fixtures/chunked_rerun/suite01/f3.feature new file mode 100644 index 0000000..d04ef30 --- /dev/null +++ b/tests/fixtures/chunked_rerun/suite01/f3.feature @@ -0,0 +1,4 @@ +Feature: Feature 3 + Scenario: Scenario 3 + Given I log behat command to "chunk_test.json" + Then this test will be successful