diff --git a/README.md b/README.md index 72bf7e1..e963cef 100644 --- a/README.md +++ b/README.md @@ -114,47 +114,48 @@ By default, the action includes any files in your project that have a `Test` lab The **Run MATLAB Tests** action accepts optional inputs. For example, you can add folders to the MATLAB search path, control which tests to run, and generate various artifacts. -| Input | Description | -| ---------------------------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | -| `source-folder` |

(Optional) Location of the folder containing source code, specified as a path relative to the project root folder. The specified folder and its subfolders are added to the top of the MATLAB search path. If you specify `source-folder` and then generate a coverage report, the action uses only the source code in the specified folder and its subfolders to generate the report. You can specify multiple folders using a colon-separated or semicolon-separated list.

**Example:** `source-folder: source`
**Example:** `source-folder: source/folderA; source/folderB`

| -| `select-by-folder` |

(Optional) Location of the folder used to select test suite elements, specified as a path relative to the project root folder. To create a test suite, the action uses only the tests in the specified folder and its subfolders. You can specify multiple folders using a colon-separated or semicolon-separated list.

**Example:** `select-by-folder: test`
**Example:** `select-by-folder: test/folderA; test/folderB`

| -| `select-by-tag` |

(Optional) Test tag used to select test suite elements. To create a test suite, the action uses only the test elements with the specified tag.

**Example:** `select-by-tag: Unit`

| -| `strict` |

(Optional) Option to apply strict checks when running tests, specified as `false` or `true`. By default, the value is `false`. If you specify a value of `true`, the action generates a qualification failure whenever a test issues a warning.

**Example:** `strict: true`

| -| `use-parallel` |

(Optional) Option to run tests in parallel, specified as `false` or `true`. By default, the value is `false` and tests run in serial. If the test runner configuration is suited for parallelization, you can specify a value of `true` to run tests in parallel. This input requires a Parallel Computing Toolbox license.

**Example:** `use-parallel: true`

| -| `output-detail` |

(Optional) Amount of event detail displayed for the test run, specified as `none`, `terse`, `concise`, `detailed`, or `verbose`. By default, the action displays failing and logged events at the `detailed` level and test run progress at the `concise` level.

**Example:** `output-detail: verbose`

| -| `logging-level` |

(Optional) Maximum verbosity level for logged diagnostics included for the test run, specified as `none`, `terse`, `concise`, `detailed`, or `verbose`. By default, the action includes diagnostics logged at the `terse` level.

**Example:** `logging-level: detailed`

| -| `test-results-html` |

(Optional) Location to write the test results in HTML format, specified as a folder path relative to the project root folder.

**Example:** `test-results-html: test-results`

| -| `test-results-pdf` |

(Optional) Location to write the test results in PDF format, specified as a file path relative to the project root folder. On macOS platforms, this input is supported in MATLAB R2020b and later.

**Example:** `test-results-pdf: test-results/results.pdf`

| -| `test-results-junit` |

(Optional) Location to write the test results in JUnit-style XML format, specified as a file path relative to the project root folder.

**Example:** `test-results-junit: test-results/results.xml`

| -| `test-results-simulink-test` |

(Optional) Location to export Simulink Test Manager results in MLDATX format, specified as a file path relative to the project root folder. This input requires a Simulink Test license and is supported in MATLAB R2019a and later.

**Example:** `test-results-simulink-test: test-results/results.mldatx`

| -| `code-coverage-html` |

(Optional) Location to write the code coverage results in HTML format, specified as a folder path relative to the project root folder. The results include statement and function coverage metrics.

**Example:** `code-coverage-html: code-coverage`

| -| `code-coverage-cobertura` |

(Optional) Location to write the code coverage results in Cobertura XML format, specified as a file path relative to the project root folder.

**Example:** `code-coverage-cobertura: code-coverage/coverage.xml`

| -| `code-coverage-metric-level` |

(Optional) Level of code coverage metrics to include in the GitHub job summary, specified as `statement`, `decision`, `condition`, or `mcdc`. By default, the action uses the highest level (`mcdc`) when generating the coverage summary, which results in a table that includes the function, statement, decision, condition, and modified condition/decision coverage (MC/DC) metrics. Use this input to include metrics up to and including a specific level, instead of the highest possible level. For example, to include only function and statement coverage metrics, specify `code-coverage-metric-level` as `statement`. For more information about coverage metric levels, see [Types of Code Coverage for MATLAB Source Code](https://www.mathworks.com/help/matlab-test/ug/types-of-code-coverage-for-matlab-source-code.html).

For the action to include coverage metrics in the GitHub job summary, the following requirements must be met:

**Example:** `code-coverage-metric-level: statement`

| -| `model-coverage-html` |

(Optional) Location to write the model coverage results in HTML format, specified as a folder path relative to the project root folder. This input requires a Simulink Coverage™ license and is supported in MATLAB R2018b and later.

**Example:** `model-coverage-html: model-coverage`

| -| `model-coverage-cobertura` |

(Optional) Location to write the model coverage results in Cobertura XML format, specified as a file path relative to the project root folder. This input requires a Simulink Coverage license and is supported in MATLAB R2018b and later.

**Example:** `model-coverage-cobertura: model-coverage/coverage.xml`

| -| `startup-options` |

(Optional) MATLAB startup options, specified as a list of options separated by spaces. For more information about startup options, see [Commonly Used Startup Options](https://www.mathworks.com/help/matlab/matlab_env/commonly-used-startup-options.html).

Using this input to specify the `-batch` or `-r` option is not supported.

**Example:** `startup-options: -nojvm`
**Example:** `startup-options: -nojvm -logfile output.log`

| +| Input | Description | +| ---------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ | +| `source-folder` |

(Optional) Location of the folder containing source code, specified as a path relative to the project root folder. The specified folder and its subfolders are added to the top of the MATLAB search path. If you specify `source-folder` and then generate a coverage report, the action uses only the source code in the specified folder and its subfolders to generate the report. You can specify multiple folders using a colon-separated or semicolon-separated list.

**Example:** `source-folder: source`
**Example:** `source-folder: source/folderA; source/folderB`

| +| `select-by-folder` |

(Optional) Location of the folder used to select test suite elements, specified as a path relative to the project root folder. To create a test suite, the action uses only the tests in the specified folder and its subfolders. You can specify multiple folders using a colon-separated or semicolon-separated list.

**Example:** `select-by-folder: test`
**Example:** `select-by-folder: test/folderA; test/folderB`

| +| `select-by-tag` |

(Optional) Test tag used to select test suite elements. To create a test suite, the action uses only the test elements with the specified tag.

**Example:** `select-by-tag: Unit`

| +| `strict` |

(Optional) Option to apply strict checks when running tests, specified as `false` or `true`. By default, the value is `false`. If you specify a value of `true`, the action generates a qualification failure whenever a test issues a warning.

**Example:** `strict: true`

| +| `use-parallel` |

(Optional) Option to run tests in parallel, specified as `false` or `true`. By default, the value is `false` and tests run in serial. If the test runner configuration is suited for parallelization, you can specify a value of `true` to run tests in parallel. This input requires a Parallel Computing Toolbox license.

**Example:** `use-parallel: true`

| +| `output-detail` |

(Optional) Amount of event detail displayed for the test run, specified as `none`, `terse`, `concise`, `detailed`, or `verbose`. By default, the action displays failing and logged events at the `detailed` level and test run progress at the `concise` level.

**Example:** `output-detail: verbose`

| +| `logging-level` |

(Optional) Maximum verbosity level for logged diagnostics included for the test run, specified as `none`, `terse`, `concise`, `detailed`, or `verbose`. By default, the action includes diagnostics logged at the `terse` level.

**Example:** `logging-level: detailed`

| +| `test-results-html` |

(Optional) Location to write the test results in HTML format, specified as a folder path relative to the project root folder.

**Example:** `test-results-html: test-results`

| +| `test-results-pdf` |

(Optional) Location to write the test results in PDF format, specified as a file path relative to the project root folder. On macOS platforms, this input is supported in MATLAB R2020b and later.

**Example:** `test-results-pdf: test-results/results.pdf`

| +| `test-results-junit` |

(Optional) Location to write the test results in JUnit-style XML format, specified as a file path relative to the project root folder.

**Example:** `test-results-junit: test-results/results.xml`

| +| `test-results-simulink-test` |

(Optional) Location to export Simulink Test Manager results in MLDATX format, specified as a file path relative to the project root folder. This input requires a Simulink Test license and is supported in MATLAB R2019a and later.

**Example:** `test-results-simulink-test: test-results/results.mldatx`

| +| `code-coverage-html` |

(Optional) Location to write the code coverage results in HTML format, specified as a folder path relative to the project root folder. The results include statement and function coverage metrics.

**Example:** `code-coverage-html: code-coverage`

| +| `code-coverage-cobertura` |

(Optional) Location to write the code coverage results in Cobertura XML format, specified as a file path relative to the project root folder.

**Example:** `code-coverage-cobertura: code-coverage/coverage.xml`

| +| `code-coverage-metric-level` |

(Optional) Level of code coverage metrics to include in the GitHub job summary, specified as `statement`, `decision`, `condition`, or `mcdc`. By default, the action uses the highest level (`mcdc`) when generating the coverage summary, which results in a table that includes the function, statement, decision, condition, and modified condition/decision coverage (MC/DC) metrics. Use this input to include metrics up to and including a specific level, instead of the highest possible level. For example, to include only function and statement coverage metrics, specify `code-coverage-metric-level` as `statement`. For more information about coverage metric levels, see [Types of Code Coverage for MATLAB Source Code](https://www.mathworks.com/help/matlab-test/ug/types-of-code-coverage-for-matlab-source-code.html).

For the action to include coverage metrics in the GitHub job summary, the following requirements must be met:

**Example:** `code-coverage-metric-level: statement`

| +| `model-coverage-html` |

(Optional) Location to write the model coverage results in HTML format, specified as a folder path relative to the project root folder. This input requires a Simulink Coverage™ license and is supported in MATLAB R2018b and later.

**Example:** `model-coverage-html: model-coverage`

| +| `model-coverage-cobertura` |

(Optional) Location to write the model coverage results in Cobertura XML format, specified as a file path relative to the project root folder. This input requires a Simulink Coverage license and is supported in MATLAB R2018b and later.

**Example:** `model-coverage-cobertura: model-coverage/coverage.xml`

| +| `generate-summary` |

(Optional) Option to generate summaries of test and coverage results in the GitHub job summary, specified as `true` or `false`. By default, the value is `true`. If you specify a value of `false`, the action does not generate test and coverage summaries in the GitHub job summary. Generating test and coverage summaries requires a MATLAB Test license.

For more information about test and coverage summaries, see [View Test and Coverage Results](#view-test-and-coverage-results).

**Example:** `generate-summary: false`

| +| `startup-options` |

(Optional) MATLAB startup options, specified as a list of options separated by spaces. For more information about startup options, see [Commonly Used Startup Options](https://www.mathworks.com/help/matlab/matlab_env/commonly-used-startup-options.html).

Using this input to specify the `-batch` or `-r` option is not supported.

**Example:** `startup-options: -nojvm`
**Example:** `startup-options: -nojvm -logfile output.log`

| ## View Test and Coverage Results -With a MATLAB Test license, you can view test results and code coverage metrics for your source code on the workflow run summary page. +If you have a MATLAB Test license and the `generate-summary` input is `true`, you can view test results and code coverage metrics for your source code in the GitHub job summary. -Workflow run summary page showing four passed tests from a single test file. The MATLAB Code Coverage table displays the collected code coverage metrics for the source code. In the All tests table, the expanded row displays a table of test results for the test file. +GitHub job summary showing four passed tests from a single test file. The MATLAB Code Coverage table displays the collected code coverage metrics for the source code. In the All tests table, the expanded row displays a table of test results for the test file. ### Test Results Summary -If you have a MATLAB Test license, after your workflow runs, the workflow run summary page displays the number of tests in the test suite, as well as the number of tests that passed, failed, were incomplete (due to assumption failure), and did not run (due to fatal assertion failure). Additionally, the **All tests** table on the page includes a row for each executed test file. Each table row displays the status of a test file, its name, and the time it took to run. You can hover over a filename in the table to see the file path within your project. +If you have a MATLAB Test license, after your workflow runs, the GitHub job summary displays the number of tests in the test suite, as well as the number of tests that passed, failed, were incomplete (due to assumption failure), and did not run (due to fatal assertion failure). Additionally, the **All tests** table in the summary includes a row for each executed test file. Each table row displays the status of a test file, its name, and the time it took to run. You can hover over a filename in the table to see the file path within your project. You can interact with the **All tests** table using the expanders to the left of test filenames and diagnostics. In an expanded table row, individual test results from the corresponding test file appear in a table under the test filename. Each row of the inner table shows the status, procedure name, diagnostics, and duration of a test in the file. > [!TIP] -> With a MATLAB Test license, you also see your MATLAB and Simulink test results on the workflow run summary page when you take any of these actions: +> With a MATLAB Test license, you also see your MATLAB and Simulink test results in the GitHub job summary when you take any of these actions: > -> - Run tests with a [`matlab.buildtool.tasks.TestTask`](https://www.mathworks.com/help/matlab/ref/matlab.buildtool.tasks.testtask-class.html) instance using the [Run MATLAB Build](https://github.com/matlab-actions/run-build/) action. +> - Run tests with a [`matlab.buildtool.tasks.TestTask`](https://www.mathworks.com/help/matlab/ref/matlab.buildtool.tasks.testtask-class.html) instance using the [Run MATLAB Build](https://github.com/matlab-actions/run-build/) or [Run MATLAB Command](https://github.com/matlab-actions/run-command/) action. > - Run tests with a default test runner using the [Run MATLAB Command](https://github.com/matlab-actions/run-command/) action. You can create a default test runner using the [`matlab.unittest.TestRunner.withDefaultPlugins`](https://www.mathworks.com/help/matlab/ref/matlab.unittest.testrunner.withdefaultplugins.html) method. ### Code Coverage Summary -The **MATLAB Code Coverage** table on the workflow run summary page displays the collected code coverage metrics for the MATLAB source code specified by the `source-folder` input of the **Run MATLAB Tests** action. For the action to generate this table, the following requirements must be met: +The **MATLAB Code Coverage** table in the GitHub job summary displays the collected code coverage metrics for the MATLAB source code specified by the `source-folder` input of the **Run MATLAB Tests** action. For the action to generate this table, the following requirements must be met: - MATLAB Test must be installed and licensed on the runner. - You must not specify the `code-coverage-html` or `code-coverage-cobertura` action inputs.