diff --git a/README.md b/README.md index 4b428624..983f0d46 100644 --- a/README.md +++ b/README.md @@ -102,7 +102,7 @@ pwsh actions/Invoke-OSAction.ps1 -Describe run-unit-tests ## Runner Types -Workflows distinguish between standard GitHub-hosted images and integration runners with preinstalled tooling. See [docs/runner-types.md](docs/runner-types.md) for a detailed comparison. +Workflows distinguish between standard GitHub-hosted images and integration runners with preinstalled tooling. See [docs/runner-types.md](docs/runner-types.md) for a detailed comparison. Administrators can update runner metadata without a pull request by following the [Update Runner Runtime guide](docs/update-runner-runtime.md). ## Testing diff --git a/artifacts/linux/summary-standard.md b/artifacts/linux/summary-standard.md index 714e5292..e2578b7c 100644 --- a/artifacts/linux/summary-standard.md +++ b/artifacts/linux/summary-standard.md @@ -1,7 +1,7 @@ ### Test Summary | OS | Passed | Failed | Skipped | Duration (s) | Pass Rate (%) | | --- | --- | --- | --- | --- | --- | -| overall | 60 | 0 | 0 | 24.77 | 100.00 | -| linux | 60 | 0 | 0 | 24.77 | 100.00 | +| overall | 60 | 0 | 0 | 17.37 | 100.00 | +| linux | 60 | 0 | 0 | 17.37 | 100.00 | _For detailed per-test information, see [traceability-standard.md](traceability-standard.md)._ \ No newline at end of file diff --git a/artifacts/linux/summary.md b/artifacts/linux/summary.md index 61f07982..f9860fc7 100644 --- a/artifacts/linux/summary.md +++ b/artifacts/linux/summary.md @@ -1,8 +1,8 @@ ### Test Summary | OS | Passed | Failed | Skipped | Duration (s) | Pass Rate (%) | | --- | --- | --- | --- | --- | --- | -| overall | 60 | 0 | 0 | 24.77 | 100.00 | -| linux | 60 | 0 | 0 | 24.77 | 100.00 | +| overall | 60 | 0 | 0 | 17.37 | 100.00 | +| linux | 60 | 0 | 0 | 17.37 | 100.00 | ### Requirement Summary | Requirement ID | Description | Owner | Total Tests | Passed | Failed | Skipped | Pass Rate (%) | diff --git a/artifacts/linux/traceability-standard.md b/artifacts/linux/traceability-standard.md index a5e5091e..fca65adb 100644 --- a/artifacts/linux/traceability-standard.md +++ b/artifacts/linux/traceability-standard.md @@ -4,19 +4,19 @@ | Requirement | Test ID | Status | Duration (s) | Owner | Evidence | | --- | --- | --- | --- | --- | --- | -| REQ-023 | parses-nested-junit-structures | Passed | 0.012 | | | +| REQ-023 | parses-nested-junit-structures | Passed | 0.009 | | | #### REQ-024 (100% passed) | Requirement | Test ID | Status | Duration (s) | Owner | Evidence | | --- | --- | --- | --- | --- | --- | -| REQ-024 | captures-root-testsuites-attributes | Passed | 0.011 | | | +| REQ-024 | captures-root-testsuites-attributes | Passed | 0.005 | | | #### REQ-025 (100% passed) | Requirement | Test ID | Status | Duration (s) | Owner | Evidence | | --- | --- | --- | --- | --- | --- | -| REQ-025 | captures-testsuite-attributes | Passed | 0.008 | | | +| REQ-025 | captures-testsuite-attributes | Passed | 0.005 | | | #### REQ-026 (100% passed) @@ -28,7 +28,7 @@ | Requirement | Test ID | Status | Duration (s) | Owner | Evidence | | --- | --- | --- | --- | --- | --- | -| REQ-027 | captures-testcase-attributes-and-skipped-message | Passed | 0.002 | | | +| REQ-027 | captures-testcase-attributes-and-skipped-message | Passed | 0.005 | | | #### REQ-028 (100% passed) @@ -46,7 +46,7 @@ | Requirement | Test ID | Status | Duration (s) | Owner | Evidence | | --- | --- | --- | --- | --- | --- | -| REQ-030 | builds-traceability-matrix-with-skipped-reasons | Passed | 0.002 | | | +| REQ-030 | builds-traceability-matrix-with-skipped-reasons | Passed | 0.001 | | | #### REQ-031 (100% passed) @@ -64,60 +64,60 @@ | Requirement | Test ID | Status | Duration (s) | Owner | Evidence | | --- | --- | --- | --- | --- | --- | -| REQ-033 | throws-error-for-malformed-xml | Passed | 0.002 | | | +| REQ-033 | throws-error-for-malformed-xml | Passed | 0.001 | | |
Unmapped (100% passed) | Requirement | Test ID | Status | Duration (s) | Owner | Evidence | | --- | --- | --- | --- | --- | --- | -| Unmapped | associates-classname-with-requirement | Passed | 0.013 | | | -| Unmapped | buildissuebranchname-formats-branch-name | Passed | 0.002 | | | +| Unmapped | associates-classname-with-requirement | Passed | 0.016 | | | +| Unmapped | buildissuebranchname-formats-branch-name | Passed | 0.001 | | | | Unmapped | buildissuebranchname-rejects-non-numeric-input | Passed | 0.001 | | | | Unmapped | buildprofile1.iconeditor.addtokentolabview.dispatcher.dry-runs-add-token-to-labview-with-expected-arguments- | Passed | 0.482 | | | | Unmapped | buildprofile1.iconeditor.applyvipc.dispatcher.dry-runs-apply-vipc-with-expected-arguments- | Passed | 0.042 | | | | Unmapped | buildprofile1.iconeditor.buildvipackage.dispatcher.dry-runs-build-vi-package-with-expected-arguments- | Passed | 0.106 | | | | Unmapped | buildprofile1.iconeditor.closelabview.dispatcher.dry-runs-close-labview-with-expected-arguments- | Passed | 0.036 | | | | Unmapped | buildsummary-splits-totals-by-os | Passed | 0.001 | | | -| Unmapped | collecttestcases-captures-requirement-property | Passed | 0.011 | | | -| Unmapped | collecttestcases-uses-evidence-property-and-falls-back-to-directory-scan | Passed | 0.009 | | | -| Unmapped | collecttestcases-uses-machine-name-property-for-owner | Passed | 0.008 | | | -| Unmapped | computestatuscounts-tallies-test-statuses | Passed | 0.001 | | | -| Unmapped | detects-downloaded-artifacts-path | Passed | 1.280 | | | +| Unmapped | collecttestcases-captures-requirement-property | Passed | 0.014 | | | +| Unmapped | collecttestcases-uses-evidence-property-and-falls-back-to-directory-scan | Passed | 0.014 | | | +| Unmapped | collecttestcases-uses-machine-name-property-for-owner | Passed | 0.009 | | | +| Unmapped | computestatuscounts-tallies-test-statuses | Passed | 0.000 | | | +| Unmapped | detects-downloaded-artifacts-path | Passed | 0.902 | | | | Unmapped | dispatchers-and-parameters-include-descriptions | Passed | 0.003 | | | -| Unmapped | errors-when-strict-unmapped-mode-enabled | Passed | 1.369 | | | +| Unmapped | errors-when-strict-unmapped-mode-enabled | Passed | 0.874 | | | | Unmapped | escapemarkdown-escapes-special-characters | Passed | 0.002 | | | | Unmapped | escapemarkdown-leaves-plain-text-untouched | Passed | 0.000 | | | -| Unmapped | fails-when-commit-lacks-requirement-reference | Passed | 1.427 | | | -| Unmapped | fails-when-requirement-lacks-test-coverage | Passed | 1.607 | | | -| Unmapped | fails-when-tests-are-unmapped | Passed | 1.243 | | | -| Unmapped | fails-when-tests-reference-unknown-requirements | Passed | 1.286 | | | -| Unmapped | fails-without-runner-metadata | Passed | 0.861 | | | +| Unmapped | fails-when-commit-lacks-requirement-reference | Passed | 0.982 | | | +| Unmapped | fails-when-requirement-lacks-test-coverage | Passed | 1.125 | | | +| Unmapped | fails-when-tests-are-unmapped | Passed | 0.888 | | | +| Unmapped | fails-when-tests-reference-unknown-requirements | Passed | 0.929 | | | +| Unmapped | fails-without-runner-metadata | Passed | 0.654 | | | | Unmapped | formaterror-handles-plain-objects | Passed | 0.000 | | | -| Unmapped | formaterror-handles-primitives | Passed | 0.001 | | | -| Unmapped | formaterror-handles-real-error-objects | Passed | 0.004 | | | +| Unmapped | formaterror-handles-primitives | Passed | 0.000 | | | +| Unmapped | formaterror-handles-real-error-objects | Passed | 0.006 | | | | Unmapped | formaterror-handles-unstringifiable-values | Passed | 0.001 | | | -| Unmapped | generate-ci-summary-features | Passed | 0.037 | | | -| Unmapped | groups-owners-and-includes-requirements-and-evidence | Passed | 1.620 | | | -| Unmapped | grouptomarkdown-omits-numeric-identifiers | Passed | 0.007 | | | -| Unmapped | grouptomarkdown-supports-optional-limit-for-truncation | Passed | 0.001 | | | +| Unmapped | generate-ci-summary-features | Passed | 0.034 | | | +| Unmapped | groups-owners-and-includes-requirements-and-evidence | Passed | 1.038 | | | +| Unmapped | grouptomarkdown-omits-numeric-identifiers | Passed | 0.002 | | | +| Unmapped | grouptomarkdown-supports-optional-limit-for-truncation | Passed | 0.003 | | | | Unmapped | handles-root-level-testcases | Passed | 0.001 | | | -| Unmapped | handles-zipped-junit-artifacts | Passed | 1.056 | | | -| Unmapped | ignores-stale-junit-files-outside-artifacts-path | Passed | 1.163 | | | -| Unmapped | loadrequirements-logs-warning-on-invalid-json | Passed | 0.008 | | | -| Unmapped | loadrequirements-merges-multiple-files | Passed | 0.010 | | | -| Unmapped | loadrequirements-warns-and-skips-invalid-entries | Passed | 0.005 | | | -| Unmapped | logs-a-warning-when-no-junit-files-are-found | Passed | 1.281 | | | -| Unmapped | partitions-requirement-groups-by-runner\_type | Passed | 1.036 | | | -| Unmapped | passes-with-coverage-and-requirement-reference | Passed | 1.508 | | | +| Unmapped | handles-zipped-junit-artifacts | Passed | 0.734 | | | +| Unmapped | ignores-stale-junit-files-outside-artifacts-path | Passed | 0.769 | | | +| Unmapped | loadrequirements-logs-warning-on-invalid-json | Passed | 0.018 | | | +| Unmapped | loadrequirements-merges-multiple-files | Passed | 0.005 | | | +| Unmapped | loadrequirements-warns-and-skips-invalid-entries | Passed | 0.010 | | | +| Unmapped | logs-a-warning-when-no-junit-files-are-found | Passed | 0.891 | | | +| Unmapped | partitions-requirement-groups-by-runner\_type | Passed | 0.748 | | | +| Unmapped | passes-with-coverage-and-requirement-reference | Passed | 0.892 | | | | Unmapped | requirementssummarytomarkdown-escapes-pipes-in-description | Passed | 0.001 | | | -| Unmapped | skips-invalid-junit-files-and-still-generates-summary | Passed | 1.681 | | | -| Unmapped | summarytomarkdown-handles-no-tests | Passed | 0.001 | | | +| Unmapped | skips-invalid-junit-files-and-still-generates-summary | Passed | 1.079 | | | +| Unmapped | summarytomarkdown-handles-no-tests | Passed | 0.000 | | | | Unmapped | summarytomarkdown-sorts-os-alphabetically-and-escapes-special-characters | Passed | 0.001 | | | -| Unmapped | throws-when-no-junit-files-found-and-strict-mode-enabled | Passed | 0.840 | | | -| Unmapped | uses-latest-artifact-directory-when-multiple-are-present | Passed | 1.125 | | | -| Unmapped | warns-when-all-tests-are-unmapped | Passed | 1.550 | | | -| Unmapped | writeerrorsummary-appends-error-details-to-summary-file | Passed | 0.005 | | | +| Unmapped | throws-when-no-junit-files-found-and-strict-mode-enabled | Passed | 0.631 | | | +| Unmapped | uses-latest-artifact-directory-when-multiple-are-present | Passed | 0.760 | | | +| Unmapped | warns-when-all-tests-are-unmapped | Passed | 1.173 | | | +| Unmapped | writeerrorsummary-appends-error-details-to-summary-file | Passed | 0.004 | | | | Unmapped | writeerrorsummary-skips-summary-file-for-non-error-throws | Passed | 0.004 | | | -| Unmapped | writes-outputs-to-os-specific-directory | Passed | 1.985 | | | +| Unmapped | writes-outputs-to-os-specific-directory | Passed | 1.447 | | |
\ No newline at end of file diff --git a/artifacts/linux/traceability.json b/artifacts/linux/traceability.json index d6118b87..fc929f4b 100644 --- a/artifacts/linux/traceability.json +++ b/artifacts/linux/traceability.json @@ -9,7 +9,7 @@ "name": "[REQ-023] parses nested JUnit structures", "className": "test", "status": "Passed", - "duration": 0.012007, + "duration": 0.008874, "requirements": [ "REQ-023" ], @@ -26,7 +26,7 @@ "name": "[REQ-024] captures root testsuites attributes", "className": "test", "status": "Passed", - "duration": 0.01146, + "duration": 0.005432, "requirements": [ "REQ-024" ], @@ -43,7 +43,7 @@ "name": "[REQ-025] captures testsuite attributes", "className": "test", "status": "Passed", - "duration": 0.008366, + "duration": 0.005189, "requirements": [ "REQ-025" ], @@ -60,7 +60,7 @@ "name": "[REQ-026] captures suite properties", "className": "test", "status": "Passed", - "duration": 0.001167, + "duration": 0.001028, "requirements": [ "REQ-026" ], @@ -77,7 +77,7 @@ "name": "[REQ-027] captures testcase attributes and skipped message", "className": "test", "status": "Passed", - "duration": 0.001737, + "duration": 0.005125, "requirements": [ "REQ-027" ], @@ -94,7 +94,7 @@ "name": "[REQ-028] extracts requirement identifiers", "className": "test", "status": "Passed", - "duration": 0.003095, + "duration": 0.003049, "requirements": [ "REQ-028" ], @@ -111,7 +111,7 @@ "name": "[REQ-029] aggregates status by requirement and suite", "className": "test", "status": "Passed", - "duration": 0.001544, + "duration": 0.001736, "requirements": [ "REQ-029" ], @@ -128,7 +128,7 @@ "name": "[REQ-030] builds traceability matrix with skipped reasons", "className": "test", "status": "Passed", - "duration": 0.00188, + "duration": 0.001402, "requirements": [ "REQ-030" ], @@ -145,7 +145,7 @@ "name": "[REQ-031] validates missing fields", "className": "test", "status": "Passed", - "duration": 0.001805, + "duration": 0.001683, "requirements": [ "REQ-031" ], @@ -162,7 +162,7 @@ "name": "[REQ-032] preserves unknown attributes", "className": "test", "status": "Passed", - "duration": 0.00096, + "duration": 0.001095, "requirements": [ "REQ-032" ], @@ -179,7 +179,7 @@ "name": "[REQ-033] throws error for malformed XML", "className": "test", "status": "Passed", - "duration": 0.001986, + "duration": 0.001289, "requirements": [ "REQ-033" ], @@ -195,7 +195,7 @@ "name": "associates classname with requirement", "className": "test", "status": "Passed", - "duration": 0.013356, + "duration": 0.015591, "requirements": [], "os": "linux" }, @@ -204,7 +204,7 @@ "name": "buildIssueBranchName formats branch name", "className": "test", "status": "Passed", - "duration": 0.001912, + "duration": 0.001228, "requirements": [], "os": "linux" }, @@ -213,7 +213,7 @@ "name": "buildIssueBranchName rejects non-numeric input", "className": "test", "status": "Passed", - "duration": 0.001386, + "duration": 0.000877, "requirements": [], "os": "linux" }, @@ -258,7 +258,7 @@ "name": "buildSummary splits totals by OS", "className": "test", "status": "Passed", - "duration": 0.000887, + "duration": 0.000795, "requirements": [], "os": "linux" }, @@ -267,7 +267,7 @@ "name": "collectTestCases captures requirement property", "className": "test", "status": "Passed", - "duration": 0.011256, + "duration": 0.014425, "requirements": [], "os": "linux" }, @@ -276,7 +276,7 @@ "name": "collectTestCases uses evidence property and falls back to directory scan", "className": "test", "status": "Passed", - "duration": 0.008639, + "duration": 0.01397, "requirements": [], "os": "linux" }, @@ -285,7 +285,7 @@ "name": "collectTestCases uses machine-name property for owner", "className": "test", "status": "Passed", - "duration": 0.007982, + "duration": 0.009292, "requirements": [], "os": "linux" }, @@ -294,7 +294,7 @@ "name": "computeStatusCounts tallies test statuses", "className": "test", "status": "Passed", - "duration": 0.000535, + "duration": 0.000387, "requirements": [], "os": "linux" }, @@ -303,7 +303,7 @@ "name": "detects downloaded artifacts path", "className": "test", "status": "Passed", - "duration": 1.279692, + "duration": 0.90174, "requirements": [], "os": "linux" }, @@ -312,7 +312,7 @@ "name": "Dispatchers and parameters include descriptions", "className": "test", "status": "Passed", - "duration": 0.002614, + "duration": 0.003299, "requirements": [], "os": "linux" }, @@ -321,7 +321,7 @@ "name": "errors when strict unmapped mode enabled", "className": "test", "status": "Passed", - "duration": 1.368995, + "duration": 0.873634, "requirements": [], "os": "linux" }, @@ -330,7 +330,7 @@ "name": "escapeMarkdown escapes special characters", "className": "test", "status": "Passed", - "duration": 0.002228, + "duration": 0.001622, "requirements": [], "os": "linux" }, @@ -339,7 +339,7 @@ "name": "escapeMarkdown leaves plain text untouched", "className": "test", "status": "Passed", - "duration": 0.000272, + "duration": 0.000216, "requirements": [], "os": "linux" }, @@ -348,7 +348,7 @@ "name": "fails when commit lacks requirement reference", "className": "test", "status": "Passed", - "duration": 1.42704, + "duration": 0.981984, "requirements": [], "os": "linux" }, @@ -357,7 +357,7 @@ "name": "fails when requirement lacks test coverage", "className": "test", "status": "Passed", - "duration": 1.606652, + "duration": 1.124586, "requirements": [], "os": "linux" }, @@ -366,7 +366,7 @@ "name": "fails when tests are unmapped", "className": "test", "status": "Passed", - "duration": 1.243432, + "duration": 0.887565, "requirements": [], "os": "linux" }, @@ -375,7 +375,7 @@ "name": "fails when tests reference unknown requirements", "className": "test", "status": "Passed", - "duration": 1.286002, + "duration": 0.92941, "requirements": [], "os": "linux" }, @@ -384,7 +384,7 @@ "name": "fails without runner metadata", "className": "test", "status": "Passed", - "duration": 0.861494, + "duration": 0.654284, "requirements": [], "os": "linux" }, @@ -393,7 +393,7 @@ "name": "formatError handles plain objects", "className": "test", "status": "Passed", - "duration": 0.000492, + "duration": 0.000497, "requirements": [], "os": "linux" }, @@ -402,7 +402,7 @@ "name": "formatError handles primitives", "className": "test", "status": "Passed", - "duration": 0.000548, + "duration": 0.000233, "requirements": [], "os": "linux" }, @@ -411,7 +411,7 @@ "name": "formatError handles real Error objects", "className": "test", "status": "Passed", - "duration": 0.003753, + "duration": 0.006162, "requirements": [], "os": "linux" }, @@ -420,7 +420,7 @@ "name": "formatError handles unstringifiable values", "className": "test", "status": "Passed", - "duration": 0.001101, + "duration": 0.000979, "requirements": [], "os": "linux" }, @@ -429,7 +429,7 @@ "name": "generate-ci-summary features", "className": "test", "status": "Passed", - "duration": 0.037181, + "duration": 0.033642, "requirements": [], "os": "linux" }, @@ -438,7 +438,7 @@ "name": "groups owners and includes requirements and evidence", "className": "test", "status": "Passed", - "duration": 1.620358, + "duration": 1.037685, "requirements": [], "os": "linux" }, @@ -447,7 +447,7 @@ "name": "groupToMarkdown omits numeric identifiers", "className": "test", "status": "Passed", - "duration": 0.00654, + "duration": 0.002005, "requirements": [], "os": "linux" }, @@ -456,7 +456,7 @@ "name": "groupToMarkdown supports optional limit for truncation", "className": "test", "status": "Passed", - "duration": 0.000931, + "duration": 0.002994, "requirements": [], "os": "linux" }, @@ -465,7 +465,7 @@ "name": "handles root-level testcases", "className": "test", "status": "Passed", - "duration": 0.000828, + "duration": 0.00081, "requirements": [], "os": "linux" }, @@ -474,7 +474,7 @@ "name": "handles zipped JUnit artifacts", "className": "test", "status": "Passed", - "duration": 1.055923, + "duration": 0.734289, "requirements": [], "os": "linux" }, @@ -483,7 +483,7 @@ "name": "ignores stale JUnit files outside artifacts path", "className": "test", "status": "Passed", - "duration": 1.163121, + "duration": 0.769075, "requirements": [], "os": "linux" }, @@ -492,7 +492,7 @@ "name": "loadRequirements logs warning on invalid JSON", "className": "test", "status": "Passed", - "duration": 0.008242, + "duration": 0.017992, "requirements": [], "os": "linux" }, @@ -501,7 +501,7 @@ "name": "loadRequirements merges multiple files", "className": "test", "status": "Passed", - "duration": 0.009819, + "duration": 0.005118, "requirements": [], "os": "linux" }, @@ -510,7 +510,7 @@ "name": "loadRequirements warns and skips invalid entries", "className": "test", "status": "Passed", - "duration": 0.004795, + "duration": 0.010171, "requirements": [], "os": "linux" }, @@ -519,7 +519,7 @@ "name": "logs a warning when no JUnit files are found", "className": "test", "status": "Passed", - "duration": 1.28104, + "duration": 0.890802, "requirements": [], "os": "linux" }, @@ -528,7 +528,7 @@ "name": "partitions requirement groups by runner_type", "className": "test", "status": "Passed", - "duration": 1.035724, + "duration": 0.747529, "requirements": [], "os": "linux" }, @@ -537,7 +537,7 @@ "name": "passes with coverage and requirement reference", "className": "test", "status": "Passed", - "duration": 1.508429, + "duration": 0.892146, "requirements": [], "os": "linux" }, @@ -546,7 +546,7 @@ "name": "requirementsSummaryToMarkdown escapes pipes in description", "className": "test", "status": "Passed", - "duration": 0.000755, + "duration": 0.000673, "requirements": [], "os": "linux" }, @@ -555,7 +555,7 @@ "name": "skips invalid JUnit files and still generates summary", "className": "test", "status": "Passed", - "duration": 1.680781, + "duration": 1.079416, "requirements": [], "os": "linux" }, @@ -564,7 +564,7 @@ "name": "summaryToMarkdown handles no tests", "className": "test", "status": "Passed", - "duration": 0.00063, + "duration": 0.000467, "requirements": [], "os": "linux" }, @@ -573,7 +573,7 @@ "name": "summaryToMarkdown sorts OS alphabetically and escapes special characters", "className": "test", "status": "Passed", - "duration": 0.000765, + "duration": 0.001463, "requirements": [], "os": "linux" }, @@ -582,7 +582,7 @@ "name": "throws when no JUnit files found and strict mode enabled", "className": "test", "status": "Passed", - "duration": 0.839618, + "duration": 0.631476, "requirements": [], "os": "linux" }, @@ -591,7 +591,7 @@ "name": "uses latest artifact directory when multiple are present", "className": "test", "status": "Passed", - "duration": 1.124753, + "duration": 0.760357, "requirements": [], "os": "linux" }, @@ -600,7 +600,7 @@ "name": "warns when all tests are unmapped", "className": "test", "status": "Passed", - "duration": 1.549502, + "duration": 1.1726, "requirements": [], "os": "linux" }, @@ -609,7 +609,7 @@ "name": "writeErrorSummary appends error details to summary file", "className": "test", "status": "Passed", - "duration": 0.005018, + "duration": 0.004009, "requirements": [], "os": "linux" }, @@ -618,7 +618,7 @@ "name": "writeErrorSummary skips summary file for non-Error throws", "className": "test", "status": "Passed", - "duration": 0.004102, + "duration": 0.004272, "requirements": [], "os": "linux" }, @@ -627,7 +627,7 @@ "name": "writes outputs to OS-specific directory", "className": "test", "status": "Passed", - "duration": 1.985494, + "duration": 1.44713, "requirements": [], "os": "linux" } @@ -639,7 +639,7 @@ "passed": 60, "failed": 0, "skipped": 0, - "duration": 24.766624000000004, + "duration": 17.370799, "rate": 100 }, "byOs": { @@ -647,7 +647,7 @@ "passed": 60, "failed": 0, "skipped": 0, - "duration": 24.766624000000004, + "duration": 17.370799, "rate": 100 } } diff --git a/artifacts/linux/traceability.md b/artifacts/linux/traceability.md index a5e5091e..fca65adb 100644 --- a/artifacts/linux/traceability.md +++ b/artifacts/linux/traceability.md @@ -4,19 +4,19 @@ | Requirement | Test ID | Status | Duration (s) | Owner | Evidence | | --- | --- | --- | --- | --- | --- | -| REQ-023 | parses-nested-junit-structures | Passed | 0.012 | | | +| REQ-023 | parses-nested-junit-structures | Passed | 0.009 | | | #### REQ-024 (100% passed) | Requirement | Test ID | Status | Duration (s) | Owner | Evidence | | --- | --- | --- | --- | --- | --- | -| REQ-024 | captures-root-testsuites-attributes | Passed | 0.011 | | | +| REQ-024 | captures-root-testsuites-attributes | Passed | 0.005 | | | #### REQ-025 (100% passed) | Requirement | Test ID | Status | Duration (s) | Owner | Evidence | | --- | --- | --- | --- | --- | --- | -| REQ-025 | captures-testsuite-attributes | Passed | 0.008 | | | +| REQ-025 | captures-testsuite-attributes | Passed | 0.005 | | | #### REQ-026 (100% passed) @@ -28,7 +28,7 @@ | Requirement | Test ID | Status | Duration (s) | Owner | Evidence | | --- | --- | --- | --- | --- | --- | -| REQ-027 | captures-testcase-attributes-and-skipped-message | Passed | 0.002 | | | +| REQ-027 | captures-testcase-attributes-and-skipped-message | Passed | 0.005 | | | #### REQ-028 (100% passed) @@ -46,7 +46,7 @@ | Requirement | Test ID | Status | Duration (s) | Owner | Evidence | | --- | --- | --- | --- | --- | --- | -| REQ-030 | builds-traceability-matrix-with-skipped-reasons | Passed | 0.002 | | | +| REQ-030 | builds-traceability-matrix-with-skipped-reasons | Passed | 0.001 | | | #### REQ-031 (100% passed) @@ -64,60 +64,60 @@ | Requirement | Test ID | Status | Duration (s) | Owner | Evidence | | --- | --- | --- | --- | --- | --- | -| REQ-033 | throws-error-for-malformed-xml | Passed | 0.002 | | | +| REQ-033 | throws-error-for-malformed-xml | Passed | 0.001 | | |
Unmapped (100% passed) | Requirement | Test ID | Status | Duration (s) | Owner | Evidence | | --- | --- | --- | --- | --- | --- | -| Unmapped | associates-classname-with-requirement | Passed | 0.013 | | | -| Unmapped | buildissuebranchname-formats-branch-name | Passed | 0.002 | | | +| Unmapped | associates-classname-with-requirement | Passed | 0.016 | | | +| Unmapped | buildissuebranchname-formats-branch-name | Passed | 0.001 | | | | Unmapped | buildissuebranchname-rejects-non-numeric-input | Passed | 0.001 | | | | Unmapped | buildprofile1.iconeditor.addtokentolabview.dispatcher.dry-runs-add-token-to-labview-with-expected-arguments- | Passed | 0.482 | | | | Unmapped | buildprofile1.iconeditor.applyvipc.dispatcher.dry-runs-apply-vipc-with-expected-arguments- | Passed | 0.042 | | | | Unmapped | buildprofile1.iconeditor.buildvipackage.dispatcher.dry-runs-build-vi-package-with-expected-arguments- | Passed | 0.106 | | | | Unmapped | buildprofile1.iconeditor.closelabview.dispatcher.dry-runs-close-labview-with-expected-arguments- | Passed | 0.036 | | | | Unmapped | buildsummary-splits-totals-by-os | Passed | 0.001 | | | -| Unmapped | collecttestcases-captures-requirement-property | Passed | 0.011 | | | -| Unmapped | collecttestcases-uses-evidence-property-and-falls-back-to-directory-scan | Passed | 0.009 | | | -| Unmapped | collecttestcases-uses-machine-name-property-for-owner | Passed | 0.008 | | | -| Unmapped | computestatuscounts-tallies-test-statuses | Passed | 0.001 | | | -| Unmapped | detects-downloaded-artifacts-path | Passed | 1.280 | | | +| Unmapped | collecttestcases-captures-requirement-property | Passed | 0.014 | | | +| Unmapped | collecttestcases-uses-evidence-property-and-falls-back-to-directory-scan | Passed | 0.014 | | | +| Unmapped | collecttestcases-uses-machine-name-property-for-owner | Passed | 0.009 | | | +| Unmapped | computestatuscounts-tallies-test-statuses | Passed | 0.000 | | | +| Unmapped | detects-downloaded-artifacts-path | Passed | 0.902 | | | | Unmapped | dispatchers-and-parameters-include-descriptions | Passed | 0.003 | | | -| Unmapped | errors-when-strict-unmapped-mode-enabled | Passed | 1.369 | | | +| Unmapped | errors-when-strict-unmapped-mode-enabled | Passed | 0.874 | | | | Unmapped | escapemarkdown-escapes-special-characters | Passed | 0.002 | | | | Unmapped | escapemarkdown-leaves-plain-text-untouched | Passed | 0.000 | | | -| Unmapped | fails-when-commit-lacks-requirement-reference | Passed | 1.427 | | | -| Unmapped | fails-when-requirement-lacks-test-coverage | Passed | 1.607 | | | -| Unmapped | fails-when-tests-are-unmapped | Passed | 1.243 | | | -| Unmapped | fails-when-tests-reference-unknown-requirements | Passed | 1.286 | | | -| Unmapped | fails-without-runner-metadata | Passed | 0.861 | | | +| Unmapped | fails-when-commit-lacks-requirement-reference | Passed | 0.982 | | | +| Unmapped | fails-when-requirement-lacks-test-coverage | Passed | 1.125 | | | +| Unmapped | fails-when-tests-are-unmapped | Passed | 0.888 | | | +| Unmapped | fails-when-tests-reference-unknown-requirements | Passed | 0.929 | | | +| Unmapped | fails-without-runner-metadata | Passed | 0.654 | | | | Unmapped | formaterror-handles-plain-objects | Passed | 0.000 | | | -| Unmapped | formaterror-handles-primitives | Passed | 0.001 | | | -| Unmapped | formaterror-handles-real-error-objects | Passed | 0.004 | | | +| Unmapped | formaterror-handles-primitives | Passed | 0.000 | | | +| Unmapped | formaterror-handles-real-error-objects | Passed | 0.006 | | | | Unmapped | formaterror-handles-unstringifiable-values | Passed | 0.001 | | | -| Unmapped | generate-ci-summary-features | Passed | 0.037 | | | -| Unmapped | groups-owners-and-includes-requirements-and-evidence | Passed | 1.620 | | | -| Unmapped | grouptomarkdown-omits-numeric-identifiers | Passed | 0.007 | | | -| Unmapped | grouptomarkdown-supports-optional-limit-for-truncation | Passed | 0.001 | | | +| Unmapped | generate-ci-summary-features | Passed | 0.034 | | | +| Unmapped | groups-owners-and-includes-requirements-and-evidence | Passed | 1.038 | | | +| Unmapped | grouptomarkdown-omits-numeric-identifiers | Passed | 0.002 | | | +| Unmapped | grouptomarkdown-supports-optional-limit-for-truncation | Passed | 0.003 | | | | Unmapped | handles-root-level-testcases | Passed | 0.001 | | | -| Unmapped | handles-zipped-junit-artifacts | Passed | 1.056 | | | -| Unmapped | ignores-stale-junit-files-outside-artifacts-path | Passed | 1.163 | | | -| Unmapped | loadrequirements-logs-warning-on-invalid-json | Passed | 0.008 | | | -| Unmapped | loadrequirements-merges-multiple-files | Passed | 0.010 | | | -| Unmapped | loadrequirements-warns-and-skips-invalid-entries | Passed | 0.005 | | | -| Unmapped | logs-a-warning-when-no-junit-files-are-found | Passed | 1.281 | | | -| Unmapped | partitions-requirement-groups-by-runner\_type | Passed | 1.036 | | | -| Unmapped | passes-with-coverage-and-requirement-reference | Passed | 1.508 | | | +| Unmapped | handles-zipped-junit-artifacts | Passed | 0.734 | | | +| Unmapped | ignores-stale-junit-files-outside-artifacts-path | Passed | 0.769 | | | +| Unmapped | loadrequirements-logs-warning-on-invalid-json | Passed | 0.018 | | | +| Unmapped | loadrequirements-merges-multiple-files | Passed | 0.005 | | | +| Unmapped | loadrequirements-warns-and-skips-invalid-entries | Passed | 0.010 | | | +| Unmapped | logs-a-warning-when-no-junit-files-are-found | Passed | 0.891 | | | +| Unmapped | partitions-requirement-groups-by-runner\_type | Passed | 0.748 | | | +| Unmapped | passes-with-coverage-and-requirement-reference | Passed | 0.892 | | | | Unmapped | requirementssummarytomarkdown-escapes-pipes-in-description | Passed | 0.001 | | | -| Unmapped | skips-invalid-junit-files-and-still-generates-summary | Passed | 1.681 | | | -| Unmapped | summarytomarkdown-handles-no-tests | Passed | 0.001 | | | +| Unmapped | skips-invalid-junit-files-and-still-generates-summary | Passed | 1.079 | | | +| Unmapped | summarytomarkdown-handles-no-tests | Passed | 0.000 | | | | Unmapped | summarytomarkdown-sorts-os-alphabetically-and-escapes-special-characters | Passed | 0.001 | | | -| Unmapped | throws-when-no-junit-files-found-and-strict-mode-enabled | Passed | 0.840 | | | -| Unmapped | uses-latest-artifact-directory-when-multiple-are-present | Passed | 1.125 | | | -| Unmapped | warns-when-all-tests-are-unmapped | Passed | 1.550 | | | -| Unmapped | writeerrorsummary-appends-error-details-to-summary-file | Passed | 0.005 | | | +| Unmapped | throws-when-no-junit-files-found-and-strict-mode-enabled | Passed | 0.631 | | | +| Unmapped | uses-latest-artifact-directory-when-multiple-are-present | Passed | 0.760 | | | +| Unmapped | warns-when-all-tests-are-unmapped | Passed | 1.173 | | | +| Unmapped | writeerrorsummary-appends-error-details-to-summary-file | Passed | 0.004 | | | | Unmapped | writeerrorsummary-skips-summary-file-for-non-error-throws | Passed | 0.004 | | | -| Unmapped | writes-outputs-to-os-specific-directory | Passed | 1.985 | | | +| Unmapped | writes-outputs-to-os-specific-directory | Passed | 1.447 | | |
\ No newline at end of file diff --git a/ci_evidence.txt b/ci_evidence.txt index b52c97eb..2415f525 100644 --- a/ci_evidence.txt +++ b/ci_evidence.txt @@ -1 +1 @@ -{"pipeline":"Unknown","git_sha":"aef6fbdeb462ead1d824f06c451ea42466d3b3a7","req_status":{"REQ-023":"PASS","REQ-024":"PASS","REQ-025":"PASS","REQ-026":"PASS","REQ-027":"PASS","REQ-028":"PASS","REQ-029":"PASS","REQ-030":"PASS","REQ-031":"PASS","REQ-032":"PASS","REQ-033":"PASS"}} \ No newline at end of file +{"pipeline":"Unknown","git_sha":"12e895b3f719e84f5f0e1c78263f698c05fe0367","req_status":{"REQ-023":"PASS","REQ-024":"PASS","REQ-025":"PASS","REQ-026":"PASS","REQ-027":"PASS","REQ-028":"PASS","REQ-029":"PASS","REQ-030":"PASS","REQ-031":"PASS","REQ-032":"PASS","REQ-033":"PASS"}} \ No newline at end of file diff --git a/docs/runner-types.md b/docs/runner-types.md index eadbef22..ccd489f7 100644 --- a/docs/runner-types.md +++ b/docs/runner-types.md @@ -56,11 +56,4 @@ By explicitly classifying jobs, teams can scale the project efficiently—routin ## Updating runner metadata -Administrators can modify runner mappings without opening a pull request by using the `update-runner-runtime` workflow. Trigger the workflow from the Actions tab and supply the following inputs: - -* `runner_key` – key of the runner in the requirements file. -* `runner_label` – label applied to the runner. -* `runner_type` – value for `runner_type` such as `standard` or `integration`. -* `skip_dry_run` – set to `true` to force real execution. - -The workflow updates the corresponding `requirements*.json` file, commits the change with `GITHUB_TOKEN`, and optionally runs registry derivation, traceability checks, and `actionlint` to validate the update. +Administrators can modify runner mappings without opening a pull request by running the [`update-runner-runtime`](../.github/workflows/update-runner-runtime.yml) workflow. For step‑by‑step instructions, input examples, and troubleshooting tips, see the [Update Runner Runtime guide](update-runner-runtime.md). diff --git a/docs/update-runner-runtime.md b/docs/update-runner-runtime.md new file mode 100644 index 00000000..7fe1e389 --- /dev/null +++ b/docs/update-runner-runtime.md @@ -0,0 +1,46 @@ +# Update Runner Runtime Workflow + +This guide walks administrators through updating runner metadata without opening a pull request. The [`update-runner-runtime.yml`](../.github/workflows/update-runner-runtime.yml) workflow edits the `requirements*.json` files and pushes the change to the default branch. + +## Triggering the workflow + +1. In the repository on GitHub, open the **Actions** tab. +2. Select **Update Runner Runtime**. +3. Choose **Run workflow**. +4. Provide the required inputs and click **Run workflow**. + +The job runs on a GitHub-hosted Ubuntu runner. + +## Inputs + +| Name | Description | Example | +| ---- | ----------- | ------- | +| `runner_key` | Key of the runner entry in `requirements*.json`. | `self-hosted-windows-lv-integration` | +| `runner_label` | Value for `runner_label`. Leave blank to keep the current label. | `self-hosted-windows-lv` | +| `runner_type` | Value for `runner_type`, such as `standard` or `integration`. | `integration` | +| `skip_dry_run` | `true` to set `skip_dry_run`, `false` to clear it. | `true` | + +All inputs are strings; omit an input to leave the field unchanged. + +## Expected output + +`scripts/update-runner-runtime.js` updates every requirements file containing `runner_key` and prints the list of files it touched, for example: + +```text +Updated requirements.json +Updated requirements-core.json +``` + +The workflow commits these files with the message `chore: update runner runtime for ` using the repository's `GITHUB_TOKEN` and pushes directly to the default branch. + +If the updated values match the existing file contents, the commit step logs `No changes to commit` and nothing is pushed. + +## Common errors and troubleshooting + +| Error | Explanation | Resolution | +| ----- | ----------- | ---------- | +| `Runner not found` | No requirements file contains the supplied `runner_key`. | Verify the key against `requirements*.json` and try again. | +| `No changes to commit` | The inputs match the current values. | Provide different values or confirm the update already exists. | +| Permission denied | The workflow lacks write permission. | Ensure `GITHUB_TOKEN` has `contents: write` and that you have access to run the workflow. | + +If the workflow fails, open the job logs to inspect the `Update requirements` step for details. diff --git a/error-summary.md b/error-summary.md index 56882cfb..0bbaca3f 100644 --- a/error-summary.md +++ b/error-summary.md @@ -104,6 +104,22 @@ Error: All tests are unmapped; verify requirements mapping. ``` +### Error generating CI summary + +``` +Error: No JUnit files found + at main (/workspace/open-source/scripts/generate-ci-summary.ts:90:15) +``` + + +### Error generating CI summary + +``` +Error: All tests are unmapped; verify requirements mapping. + at main (/workspace/open-source/scripts/generate-ci-summary.ts:105:13) +``` + + ### Error generating CI summary ``` diff --git a/test-results/node-junit.xml b/test-results/node-junit.xml index 28849c3a..0d001620 100644 --- a/test-results/node-junit.xml +++ b/test-results/node-junit.xml @@ -1,61 +1,61 @@ - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + @@ -63,5 +63,5 @@ - +