feat(quality): OpenAPI spec path coverage across both transports#222
Merged
Merged
Conversation
The existing generated_contracts metric only counts generated-client calls
(x.client.OpWithResponse(...)), so it is blind to services built on the raw
httpclient (notably the entire pullrequest service), under-reporting real API
coverage.
Add a standalone `-spec-coverage` mode to tools/quality-report that measures
coverage at the (HTTP method, path) level, joining the Bitbucket OpenAPI spec
against operations reached through:
- the generated typed client (restricted to the used set), and
- the hand-rolled httpclient (paths resolved statically from services source,
including helper functions, fmt.Sprintf assembly, local path vars, and
parameter-literal substitution for shared action helpers).
Writes a deterministic, committed docs/quality/spec-coverage.json with per-tag
coverage and the full list of unimplemented gaps. The artifact does not depend
on coverage profiles or live tests, so it is generated/verified independently
and gated in CI via the CI-safe unit-tests job.
Resolves #208
Codecov Report❌ Patch coverage is
Additional details and impacted files@@ Coverage Diff @@
## main #222 +/- ##
==========================================
- Coverage 83.90% 83.31% -0.60%
==========================================
Files 80 81 +1
Lines 16624 17000 +376
==========================================
+ Hits 13949 14164 +215
- Misses 1933 2080 +147
- Partials 742 756 +14
Flags with carried forward coverage won't be shown. Click here to find out more. ☔ View full report in Codecov by Sentry. 🚀 New features to boost your workflow:
|
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Adds an automated OpenAPI spec path coverage report so unimplemented API surface is visible and tracked in CI.
The existing
generated_contractsmetric only counts generated-client calls of the formx.client.OpWithResponse(...). It is blind to services built on the rawinternal/transport/httpclient— most notably the entirepullrequestservice — so it under-reports real API coverage (it implies PR endpoints are ~0% covered when create/merge/decline/reopen/approve/auto-merge/participants are all implemented).What this does
A new standalone
-spec-coveragemode intools/quality-reportmeasures coverage at the(HTTP method, path)level, joining the Bitbucket 9.4 OpenAPI spec against operations reached through both transports:internal/openapi/generated), restricted to the operations actually called frominternal/services.GetJSON/PostJSON/...), with request paths resolved statically from the services source — handling string literals,fmt.Sprintfassembly, single-return path helpers, local path variables, and parameter-literal substitution (so the sharedtransition("merge"|"decline"|"reopen")helper resolves to its real endpoints).Resolution is undercount-safe: unresolvable dynamic segments collapse to a non-matching placeholder rather than wildcard-matching, so the number is a floor, never inflated.
Deliverables
docs/quality/spec-coverage.json— committed, deterministic artifact: 112/546 (20.5%) with per-tag breakdown and the full list of unimplemented gaps (method, path, tag, summary). Does not depend on coverage profiles or live tests.quality:spec-coverage,quality:spec-coverage:update,quality:spec-coverage:verify.unit-testsjob.WithBodyparsing.docs/quality/README.mdandAGENTS.mdupdated with the regeneration workflow.Verification
go test ./tools/quality-report/— passing;gofmt/go vetclean.task quality:spec-coverage:verifypasses against the committed file.Resolves #208
https://claude.ai/code/session_012Ax4aEmr2XgmGi52Grzhk3
Generated by Claude Code