Feat : Add consumer API endpoint audit

[PragalvaXFREZ]

What this PR does

Adds a consumer audit pipeline (go run ./cmd/consumer-audit) that answers: for every endpoint defined in meshery/schemas, is it implemented by meshery/meshery and/or meshery-cloud, and does the implementation actually use the shared schema types?

The audit produces a structured report — printable to the terminal or reconciled into a Google Sheet — with one row per endpoint and the following columns:

Column	Meaning
Endpoint Status	Whether the endpoint is active, unimplemented, or missing in each consumer
x-annotated	Derived from x-internal: None / Meshery / Cloud only
Schema-Backed (Meshery/Cloud)	TRUE if the registered handler imports github.com/meshery/schemas/models/…
Schema-Driven (Meshery/Cloud)	TRUE if the handler's request/response types match the schema's payload shapes
Schema Completeness (Meshery/Cloud)	TRUE if the construct's schema defines both request and response shapes

---

Pipeline stages

1. Endpoint index (validation/endpoint_index.go)

Walks every schemas/constructs/*/api.yml, extracts all operations, and resolves their request/2xx-response schema shapes (following $refs). Reuses walkValidatedConstructSpecs from the existing validator.

2. Consumer route parsers

• validation/consumer_gorilla.go — AST parser for Meshery's Gorilla/mux router (server/router/server.go). Handles chained .Methods().Handle(), PathPrefix().Handler(), and anonymous wrappers.
• validation/consumer_echo.go — AST parser for Meshery Cloud's Echo router. Handles group prefixes, :param → {param} rewriting, Any(), and fmt.Sprintf path construction via a small constant resolver table.

3. Handler indexing (validation/consumer.go)

Walks server/handlers, server/models, server/services, server/dao in each consumer repo. For each HTTP handler function (matched by signature shape), extracts:

• Whether the file imports github.com/meshery/schemas/models/…
• The request type (from Decode/Bind call sites)
• The response type (from Encode/JSON call sites, or inferred from return types of service calls)

Multiple handlers with the same name are flagged as ambiguous rather than silently binding to the first match.

4. Matcher (validation/matcher.go)

Outer-join of schema endpoints against consumer endpoints. Three buckets: Matched, Schema-only (unimplemented), Consumer-only (no schema). Two-pass matching: exact path-parameter names first, then loose (names stripped) to surface param-naming drift in the Notes column without losing the match.

5. Audit rows and reconciliation (validation/consumer_audit.go, validation/sheets.go)

Builds one ConsumerAuditRow per endpoint, then optionally reconciles against a Google Sheet:

• Previous rows are read from the sheet; new/changed/deleted transitions are detected and timestamped (UTC).
• Deletion history is stored as a JSON ledger in column Z1, not as tombstone rows.
• User-owned columns O–Y are never touched.
• Each Sheets round-trip is capped at a 2-minute timeout.

6. CLI (cmd/consumer-audit/main.go)

go run ./cmd/consumer-audit
go run ./cmd/consumer-audit --meshery-repo=../meshery --cloud-repo=../meshery-cloud
go run ./cmd/consumer-audit --meshery-repo=../meshery --cloud-repo=../meshery-cloud \
    --sheet-id=<id> --credentials=<path>

Prints a summary table and, with --verbose, per-endpoint Schema-only and Consumer-only lists. When --sheet-id is set, also prints a diff of new/changed/deleted rows detected on this run.

7. Source tree abstraction (validation/source_tree.go)

sourceTree interface (ReadFile, Walk, Ref) lets tests inject in-memory file trees instead of real checkouts, keeping integration tests hermetic.

---

Files changed

File	Role
cmd/consumer-audit/main.go	CLI entrypoint
validation/consumer_audit.go	Orchestration, row construction, summary
validation/consumer.go	Handler indexing, type inference
validation/consumer_gorilla.go	Gorilla/mux route parser
validation/consumer_echo.go	Echo route parser
validation/endpoint_index.go	Schema endpoint + shape indexer
validation/matcher.go	Schema↔consumer outer join
validation/sheets.go	Google Sheets read/write + reconciliation
validation/source_tree.go	Filesystem abstraction for testing
validation/audit.go	Unchanged — walkValidatedConstructSpecs reused
Makefile	make consumer-audit target
.gitignore	Ignore local api-audit binary

No schema YAML files, generated Go structs, or TypeScript types are modified.

[Copilot]

Pull request overview

Adds a new api-audit pipeline under validation/ plus a cmd/api-audit CLI to index schema endpoints, parse consumer route registrations (Meshery Gorilla/mux + Meshery Cloud Echo), match them, and optionally reconcile results against a Google Sheet / prior CSV rows.

Changes:

• Implement schema endpoint indexing + schema shape extraction (request/2xx response) to drive audit rows.
• Implement consumer route parsers (Gorilla + Echo) and handler scanning to infer request/response Go types + struct fields.
• Add reconciliation + Google Sheets read/write support and a new cmd/api-audit entrypoint.

Reviewed changes

Copilot reviewed 17 out of 18 changed files in this pull request and generated 7 comments.

Show a summary per file

File	Description
validation/source_tree.go	Introduces a filesystem/in-memory abstraction for scanning consumer repos and test fixtures.
validation/sheets.go	Implements reconciliation logic and Google Sheets read/write integration for audit output.
validation/matcher.go	Adds schema↔consumer matching, classification logic, and field/type diffing utilities.
validation/matcher_test.go	Unit tests for matching, normalization, and classification behavior.
validation/gotype_index.go	Builds an index of Go structs and extracts JSON-tagged field/type shapes for verification.
validation/gotype_index_test.go	Tests for JSON tag parsing and Go type string rendering.
validation/endpoint_index.go	Walks constructs to index endpoints and derive request/response schema shapes.
validation/endpoint_index_test.go	Integration-ish tests that build an index from the real repo tree and validate invariants.
validation/consumer.go	Joins route registrations to handler files and infers request/response payload types + fields.
validation/consumer_gorilla.go	Gorilla/mux router AST parser for Meshery route registrations.
validation/consumer_gorilla_test.go	Tests for Gorilla parser patterns (middleware chains, PathPrefix, anonymous wrappers, sorting).
validation/consumer_field_extract_test.go	Tests for extracting struct fields from handler-local and schema-imported types.
validation/consumer_echo.go	Echo router AST parser for Meshery Cloud route registrations (including fmt.Sprintf + const resolution).
validation/consumer_echo_test.go	Tests for Echo parser patterns (group prefixes, param normalization, WrapHandler, multi-file parsing).
validation/apiaudit.go	Orchestrates the end-to-end audit: index schemas, parse consumers, match, build rows, summarize.
validation/apiaudit_test.go	Verifies summary accounting for the “Partial” bucket.
cmd/api-audit/main.go	Adds a runnable CLI (go run ./cmd/api-audit) to execute the audit and print results.
.gitignore	Ignores a locally built api-audit binary.

[gemini-code-assist]

Code Review

This pull request introduces an API audit pipeline to validate OpenAPI schemas against handler implementations in the Meshery and Meshery Cloud repositories. The implementation features AST-based route parsing, field-level verification, and Google Sheets integration for reconciliation. Feedback identifies a critical compilation error due to missing variable definitions and a high-severity risk of handler name collisions in the indexing logic. Additionally, recommendations were made to optimize performance by reducing redundant file I/O and loop complexity, and to improve testability and scalability by avoiding non-deterministic time calls and hardcoded spreadsheet ranges.

[PragalvaXFREZ]

/gemini review

[Copilot]

Pull request overview

Copilot reviewed 12 out of 13 changed files in this pull request and generated 4 comments.

[gemini-code-assist]

Code Review

This pull request introduces a consumer audit tool designed to reconcile OpenAPI schemas with their actual implementations in the Meshery and Meshery Cloud repositories. The implementation includes a new CLI, Makefile targets, and a validation package that performs AST-based parsing of Go source code to identify route registrations and handler payload types. Feedback highlights several areas for improvement, including refining the heuristics for identifying request and response types, respecting Go's block scoping during variable collection, and enhancing the robustness of the router parsers. Additionally, concerns were raised regarding the non-atomic nature of Google Sheets updates and the use of hardcoded cell ranges, which could impact data integrity and scalability.

[PragalvaXFREZ]

/gemini review

[PragalvaXFREZ]

/gemini review

[gemini-code-assist]

Code Review

This pull request introduces a comprehensive consumer audit tool designed to reconcile OpenAPI schemas with actual handler implementations in the Meshery and Meshery Cloud repositories. The implementation includes AST-based Go code parsing for Gorilla and Echo routers, an endpoint indexing system, and Google Sheets integration for tracking coverage and implementation drift over time. The review feedback suggests refactoring hardcoded status strings into exported constants to improve maintainability and ensure consistency between the validation logic and the CLI output.

[Copilot]

Pull request overview

Copilot reviewed 12 out of 14 changed files in this pull request and generated 4 comments.

[PragalvaXFREZ]

/gemini review

[Copilot]

Pull request overview

Copilot reviewed 12 out of 15 changed files in this pull request and generated 7 comments.

[Copilot]

printAuditReport prints schema-backed and schema-completeness counts as 0 when a consumer repo wasn’t provided (because the tallies remain at their zero values). This is misleading in the report output; those cells should display "-" (like the other rows) when meshery/cloud scanning is disabled. Consider using the existing cell() helper for these rows as well.

Suggested change

	t.AddRow("Schema Backed", "-", s.Meshery.BackedTrue, s.Cloud.BackedTrue)
	t.AddRow("Schema Completeness (TRUE)", "-", s.Meshery.CompletenessTrue, s.Cloud.CompletenessTrue)
	t.AddRow("Schema Backed", "-",
	cell(s.Meshery.BackedTrue, s.MesheryEndpoints > 0),
	cell(s.Cloud.BackedTrue, s.CloudEndpoints > 0))
	t.AddRow("Schema Completeness (TRUE)", "-",
	cell(s.Meshery.CompletenessTrue, s.MesheryEndpoints > 0),
	cell(s.Cloud.CompletenessTrue, s.CloudEndpoints > 0))

[Copilot]

schemaEndpoint.Deprecated is documented as “operation-level OR construct-level”, but construct-level deprecated specs are filtered out entirely by walkValidatedConstructSpecs (deprecated constructs are skipped), so this field only reflects op.Deprecated for the indexed endpoints. Please update the comment to match the actual behavior to avoid confusion for consumers of the index.

Suggested change

	Deprecated bool // operation-level OR construct-level
	Deprecated bool // operation-level only for indexed endpoints

[gemini-code-assist]

Code Review

This pull request introduces a comprehensive consumer audit system designed to reconcile OpenAPI schemas with handler implementations in the Meshery and Meshery Cloud repositories. It includes a new CLI tool, Makefile targets, and a validation package that utilizes Go AST parsing to identify endpoint coverage and implementation drift, with support for Google Sheets integration for state reconciliation. The review feedback identifies critical compilation issues due to missing variable definitions (serverGeneratedFields, dbMirroredFields, and validInternalTags) and suggests several performance optimizations, such as reducing redundant file I/O and combining AST traversal passes. Additionally, minor logic redundancies and formatting inconsistencies in timestamp generation were noted.