Extensions SDK testing: PipelinePolicy (out-of-tree extension)

[crstrn13]

Overview

Bootstrap end-to-end testing for the Kuadrant Extensions SDK using PipelinePolicy — an out-of-tree extension that declaratively defines action pipelines (gRPC upstreams, request-phase actions, response-phase actions) via a CRD.

PipelinePolicy is the first out-of-tree extension built with the SDK. Testing it validates both the extension's functionality and the SDK's developer experience for third-party consumers.

Scope

This epic covers:

1. Model & fixtures — Python class for PipelinePolicy, pytest fixtures for creating and managing PipelinePolicy resources, and a basic happy-path smoke test.
2. Comprehensive test cases — Deeper coverage of all action types, edge cases, error paths, targeting modes, and interactions with other Kuadrant policies.

Out of scope: Deployment and installation of the extension (CRDs, RBAC, initContainer patching). The extension is pre-installed on the test cluster via the helm chart.

CRD Spec Summary

Field	Description
targetRef	Gateway API LocalPolicyTargetReferenceWithSectionName (HTTPRoute or Gateway)
actionMethods[]	gRPC upstream definitions (name, url, service, method, messageTemplate)
request[]	Ordered request-phase actions: allow (CEL predicate) or grpc_method (call upstream)
response[]	Ordered response-phase actions: add_headers or with_response_code

Sub-issues

• PipelinePolicy model, fixtures, and basic happy-path test #975 — PipelinePolicy model, fixtures, and basic happy-path test
• Comprehensive PipelinePolicy test cases #976 — Comprehensive PipelinePolicy test cases

[SurbhiAgarwal1]

Hi @crstrn13 , This epic looks really interesting because it’s not just testing a single feature — it’s basically defining how future third-party Kuadrant extensions will be validated.

I think there’s an opportunity here to make the testing approach more reusable and less tightly coupled to PipelinePolicy specifically.

Instead of implementing extension-specific deployment logic directly inside the tests, we could introduce a small reusable Extension Test Harness abstraction that handles:

• CRD installation
• RBAC setup
• operator patching
• rollout waiting
• cleanup/restoration

Then PipelinePolicy simply becomes the first extension using that harness.

Something along the lines of:

type ExtensionInstaller interface {
    Install(ctx context.Context) error
    Cleanup(ctx context.Context) error
    WaitReady(ctx context.Context) error
}

This would help future out-of-tree SDK extensions plug into the same E2E framework with minimal extra work.

Another thing that could help long-term stability:

Since patching the operator deployment is disruptive, the tests could snapshot the original Deployment before mutation and automatically restore it during cleanup. That would make the suite more isolated and reduce CI flakiness from leftover state.

I also think it would be valuable to validate not only the PipelinePolicy functionality itself, but also the SDK integration contract:

• extension discovery
• socket registration
• reconnect behavior after operator restart
• failure isolation when an extension crashes
• invalid config handling

That would give much stronger confidence in the SDK developer experience for future third-party consumers.

For the traffic validation side, a lightweight fake gRPC upstream server inside the testsuite could make assertions much more deterministic:

• validate payload templates
• simulate delays/errors
• verify action ordering
• verify response mutations

This could make the tests both easier to debug and more production-like without depending on external services.

Overall I think this epic could evolve into a reusable “extension certification” framework for Kuadrant rather than only a PipelinePolicy E2E suite, which would be a really strong foundation for future SDK adoption.

[crstrn13]

Hello @SurbhiAgarwal1, those are some valid points! This is something we would want to address at the deployment/provisioning level instead. The testsuite itself doesn't concern itself much with provisioning and I would rather keep it that way. But thanks a lot for the comment, there are some good takeaways here. Looking forward to more of your contributions. Cheers!

[SurbhiAgarwal1]

Hi @crstrn13 ,Thanks for the clarification! That makes sense, and I understand the separation between provisioning responsibilities and the testsuite itself much better now.

I’ll continue exploring the related issues and architecture to understand the flow more deeply.

Also, since I’m still getting familiar with the Kuadrant ecosystem, if there are any issues or areas you think would be a good fit for me to start contributing to, I’d really appreciate the guidance. I’d be happy to work on something useful and gradually learn the codebase and workflow better.