Skip to content

chore(api): grpc server (remove graphql)#24364

Draft
pront wants to merge 10 commits intomasterfrom
pront-grpc
Draft

chore(api): grpc server (remove graphql)#24364
pront wants to merge 10 commits intomasterfrom
pront-grpc

Conversation

@pront
Copy link
Member

@pront pront commented Dec 10, 2025

Summary

This PR migrates Vector's internal API from GraphQL to gRPC, significantly reducing code complexity while maintaining backward compatibility for vector top and vector tap commands.

Key Changes

Removed (GraphQL):

  • 7,206-line GraphQL schema JSON file
  • 24 GraphQL query/subscription files
  • ~6,000 lines of GraphQL schema code in src/api/schema/
  • GraphQL dependencies: async-graphql, async-graphql-warp
  • GraphQL client implementation in vector-api-client

Added (gRPC):

  • New protobuf schema: proto/vector/observability.proto (209 lines)
  • New gRPC service: src/api/grpc/service.rs (918 lines)
  • gRPC client using tonic/prost (existing deps) in vector-api-client
  • Updated vector-top and vector-tap to use gRPC streaming

Vector configuration

How did you test this PR?

Change Type

  • Bug fix
  • New feature
  • Non-functional (chore, refactoring, docs)
  • Performance

Is this a breaking change?

  • Yes
  • No

Does this PR include user facing changes?

  • Yes. Please add a changelog fragment based on our guidelines.
  • No. A maintainer will apply the no-changelog label to this PR.

References

Notes

  • Please read our Vector contributor resources.
  • Do not hesitate to use @vectordotdev/vector to reach out to us regarding this PR.
  • Some CI checks run only after we manually approve them.
    • We recommend adding a pre-push hook, please see this template.
    • Alternatively, we recommend running the following locally before pushing to the remote branch:
      • make fmt
      • make check-clippy (if there are failures it's possible some of them can be fixed with make clippy-fix)
      • make test
  • After a review is requested, please avoid force pushes to help us review incrementally.
    • Feel free to push as many commits as you want. They will be squashed into one before merging.
    • For example, you can run git merge origin master and git push.
  • If this PR introduces changes Vector dependencies (modifies Cargo.lock), please
    run make build-licenses to regenerate the license inventory and commit the changes (if any). More details here.

@github-actions github-actions bot added the domain: topology Anything related to Vector's topology code label Dec 10, 2025
@pront pront added the no-changelog Changes in this PR do not need user-facing explanations in the release changelog label Dec 12, 2025
@pront pront force-pushed the pront-grpc branch 7 times, most recently from a317991 to 45e8d0e Compare December 18, 2025 20:26
@pront pront force-pushed the pront-grpc branch 3 times, most recently from d3b8168 to 5cec09b Compare December 22, 2025 19:02
@pront pront force-pushed the pront-grpc branch 2 times, most recently from 216a7fc to 5b15df1 Compare January 9, 2026 23:50
pront and others added 5 commits February 25, 2026 18:03
The comment now explicitly mentions that Log.metadata_full preserves event metadata during conversion.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
Migrate the vector_api integration tests from the deprecated GraphQL API to the new gRPC API.

Changes:
- Updated test infrastructure to use gRPC client instead of GraphQL
- Migrated component queries from GraphQL to gRPC GetComponents RPC
- Replaced GraphQL subscriptions with gRPC streaming for tap functionality
- Added metrics population in gRPC GetComponents handler to support metrics tests
- Fixed config reload tests to work around Vector behavior where components with
  unchanged names but modified connections don't appear during the reload transition
- Increased startup timeout from 2s to 10s to handle slower test environments
- Added 500ms delay before polling after SIGHUP to let Vector process the signal

Test fixes:
- Updated all assertions to use gRPC proto types (ComponentsResponse, ComponentType, etc.)
- Changed tap tests to collect events inline instead of storing streams to avoid lifetime issues
- Simplified reload tests to completely replace components instead of keeping old names
- All 7 tests now pass: tap (4 tests), top (3 tests)

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
Add breaking change changelog documenting the removal of the GraphQL
Playground and replacement with gRPC API. Includes migration guide with
grpcurl examples for common operations.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
@github-actions github-actions bot added the domain: ci Anything related to Vector's CI environment label Feb 26, 2026
pront and others added 2 commits February 26, 2026 16:35
P0:
- Bind TCP listener before spawning to fail fast on port conflicts

P1:
- Restore fail-fast behavior on API startup failures
- Fix tap output format regression (JSON/YAML/logfmt)
- Remove metric batching to support <10 components
- Fix double throughput normalization

P2:
- Validate parameters before casting to unsigned types
- Report Vector uptime instead of subscription lifetime

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

domain: ci Anything related to Vector's CI environment domain: topology Anything related to Vector's topology code no-changelog Changes in this PR do not need user-facing explanations in the release changelog

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant