Skip to content

docs: add Protobuf support and captured document structure for Kafka capture#2820

Open
jwhartley wants to merge 2 commits intomasterfrom
docs/kafka-capture-protobuf-and-structure
Open

docs: add Protobuf support and captured document structure for Kafka capture#2820
jwhartley wants to merge 2 commits intomasterfrom
docs/kafka-capture-protobuf-and-structure

Conversation

@jwhartley
Copy link
Copy Markdown
Contributor

Summary

  • Document Protobuf as a supported message format (requires schema registry)
  • Clarify schema reference support: Protobuf imports supported, Avro/JSON refs not yet
  • Add new "Captured document structure" section explaining how key + value are merged, scalar key handling, deletion tombstones, and metadata fields
  • Fix typo: "parition" → "partition"

Context

A customer asked about Kafka capture behavior (what's captured, key handling, Protobuf support). The docs were missing several details that are implemented in the connector.

… for Kafka capture

- Add Protobuf as a supported message format (with schema registry)
- Document schema reference support (Protobuf only, not Avro/JSON)
- Add "Captured document structure" section explaining key/value merging,
  scalar keys, deletion tombstones, and metadata fields
- Fix typo: "parition" -> "partition"
@github-actions
Copy link
Copy Markdown

github-actions bot commented Mar 31, 2026

PR Preview Action v1.8.1

QR code for preview link

🚀 View preview at
https://estuary.github.io/flow/pr-preview/pr-2820/

Built to branch gh-pages at 2026-03-31 00:10 UTC.
Preview will be ready when the GitHub Pages deployment is complete.

@jwhartley jwhartley requested a review from mdibaiee March 31, 2026 00:10
@jwhartley
Copy link
Copy Markdown
Contributor Author

I'm not familiar with the source-kafka code — please review carefully for accuracy, especially the key/value merging behavior and Protobuf details.

@jwhartley jwhartley requested a review from a team March 31, 2026 00:10
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant