Skip to content

Avoid huge search RPC #41

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Draft
rdettai-sk wants to merge 2 commits into
base: sekoia
Choose a base branch
from
Draft

Avoid huge search RPC #41

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
823a82a
feat(search): Add streaming fetch_docs RPC to avoid gRPC size limits
rdettai-sk May 7, 2026
9a154a6
Remove root RPCs from GRPC
rdettai-sk May 7, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
23 changes: 4 additions & 19 deletions quickwit/quickwit-proto/protos/quickwit/search.proto
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -17,13 +17,6 @@ syntax = "proto3";
package quickwit.search;

service SearchService {
// Root search API.
// This RPC identifies the set of splits on which the query should run on,
// and dispatch the several calls to `LeafSearch`.
//
// It is also in charge of merging back the results.
rpc RootSearch(SearchRequest) returns (SearchResponse);

// Perform a leaf search on a given set of splits.
//
// It is like a regular search except that:
Expand All @@ -37,12 +30,10 @@ service SearchService {
/// This methods takes `PartialHit`s and returns `Hit`s.
rpc FetchDocs(FetchDocsRequest) returns (FetchDocsResponse);

// Root list terms API.
// This RPC identifies the set of splits on which the query should run on,
// and dispatches the several calls to `LeafListTerms`.
//
// It is also in charge of merging back the results.
rpc RootListTerms(ListTermsRequest) returns (ListTermsResponse);
/// Streams document contents from the document store.
/// This method takes `PartialHit`s and streams back `LeafHit`s in batches
/// to avoid hitting gRPC message size limits.
rpc StreamFetchDocs(FetchDocsRequest) returns (stream FetchDocsResponse);

// Performs a leaf list terms on a given set of splits.
//
Expand All @@ -52,9 +43,6 @@ service SearchService {
// - it should be applied on the given subset of splits
rpc LeafListTerms(LeafListTermsRequest) returns (LeafListTermsResponse);

// Performs a scroll request.
rpc Scroll(ScrollRequest) returns (SearchResponse);

// gRPC request used to store a key in the local storage of the targeted node.
// This RPC is used in the mini distributed immutable KV store embedded in quickwit.
rpc PutKV(PutKVRequest) returns (PutKVResponse);
Expand All @@ -68,9 +56,6 @@ service SearchService {
rpc ListFields(ListFieldsRequest) returns (ListFieldsResponse);

rpc LeafListFields(LeafListFieldsRequest) returns (ListFieldsResponse);

// Describe how a search would be processed.
rpc SearchPlan(SearchRequest) returns (SearchPlanResponse);
}

/// Scroll Request
Expand Down
Loading
Loading