Skip to content

log-ingestor: Group ingestion job identifier fields when detecting key prefix conflicts. #1805

@LinZhihao-723

Description

@LinZhihao-723

Request

The current log-ingestor verifies whether the key-prefix has conflict by checking against several ingestion job config fields explicitly (dataset, bucket name, etc).
However, this is not scalable:

  • Validating things field by field is too verbose.
  • The similar identifier (fields) will be used later when we need to share the buffer.

Possible implementation

We should create a struct to hold all identifier fields and implement Eq method for it.

Metadata

Metadata

Assignees

Labels

enhancementNew feature or request

Type

No type
No fields configured for issues without a type.

Projects

No projects

Relationships

None yet

Development

No branches or pull requests

Issue actions