Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
85 commits
Select commit Hold shift + click to select a range
8535c51
otlp: add client configuration
nickytd Nov 7, 2025
b720ddf
client: introduce option function closures
nickytd Nov 15, 2025
1b4b9c6
test: clean up plugin test
nickytd Nov 15, 2025
b8e2a9d
plugin: rename plugin from vali to logging
nickytd Nov 16, 2025
5bfd242
curator: remove vali curator component
nickytd Nov 17, 2025
d0b9b2f
telegraf: remove building vali telegraf sidecar
nickytd Nov 17, 2025
4a10f1c
remove: vali client
nickytd Nov 18, 2025
c0ab61a
metrics: adapt current set of metrics
nickytd Nov 19, 2025
71d9d3f
client: simplify dque buffer handling for the output clients
nickytd Nov 19, 2025
fd4fb5a
controller: reimplement controller client tests with noopclient
nickytd Nov 19, 2025
a59ad1e
plugin: add test suite for logging plugin
nickytd Nov 19, 2025
88046a9
plugin: add integration test
nickytd Nov 20, 2025
d2461a3
log: introduce logr.Logger interface with slog as the logging package…
nickytd Nov 20, 2025
0241f2a
build: strip vali from container image registry
nickytd Nov 20, 2025
ec1ed79
config: use string representation in target type
nickytd Nov 20, 2025
58a6e13
test: disable client buffer for tests
nickytd Nov 20, 2025
f5dee96
client: set buffer client decorator when enabled
nickytd Nov 20, 2025
f6b86fe
logs: use texh handler in debug level logging
nickytd Nov 20, 2025
cdd2f8a
client: discard any output in noopclient
nickytd Nov 20, 2025
22c58da
test: add dque buffer to plugin test
nickytd Nov 20, 2025
07b9c8a
client: add dque metric and sync
nickytd Nov 24, 2025
8ff52ad
example: support single namespace performance test load
nickytd Nov 24, 2025
c6c99eb
build: update plugin build gha
nickytd Nov 24, 2025
96ce0f6
client: add timout to dque client on close
nickytd Nov 25, 2025
0f72c3a
metrics: add clients total count metric per type
nickytd Nov 25, 2025
d3d5b11
with an OutputRecord
nickytd Nov 27, 2025
f7dfaf0
client: add stdout client
nickytd Nov 27, 2025
d38bc69
config: refactor shoot states
nickytd Nov 28, 2025
f01a768
plugin: refactor output_plugin to improve readibility
nickytd Nov 28, 2025
518dbd9
module: add v1 suffix to module name
nickytd Nov 28, 2025
2974f77
client: with otlp clients
nickytd Nov 28, 2025
54a6ef9
project: update license headers
nickytd Nov 28, 2025
0f0ab71
module: bump up dev version
nickytd Nov 30, 2025
545c327
client: set default attributes to otlp logs
nickytd Nov 30, 2025
fba62f1
config: sanitize config values
nickytd Nov 30, 2025
b31b041
client: rename clients
nickytd Nov 30, 2025
be4a583
client: add singleton metric provider test
nickytd Nov 30, 2025
b323e67
e2e/test: cleanp and prepare test description
nickytd Nov 30, 2025
1d876b0
lint: clean up
nickytd Nov 30, 2025
f91baa5
module: update module dependencies
nickytd Nov 30, 2025
e1f0e4d
module: support basic cluster scenario
nickytd Dec 1, 2025
c1cbdca
example: redesign performance test
nickytd Dec 2, 2025
ca0162d
config: normalize configuration keys to lowercase
nickytd Dec 2, 2025
efcdc86
client: swith log message type on closing dque
nickytd Dec 2, 2025
0aba96e
feat(performance-test): update Makefile to use vlogscli for log inges…
nickytd Dec 2, 2025
86d77ef
example: redesign chart values
nickytd Dec 2, 2025
b560dd8
client: remove OutputRecord type alias
nickytd Dec 2, 2025
cfdd276
client: add batch processor options
nickytd Dec 2, 2025
a8ab4e9
example: rework fetching scripts to use standard curl
nickytd Dec 3, 2025
064b2cb
performance: tune performance settings
nickytd Dec 3, 2025
a0129b5
performance: tune performance settings
nickytd Dec 3, 2025
3298314
example: add dashboards for otel-collector and vl
nickytd Dec 3, 2025
de0b19b
performance: update dashboards
nickytd Dec 4, 2025
64b1dea
client: add metric namespace to rpc metrics
nickytd Dec 4, 2025
97cdc22
client: add context propagation from plugin to clients
nickytd Dec 4, 2025
530047e
example: performance test with throtteling based on timeout
nickytd Dec 5, 2025
ae19aba
client: with rate limiter in otlp clients
nickytd Dec 5, 2025
822cbf2
example: remove dque in performance test
nickytd Dec 5, 2025
aac0e78
example: add throttle metrics and fix configuration
nickytd Dec 5, 2025
0e0cf6b
example: adding fb filesystem storage type
nickytd Dec 8, 2025
e68fd4a
test: 100c/5000l/5p/50ms
nickytd Dec 14, 2025
a552919
test: 100c/5000l/5p/25ms
nickytd Dec 16, 2025
f830f27
With DQueBatchProcessor (#405)
nickytd Dec 18, 2025
aca30f0
modules: update dependencies
nickytd Dec 18, 2025
13af670
test: 100c/10000l/10p/25ms-dque
nickytd Dec 18, 2025
3ec514b
client: with retry in emit
nickytd Dec 18, 2025
b22e146
performance: update dashboards
nickytd Dec 22, 2025
a2e790c
client: add documentation for the supported clients
nickytd Dec 22, 2025
977171b
example: add systemd logs
nickytd Dec 22, 2025
64d708e
docs: adapt project documentation
nickytd Dec 23, 2025
9c6a42b
clients: fix flaky test
nickytd Dec 23, 2025
42c7cb3
test: add e2e kind test setup
nickytd Dec 24, 2025
7a7215b
test: add fetcher to e2e kind test setup
nickytd Dec 25, 2025
40f3433
test: use log-fetcher in systemd e2e test
nickytd Dec 25, 2025
fe71346
test: enable e2e test in project makefile
nickytd Dec 25, 2025
4f1fada
linter: apply linter feedback
nickytd Dec 25, 2025
71c8ec8
test: create shoots test env
nickytd Jan 2, 2026
0ab4911
test: adjust plugin pod spec tolerations and clusterrole
nickytd Jan 4, 2026
7ed9b7d
test: add seed and shoot logs checks
nickytd Jan 4, 2026
f4130ee
test: simplify backend queries
nickytd Jan 4, 2026
d45d278
test: add event-logger test
nickytd Jan 5, 2026
929e707
test: apply linter feedback
nickytd Jan 5, 2026
b535f64
test: update e2e README.md
nickytd Jan 5, 2026
139fa4c
Set focused review mode to hyperspace
nickytd Jan 10, 2026
4b45a22
Enhance focused review mode
nickytd Jan 10, 2026
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
31 changes: 2 additions & 29 deletions .github/workflows/build.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -33,9 +33,9 @@ jobs:
strategy:
matrix:
args:
- name: fluent-bit-to-vali
- name: fluent-bit-plugin
target: fluent-bit-plugin
oci-repository: gardener/fluent-bit-to-vali
oci-repository: gardener/fluent-bit-plugin
ocm-labels:
name: gardener.cloud/cve-categorisation
value:
Expand All @@ -46,33 +46,6 @@ jobs:
integrity_requirement: none
availability_requirement: none
comment: no data is stored of processed by the installer
- name: vali-curator
target: curator
oci-repository: gardener/vali-curator
ocm-labels:
name: gardener.cloud/cve-categorisation
value:
network_exposure: private
authentication_enforced: false
user_interaction: gardener-operator
confidentiality_requirement: none
integrity_requirement: high
availability_requirement: low
- name: telegraf-iptables
target: telegraf
oci-repository: gardener/telegraf-iptables
ocm-labels:
name: gardener.cloud/cve-categorisation
value:
network_exposure: private
authentication_enforced: false
user_interaction: gardener-operator
confidentiality_requirement: none
integrity_requirement: none
availability_requirement: none
comment: >-
telegraf is not accessible from outside the seed-cluster and does not
interact with confidential data
- name: event-logger
target: event-logger
oci-repository: gardener/event-logger
Expand Down
2 changes: 2 additions & 0 deletions .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -13,3 +13,5 @@ bin
kubeconfig
*~
gosec-report.sarif
fluent-bit-output-plugin

2 changes: 1 addition & 1 deletion .golangci.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -47,4 +47,4 @@ formatters:
sections:
- standard
- default
- prefix(github.com/gardener/logging)
- prefix(github.com/gardener/logging/v1)
2 changes: 1 addition & 1 deletion .hyperspace/pull_request_bot.json
Original file line number Diff line number Diff line change
Expand Up @@ -11,7 +11,7 @@
},
"review": {
"auto_generate_review": false,
"use_custom_review_focus": false
"use_custom_review_focus": true
}
}
}
94 changes: 94 additions & 0 deletions .hyperspace/pull_request_bot_review_focus.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,94 @@
# Review Philosophy

- Comment only when there is **high confidence (>80%)** that an issue, risk, or meaningful improvement exists. Avoid speculative or low-impact feedback.
- Prioritize **signal over noise**. If a comment does not clearly improve correctness, readability, performance, security, or maintainability, do not leave it.
- Be **concise and direct**. Prefer a single, well-phrased sentence per comment whenever possible.
- Focus on **actionable feedback**. Each comment should either explain *what* is wrong, *why* it matters, or *how* to improve it.
- Avoid restating what the code already does. Assume the author can read the code.
- When reviewing text or documentation:
- Comment only if the wording is **genuinely ambiguous, misleading, or likely to cause incorrect usage**.
- Do not suggest stylistic or subjective wording changes unless they materially improve clarity or prevent misunderstanding.
- Treat every review as if the code will be **maintained by someone else six months from now**.

## Priority Areas (Review These First)

Focus review effort on the areas below, in order of **risk and long-term impact**.
Deprioritize minor style or preference-based issues unless they materially affect maintainability.

---

### Security & Safety

- Unsafe code blocks **without clear justification, scope, or documented invariants**.
- Command injection risks involving shell execution, dynamic commands, or unsanitized user input.
- Path traversal vulnerabilities when handling file paths, URLs, or external input.
- Credential exposure, hardcoded secrets, tokens, API keys, or sensitive configuration values.
- Missing or insufficient input validation on **external or untrusted data sources**.
- Improper error handling that could **leak sensitive information** through logs, error messages, or responses.
- Security-sensitive behavior that is implicit, undocumented, or relies on assumptions not enforced in code.

---

### Correctness Issues

- Logic errors that could lead to panics, crashes, undefined behavior, or incorrect results.
- Race conditions, shared-state issues, or unsafe access patterns in concurrent or async code.
- Resource leaks involving files, network connections, locks, or memory.
- Boundary issues such as off-by-one errors, empty states, or unhandled edge cases.
- Incorrect error propagation
- Optional types used where a value is guaranteed or required, adding unnecessary complexity.
- Error context that does not meaningfully improve debuggability or understanding.
- Overly defensive code that adds checks without realistic failure modes.
- Comments that restate obvious behavior instead of explaining **why** something exists.

---

### Architecture & Patterns

- Code that violates established patterns, conventions, or architectural decisions in the codebase.
- Missing or inconsistent error handling where a standard approach is already used
- Misuse of async/await, including blocking operations inside async contexts.
- Improper or incomplete trait implementations that break expectations or contracts.
- Abstractions that increase complexity without reducing duplication or improving clarity.
- Public APIs that expose unnecessary surface area or leak internal implementation details.

## Skip These (Low Value)

Do **not** leave review comments for the following, unless they directly impact
correctness, security, or long-term maintainability:

- Style or formatting concerns handled by automated tools (`go fmt`, Prettier).
- Minor naming preferences that do not materially improve clarity or correctness.
- Suggestions to add comments when the code is already self-explanatory.
- Refactoring proposals unless they fix a real bug, remove duplicated logic, or significantly reduce complexity.
- Logging suggestions unless they are required for **security, auditing, or critical observability gaps**.
- Pedantic wording or text accuracy nitpicks unless misunderstanding could lead to incorrect usage or bugs.

When in doubt, **err on the side of silence**.

## Response Format

Use the following structure for every review comment.
Do not deviate unless brevity clearly improves clarity.

1. **State the problem**
- One clear sentence describing the concrete issue.
- Avoid speculation or vague phrasing.

2. **Why it matters** (optional)
- One sentence explaining impact (correctness, safety, maintainability, or developer experience).
- Omit this step if the impact is obvious.

3. **Suggested fix**
- Provide a specific action, code snippet, or alternative approach.
- Prefer minimal, localized changes over broad refactors.

## When to Stay Silent

- If you are **not confident** that something is an actual issue, do not comment.
- Do not speculate or ask hypothetical questions disguised as feedback.
- Silence is preferred over low-confidence, low-impact, or opinion-based comments.
- If an issue depends on missing context and cannot be verified from the diff, assume the author has context and stay silent.
- Only break silence when uncertainty itself creates a **real risk** (e.g., potential security, data loss, or correctness issues).

Default to restraint. A good review is measured by **impact**, not comment count.
64 changes: 1 addition & 63 deletions Dockerfile
Original file line number Diff line number Diff line change
Expand Up @@ -5,7 +5,7 @@ WORKDIR /go/src/github.com/gardener/logging

COPY . .
RUN go mod download
RUN make plugin copy curator event-logger
RUN make plugin copy event-logger

############# distroless-static
FROM gcr.io/distroless/static-debian12:nonroot AS distroless-static
Expand All @@ -29,16 +29,6 @@ WORKDIR /

CMD ["-e", "/fluent-bit/plugins/output_plugin.so", "-c", "/fluent-bit/config/fluent-bit.conf"]

############# curator #############
FROM distroless-static AS curator

COPY --from=builder /go/src/github.com/gardener/logging/build/curator /curator

WORKDIR /
EXPOSE 2718

ENTRYPOINT [ "/curator" ]

############# eventlogger #############
FROM distroless-static AS event-logger

Expand All @@ -48,58 +38,6 @@ WORKDIR /

ENTRYPOINT [ "/event-logger" ]

############# telegraf-builder #############
FROM golang:1.25.5 AS telegraf-builder
RUN git clone --depth 1 --branch v1.26.0 https://github.com/influxdata/telegraf.git
WORKDIR /go/telegraf
ARG TARGETOS
ARG TARGETARCH
RUN --mount=type=cache,target="/root/.cache/go-build" CGO_ENABLED=0 GOOS=${TARGETOS} GOARCH=${TARGETARCH} make build

############# iptables-builder #############
FROM alpine:3.23.2 AS iptables-builder

RUN apk add --update bash sudo iptables ncurses-libs libmnl && \
rm -rf /var/cache/apk/*

WORKDIR /volume

RUN mkdir -p ./bin ./sbin ./lib ./usr/bin ./usr/sbin ./usr/lib ./usr/lib/xtables ./usr/lib/bash ./tmp ./run ./etc/bash ./etc/openvpn ./usr/lib/openvpn/plugins ./etc/iproute2 ./etc/terminfo ./etc/logrotate.d ./etc/network/if-up.d ./usr/share/udhcpc ./etc/ssl/misc ./usr/lib/engines-1.1 ./run ./usr/lib/sudo \
&& cp -d /lib/ld-musl-* ./lib && echo "package musl" \
&& cp -d /lib/libc.musl-* ./lib && echo "package musl" \
&& cp -d -r /etc/terminfo/* ./etc/terminfo && echo "package ncurses-terminfo-base" \
&& cp -d /usr/lib/libformw.so.* ./usr/lib && echo "package ncurses-libs" \
&& cp -d /usr/lib/libmenuw.so.* ./usr/lib && echo "package ncurses-libs" \
&& cp -d /usr/lib/libncursesw.so.* ./usr/lib && echo "package ncurses-libs" \
&& cp -d /usr/lib/libpanelw.so.* ./usr/lib && echo "package ncurses-libs" \
&& cp -d /usr/lib/libreadline.so.* ./usr/lib && echo "package readline" \
&& cp -d /etc/inputrc ./etc && echo "package readline" \
&& cp -d /bin/bash ./bin && echo "package bash" \
&& cp -d /etc/bash/bashrc ./etc/bash && echo "package bash" \
&& cp -d /usr/lib/bash/* ./usr/lib/bash && echo "package bash" \
&& cp -d /usr/lib/libz.* ./lib && echo "package zlib" \
&& cp -d /usr/lib/libmnl.* ./usr/lib && echo "package libmnl" \
&& cp -d /usr/lib/libnftnl* ./usr/lib && echo "package libnftnl" \
&& cp -d /etc/ethertypes ./etc && echo "package iptables" \
&& cp -d /usr/sbin/iptables* ./sbin && echo "package iptables" \
&& cp -d /usr/sbin/xtables* ./sbin && echo "package iptables" \
&& cp -d /usr/lib/libxtables* ./usr/lib && echo "package iptables" \
&& cp -d /usr/lib/xtables/* ./usr/lib/xtables && echo "package iptables" \
&& cp -d /usr/lib/sudo/* ./usr/lib/sudo && echo "package sudo" \
&& cp -d /etc/sudoers ./etc && echo "package sudo" \
&& cp -d /etc/passwd ./etc && echo "package sudo" \
&& cp -d /usr/bin/sudo ./usr/sbin && echo "package sudo" \
&& touch ./run/xtables.lock && echo "create /run/xtables.lock"

############# telegraf #############
FROM scratch AS telegraf

COPY --from=iptables-builder /volume /

COPY --from=telegraf-builder /go/telegraf/telegraf /usr/bin/telegraf

CMD [ "/usr/bin/telegraf"]

############# tune2fs-builder #############
FROM alpine:3.23.2 AS tune2fs-builder

Expand Down
65 changes: 8 additions & 57 deletions Makefile
Original file line number Diff line number Diff line change
Expand Up @@ -5,10 +5,8 @@
REPO_ROOT := $(shell dirname $(realpath $(lastword $(MAKEFILE_LIST))))
VERSION := $(shell cat VERSION)
REGISTRY ?= europe-docker.pkg.dev/gardener-project/snapshots/gardener
FLUENT_BIT_TO_VALI_IMAGE_REPOSITORY := $(REGISTRY)/fluent-bit-to-vali
FLUENT_BIT_VALI_IMAGE_REPOSITORY := $(REGISTRY)/fluent-bit-vali
VALI_CURATOR_IMAGE_REPOSITORY := $(REGISTRY)/vali-curator
TELEGRAF_IMAGE_REPOSITORY := $(REGISTRY)/telegraf-iptables
FLUENT_BIT_PLUGIN_IMAGE_REPOSITORY := $(REGISTRY)/fluent-bit-plugin
FLUENT_BIT_OUTPUT_IMAGE_REPOSITORY := $(REGISTRY)/fluent-bit-output
TUNE2FS_IMAGE_REPOSITORY := $(REGISTRY)/tune2fs
EVENT_LOGGER_IMAGE_REPOSITORY := $(REGISTRY)/event-logger
EFFECTIVE_VERSION := $(VERSION)-$(shell git rev-parse --short HEAD)
Expand All @@ -30,7 +28,7 @@ include hack/tools.mk
export PATH := $(abspath $(TOOLS_DIR)):$(PATH)

.DEFAULT_GOAL := all
all: verify plugin curator event-logger
all: tidy fmt gci plugin event-logger lint

#################################################################
# Build targets #
Expand All @@ -46,18 +44,6 @@ plugin: tidy
-ldflags="$(LD_FLAGS)" \
./cmd/fluent-bit-output-plugin

.PHONY: curator
curator: tidy
@echo "building $@ for $(BUILD_PLATFORM)/$(BUILD_ARCH)"
@GOOS=$(BUILD_PLATFORM) \
GOARCH=$(BUILD_ARCH) \
CGO_ENABLED=0 \
GO111MODULE=on \
go build \
-o $(REPO_ROOT)/build/curator \
-ldflags="$(LD_FLAGS)" \
./cmd/vali-curator

.PHONY: event-logger
event-logger: tidy
@echo "building $@ for $(BUILD_PLATFORM)/$(BUILD_ARCH)"
Expand Down Expand Up @@ -87,19 +73,11 @@ copy: tidy
docker-images:
@BUILD_ARCH=$(BUILD_ARCH) \
$(REPO_ROOT)/hack/docker-image-build.sh "fluent-bit-plugin" \
$(FLUENT_BIT_TO_VALI_IMAGE_REPOSITORY) $(IMAGE_TAG)
$(FLUENT_BIT_PLUGIN_IMAGE_REPOSITORY) $(IMAGE_TAG)

@BUILD_ARCH=$(BUILD_ARCH) \
$(REPO_ROOT)/hack/docker-image-build.sh "fluent-bit-output" \
$(FLUENT_BIT_VALI_IMAGE_REPOSITORY) $(IMAGE_TAG)

@BUILD_ARCH=$(BUILD_ARCH) \
$(REPO_ROOT)/hack/docker-image-build.sh "curator" \
$(VALI_CURATOR_IMAGE_REPOSITORY) $(IMAGE_TAG)

@BUILD_ARCH=$(BUILD_ARCH) \
$(REPO_ROOT)/hack/docker-image-build.sh "telegraf" \
$(TELEGRAF_IMAGE_REPOSITORY) $(IMAGE_TAG)
$(FLUENT_BIT_OUTPUT_IMAGE_REPOSITORY) $(IMAGE_TAG)

@BUILD_ARCH=$(BUILD_ARCH) \
$(REPO_ROOT)/hack/docker-image-build.sh "event-logger" \
Expand All @@ -112,13 +90,7 @@ docker-images:
.PHONY: docker-push
docker-push:
@$(REPO_ROOT)/hack/docker-image-push.sh "fluent-bit-plugin" \
$(FLUENT_BIT_TO_VALI_IMAGE_REPOSITORY) $(IMAGE_TAG)

@$(REPO_ROOT)/hack/docker-image-push.sh "curator" \
$(VALI_CURATOR_IMAGE_REPOSITORY) $(IMAGE_TAG)

@$(REPO_ROOT)/hack/docker-image-push.sh "telegraf" \
$(TELEGRAF_IMAGE_REPOSITORY) $(IMAGE_TAG)
$(FLUENT_BIT_PLUGIN_IMAGE_REPOSITORY) $(IMAGE_TAG)

@$(REPO_ROOT)/hack/docker-image-push.sh "event-logger" \
$(EVENT_LOGGER_IMAGE_REPOSITORY) $(IMAGE_TAG) $(EFFECTIVE_VERSION)
Expand All @@ -135,7 +107,7 @@ tidy:
@go mod tidy

.PHONY: check
check: tidy fmt gci lint
check: tidy fmt gci

.PHONY: fmt
fmt: tidy
Expand All @@ -150,7 +122,7 @@ gci: tidy
@go tool gci write $(GCI_OPT) $(SRC_DIRS)

.PHONY: lint
check: tidy
lint: tidy
@echo "Running lint..."
@go tool golangci-lint run \
--config=$(REPO_ROOT)/.golangci.yaml \
Expand Down Expand Up @@ -184,24 +156,3 @@ add-license-headers: tidy
.PHONY: clean
clean:
@rm -rf $(REPO_ROOT)/build

#########################################
# Tools #
#########################################
.PHONY: kind-up
kind-up: tidy $(KUBECTL)
@$(REPO_ROOT)/hack/kind-up.sh

#########################################
# skaffold pipeline scenarios #
#########################################
skaffold-%: export KUBECONFIG = $(REPO_ROOT)/example/kind/kubeconfig

.PHONY: skaffold-run
skaffold-run: $(SKAFFOLD)
@$(SKAFFOLD) run --kubeconfig=$(KUBECONFIG)

# skaffold-dev target requires that skaffold run has been run
.PHONY: skaffold-dev
skaffold-dev: $(SKAFFOLD)
@$(SKAFFOLD) dev --kubeconfig=$(KUBECONFIG)
Loading
Loading