Skip to content

developer-guide.md

Latest commit

 

History

History
180 lines (122 loc) · 5.66 KB

developer-guide.md

File metadata and controls

180 lines (122 loc) · 5.66 KB

Developer guide

Contributing

Any changes to the CRD API (api/v1alpha1/) must be agreed with the core team before implementation.

Development commands

make test          # Run unit/integration tests (no cluster needed)
make lint          # Run golangci-lint
make lint-fix      # Run golangci-lint with auto-fix
make fmt           # Run go fmt
make vet           # Run go vet
make build         # Build the manager binary (runs manifests, generate, fmt, vet, lint)
make manifests     # Regenerate CRD manifests and RBAC from controller-runtime markers
make generate      # Regenerate DeepCopy methods
make test-e2e      # Run end-to-end tests in a Kind cluster (creates one if needed, tears it down after)

After modifying types in api/v1alpha1/, always run make manifests generate before testing.

Run a single test package (requires make test or make setup-envtest to have been run first):

KUBEBUILDER_ASSETS="$(./bin/setup-envtest use --bin-dir ./bin -p path)" go test ./internal/controller/... -v

Filter to a specific Ginkgo spec with --ginkgo.focus:

KUBEBUILDER_ASSETS="$(./bin/setup-envtest use --bin-dir ./bin -p path)" go test ./internal/controller/... -v --ginkgo.focus "spec name"

Run a specific e2e test by label:

TEST_LABELS="<label>" make test-e2e

Prerequisites

  • Go v1.25.0+.
  • Docker or Podman.
  • kubectl v1.31+.
  • Access to a Kubernetes v1.31+ cluster.

Build and deploy from source

Build and push the operator image:

make docker-build docker-push IMG=<some-registry>/valkey-operator:tag

Install the CRDs into the cluster:

make install

Deploy the operator to the cluster:

make deploy IMG=<some-registry>/valkey-operator:tag

Create a sample ValkeyCluster:

kubectl apply -f config/samples/v1alpha1_valkeycluster.yaml

Uninstall

Delete the instances (CRs) from the cluster:

kubectl delete -f config/samples/v1alpha1_valkeycluster.yaml

Delete the CRDs from the cluster:

make uninstall

Undeploy the controller from the cluster:

⚠️ Warning: make undeploy removes all resources in the operator's namespace. Always deploy the operator in a dedicated namespace to avoid accidentally deleting unrelated workloads.

make undeploy

Build the install bundle

Generate a single YAML file containing all resources (CRDs, RBAC, deployment):

make build-installer IMG=<some-registry>/valkey-operator:tag

This produces dist/install.yaml which can be applied with kubectl apply -f.

Run the operator locally

The kubebuilder scaffolding gives a build target make run which runs the operator process locally, but towards a K8s cluster. Since neither Pod IPs are routable, nor Pod FQDNs are resolvable outside the cluster, any attempt by the operator to connect to a Valkey pod will fail.

Here is a procedure to make it work, but you might need to adapt depending on your setup.

Prerequisites

  • Linux and a distro using systemd-resolved for DNS (like Ubuntu >= 22.04, Fedora >= 36).
  • K8s cluster via minikube, with the Docker driver (default on Linux).
  • The domain name for your cluster is cluster.local.

Steps

1. Start the K8s cluster and install the operator CRD.

minikube start
make install

2. Setup local access to the services in the minikube cluster.

minikube tunnel

This creates a network route on the host for the service CIDR using the minikube IP address as a gateway. We will be able to connect to services locally. See ip route showing 10.96.0.0/12 via 192.168.49.2 dev br-ea36a389b2f9.

3. Setup local access to the pods in the minikube cluster.

for name cidr in $(kubectl get nodes -Ao jsonpath='{range .items[*]}{@.metadata.name}{" "}{@.spec.podCIDR}{"\n"}{end}'); do echo sudo ip route add $cidr via $(minikube ip -n $name); done

This adds a similar route as in the previous step, but for Pod CIDR ranges. We get the podCIDR range for each node using kubectl, then route the range to the node IP.

Now the operator running outside the K8s cluster should be able to connect to a listener using the Pod IP. Since we probably also want to access it using FQDN, like when accessing a pod in a headless service using <pod-name>.<service-name>.<namespace>.svc.cluster.local, we also need to setup the DNS.

4. Setup DNS to be able to resolve cluster.local domain names.

# Add kube-dns to the list of DNS servers
sudo resolvectl dns $(ip route | grep $(minikube ip) | awk '{print $NF}' | uniq) $(kubectl -n kube-system get svc kube-dns -o jsonpath='{.spec.clusterIP}')

# Forward request for cluster.local to the kube-dns
sudo resolvectl domain $(ip route | grep $(minikube ip) | awk '{print $NF}' | uniq) cluster.local

# Test
# Optionally run `resolvectl` to check the status.
dig kubernetes.default.svc.cluster.local

Since we want the kube-dns in the cluster to resolve all queries ending with cluster.local we need to configure our local DNS service to forward these request. We first need to get the network bridge towards minikube, using ip route | grep $(minikube ip) | awk '{print $NF}', then we also need the ServiceIP for the kube-dns service in the K8s cluster, we get this using kubectl -n kube-system get svc kube-dns -o jsonpath='{.spec.clusterIP}'. With this information we can configure the local DNS service using resolvectl.

5. Start the operator locally and create a CR to trigger the reconciler.

make run
kubectl create -f config/samples/v1alpha1_valkeycluster.yaml

The operator should now be able to connect to Valkey containers in the minikube cluster.