OpsWarden — Ops

_{Reference cluster topology — to be replaced by OpsWarden's own diagram.}

---

About

opswarden-ops is the infrastructure & deployment repository for OpsWarden: a Kubernetes deployment on DigitalOcean (DOKS), provisioned end-to-end by Terraform, routed by Traefik, with a reproducible dev environment via Nix.

It is separate from the product (opswarden-app) and deliberately optional: OpsWarden runs with a single docker compose up. This repo is the portfolio cloud showcase — it must never be a prerequisite to run the product.

Status: this repo is derived from a proven reference deployment (a working DOKS + Traefik + Postgres/Redis stack). The reusable infrastructure manifests are in place — their images modernized (Postgres 18 / Redis 8 / Traefik v3), pending re-validation on a live cluster — while the OpsWarden application services are placeholders until their images are published. Full re-targeting once the core + AI SRE are deployable. See the per-service status below.

---

Project Structure

opswarden-ops/
│
├── k8s/
│   ├── server/             # OpsWarden server (Rust/Axum) + HPA/PDB tmpl — placeholder
│   ├── client-web/         # Next.js client (or Vercel)    — placeholder
│   ├── investigation/      # AI SRE agent (RAG/FastAPI)     — placeholder
│   ├── worker/             # async Redis workers           — placeholder
│   ├── postgres/           # PostgreSQL                     — ready
│   ├── redis/              # Redis                          — ready
│   ├── traefik/            # ingress controller & LB (+ IngressClass, PDB) — ready
│   └── observability/      # cAdvisor (+ prom/grafana/loki) — partial
│
├── terraform/              # DOKS cluster provisioning (main/outputs/providers/variables.tf)
├── scripts/                # smoke.sh, load.sh, soft-affinity.sh (ops helpers)
├── assets/                 # architecture + cluster screenshots
├── Makefile                # single runner: provision → deploy → harden → verify → destroy + fmt/validate/lint
├── flake.nix / flake.lock  # Nix dev shell (kubectl, terraform, k9s, helm…)
├── .env                    # API tokens (git-ignored)
├── LICENSE / NOTICE        # Apache-2.0
└── README.md

---

Services

Service	Tech	Status
server	Rust / Axum	placeholder — k8s/server/
client-web	Next.js	placeholder — k8s/client-web/ (or Vercel)
investigation	FastAPI (AI SRE)	placeholder — k8s/investigation/
worker	async	placeholder — k8s/worker/
PostgreSQL	postgres:18-alpine	ready — k8s/postgres/
Redis	redis:8-alpine	ready — k8s/redis/
Traefik	traefik:v3.7	ready — k8s/traefik/
cAdvisor	monitoring	ready — k8s/observability/

Replicated services use pod anti-affinity to land on different nodes. Shared config lives in ConfigMaps; credentials in Secrets (rotate the placeholder values before any real deployment).

---

Reference Deployment

The reusable stack has been proven on DigitalOcean Kubernetes (DOKS) — a 2-node pool (s-2vcpu-4gb) provisioned end-to-end by Terraform in the fra1 region. The screenshots below are from that reference run; OpsWarden's own screenshots will replace the application-level ones once it is deployed.

Node pool — 2 / 2 nodes running

Worker pool detail — provisioned & tagged by Terraform

Cluster insights — CPU, load, memory, disk & I/O

Traefik — routers & services healthy, 100% success on :80 / :8080

assets/poll.png and assets/result.png show the reference workload (a voting app) that validated the cluster end-to-end. They will be replaced by OpsWarden screenshots once the application services are deployed.

---

Installation & Configuration

One-command path. The whole lifecycle is automated by the Makefile: make all (provision + deploy the ready layer), make hosts && make smoke (local DNS + end-to-end check), make harden (PDB/HPA), make destroy. Run make help for every target. No cloud account? Run the same manifests for free on a local 2-node minikube: make minikube, then make minikube-smoke. The manual steps below spell out the same flow.

Prerequisites

• Nix package manager
• A DigitalOcean account with an API token
• Git

1 — Clone & enter the environment

git clone git@github.com:opswarden-git/opswarden-ops.git && cd opswarden-ops
cp .env.example .env   # add your DigitalOcean API token
nix develop            # loads kubectl, terraform, k9s, helm…

2 — Provision the cluster

cd terraform
terraform init
terraform apply        # creates a 2-worker DOKS cluster (~5 min)
cd ..
export KUBECONFIG=$(pwd)/kubeconfig

3 — Deploy the infrastructure layer

kubectl apply -f k8s/observability/cadvisor.daemonset.yaml
kubectl apply -f k8s/postgres/
kubectl apply -f k8s/redis/
kubectl apply -f k8s/traefik/

4 — Deploy the application layer

The OpsWarden app services (server, client-web, investigation, worker) are placeholders for now. Once their images are published, fill the manifests in k8s/<service>/ and apply them:

# kubectl apply -f k8s/server/
# kubectl apply -f k8s/client-web/
# kubectl apply -f k8s/investigation/
# kubectl apply -f k8s/worker/

Teardown

cd terraform && terraform destroy   # or: make destroy

---

Production hardening

Reusable patterns ported from the reference deployment, applied with make harden:

• Disruption budgets — k8s/traefik/traefik.pdb.yaml keeps ≥1 Traefik replica during node drain; k8s/server/server.pdb.yaml is the template for the API server.
• Autoscaling — k8s/server/server.hpa.yaml (CPU-based HPA template), enabled by requests.cpu + metrics-server (make metrics / make load).
• Modern ingress — k8s/traefik/traefik.ingressclass.yaml replaces the deprecated kubernetes.io/ingress.class annotation; app Ingresses use spec.ingressClassName: traefik.
• Strict HA — replicated services use required pod anti-affinity (one replica per node). On clusters where nodes < replicas, scripts/soft-affinity.sh (make soft-affinity) relaxes it to preferred without editing the manifests.
• Smoke & load — scripts/smoke.sh checks the public path (Traefik + app routes, NixOS/minikube aware) and scripts/load.sh drives autoscaling.

HPA/PDB for the app tier (server, client-web) are templates until those images are deployed — make harden applies what is ready (Traefik) and skips the rest with a notice.

---

License

OpsWarden is distributed under the Apache License 2.0. See LICENSE.