diff --git a/docs/cndi/README.md b/docs/cndi/README.md new file mode 100644 index 00000000..ac30afeb --- /dev/null +++ b/docs/cndi/README.md @@ -0,0 +1,259 @@ +# CNDI – Self-Hosted Infrastructure Services in minutes + +**CNDI** is a completely free and open-source framework for deploying and +managing cloud-native applications on Kubernetes in any environment. It provides +a holistic approach to cluster management, encompassing infrastructure +provisioning, application deployment, GitOps automation, and Day-2 operations. +In short, CNDI gives you the **ease-of-use of a PaaS** while you retain full +control of your stack in your own cloud with no license fees. πŸš€ + +Developers can use CNDI to quickly provision **production-ready** Kubernetes +clusters _and_ deploy popular services on them (like databases, analytics, or +CI/CD tools) using a single unified configuration file. Everything is managed as +code in a single Git repository for auditability and reproducibility. You can +even create your own custom Templates, turning CNDI into a platform engineering +powerhouse. + +If you like what you see, please check us out at +[https://cndi.dev](https://cndi.dev) and +[github.com/polyseam/cndi](https://github.com/polyseam/cndi)! + +Let’s dive into what CNDI has to offer. + +## Why CNDI Deserves Your Attention + +With complete end-to-end control of your infrastructure, CNDI gives you the ease +of use you expect from a PaaS, coupled with the control, flexibility, and cost +savings of self-hosting. + +- **🌟 Single Source of Truth** – All infra & apps in one Git repo with full + auditability. +- **🌟 Production-Ready Out-of-the-Box** – Secure, monitored, GitOps flow from + day one. +- **🌟 Flexibility & Control** – Sensible defaults and simple customization. +- **🌟 Entirely Free & Open Source** – Community maintained framework and + Templates. + +To get a sense of what CNDI and it's Templates can do for you, you can +experiment with our browser-based wizard at +[cndi.dev/configurator](https://cndi.dev/configurator)! + +## Overview & Use Cases + +**What is CNDI?** CNDI stands for **Cloud-Native Deployment Interface** and is a +cloud-native framework maintained by the Polyseam team alongside Open-Source +contributors. It automates the creation of Kubernetes-based infrastructure +platforms with unified support for every modern K8s environment. + +With CNDI you can automatically: + +- **Provision Infrastructure**: Set up cloud resources (VMs, networking, etc.) + on AWS, GCP, Azure, or even locally, all defined via code. +- **Bootstrap your Kubernetes Cluster and Components**: Install a Kubernetes + distribution on those infrastructure resources complete with essential add-ons + (ingress, cert-manager, etc.). +- **Deploy Applications via GitOps**: Integrate ArgoCD for GitOps so that your + application manifests (Helm charts, YAMLs) are continuously deployed to the + cluster from your git repo. +- **Bake in DevOps Best Practices**: Everything (infra and app configs) lives in + a single Git repository – the one source of truth. Changes require git + commits, ensuring traceability and easy rollbacks. +- **Include Observability & Security**: Out-of-the-box, CNDI clusters come with + a full observability stack (Prometheus, Grafana, Loki) and enforce security + via GitOps. It integrates cert-manager for TLS and sealed-secrets for managing + secrets. + +**Key Idea:** CNDI consolidates **Infrastructure-as-Code** and **GitOps** into +one workflow. Instead of managing Terraform for infra and Helm charts for apps +separately, you describe your desired state in one **`cndi_config.yaml`** file. +The `cndi` CLI then generates the necessary Terraform and Kubernetes manifests +for your target deployment environment for you. + +### Key Features & Benefits + +- **🟒 Open Source & Community-Driven** – Apache-2.0 license, welcomes + contributions, no proprietary lock-in. +- **πŸ”§ Full-Stack Templates** – Blueprints for entire stacks (infra + cluster + + apps) covering Airflow, Kafka, PostgreSQL, WordPress, etc. +- **πŸ“¦ Complete Framework** – Infrastructure provisioning, GitOps CI/CD (Argo + CD), monitoring, logging, secrets, ExternalDNS – all integrated. +- **πŸ”’ Security & Auditability** – Git commits for all changes, secrets managed + via `.env` and sealed-secrets, cert-manager for TLS. +- **πŸš€ Quick Interactive Setup** – CLI prompts guide you through project + creation, even without deep Terraform/K8s knowledge. +- **πŸ–₯️ Online Configurator** - You can even play with configuring Templates + online with the [cndi configurator](https://cndi.dev/configurator). +- **πŸ€– GitHub Integration** – Auto-create GitHub repo and secrets, includes a + GitHub Actions workflow for deployments (`cndi-run.yaml`) and one for checks + (`cndi_onpull.yaml`). +- **♻️ Unified Config & One-Click Updates** – Edit `cndi_config.yaml` + run + `cndi overwrite`, commit & push β†’ everything regenerates. +- **πŸ“œ GitOps by Design** – Argo CD in-cluster watches your repo and + continuously syncs state. +- **πŸ’  Ejectability & Customization** – Generated Terraform and YAMLs are in + your repo; you can extend or β€œeject” at any time. +- **🌐 Multi-Cloud & Hybrid Support** – AWS, GCP, Azure, and local `dev` mode, + all with the same CLI and Templates. +- **πŸ’° Cost Efficiency** – Avoid managed-service markups; pay only for raw cloud + resources. `cndi destroy` tears everything down when not needed. + +### Templates + +CNDI comes with a variety of **pre-built templates** for common use cases. These +Templates are designed to be production-ready and can be easily customized to +fit your needs. Each Template includes a complete stack of infrastructure, +Kubernetes cluster, and applications, all defined in a single `cndi_config.yaml` +file. This allows you to quickly spin up a fully functional environment with +minimal effort, and when you need to make changes, you can do so by simply +editing that same file and running `cndi overwrite`. + +Popular Templates: + +| Template | Use Case | +| ------------------------------------------------------- | ------------------------------- | +| [Airflow](https://cndi.dev/templates/airflow) | Data pipelines & ETL | +| [Kafka](https://cndi.dev/templates/kafka) | Event streaming | +| [PostgreSQL](https://cndi.dev/templates/postgres) | SQL databases | +| [MySQL](https://cndi.dev/templates/mysql) | SQL databases | +| [MongoDB](https://cndi.dev/templates/mongodb) | NoSQL database | +| [Redis](https://cndi.dev/templates/redis) | Cache / in-memory store | +| [WordPress](https://cndi.dev/templates/wordpress) | CMS & web apps | +| [Hop](https://cndi.dev/templates/hop) | Visual data integration | +| [GPU Operator](https://cndi.dev/templates/gpu-operator) | GPU workloads / ML | +| [Functions](https://cndi.dev/templates/fns) | Serverless functions | +| [Neo4j](https://cndi.dev/templates/neo4j) | Graph database | +| [MS SQL Server](https://cndi.dev/templates/mssqlserver) | Containerized SQL Server on K8s | + +- Don't forget **Custom Templates** are easy to create and share with the + community! + +## How CNDI Works (Architecture) + +
+

Diagram

 + +```mermaid +flowchart TD + subgraph Workflow["CNDI Workflow"] + direction TB + + subgraph Workstation["Dev Machine"] + direction TB + A["`CNDI CLI
_cndi create_
or
_cndi ow_`"] -->|generates| D + A --> |generates| B + + B["Framework Files
_cndi_config.yaml_
_.env_
_GitHub Workflows_"] + + D["Cluster Artifacts
_Terraform Resources_
+
_Kubernetes manifests_"] + D --> F["_git push_"] + B --> F + end + + subgraph GitHub["GitHub"] + direction TB + F --> C["Your GitHub Repo"] + C --> E["On Pull Request
(GitHub Actions)"] + C --> K["On Merge / Push
(GitHub Actions)"] + E --> L["Static Analysis
eg. _Checkov_
"] + K --> M["CNDI Run
_cndi run_"] + M --> |calls| Q["_terraform apply_"] + end + + subgraph Cloud["Your Cloud Infrastructure"] + direction TB + Q --> N["`Supporting Resources
(eg. VPC, IAM)`"] + + subgraph Cluster["Kubernetes Cluster"] + R["Applications and Config"] + Q --> |creates| P["ArgoCD"] + P --> |guarantees| R + P --> |tracks| C + end + end + end +``` + +
+ +## Developer Experience + +1. **Project Creation (Bootstrap)** + - `cndi create` (interactive) β†’ scaffolds a new Git repo with: + - `cndi_config.yaml` (main config) + - `.env` (secrets, not committed) + - `cndi/terraform/*` (generated Terraform) + - `cndi/cluster_manifests/*` (generated K8s manifests) + - `.github/workflows/cndi-run.yaml` (CI workflow) + - `.github/workflows/cndi_onpull.yaml` (CI checks workflow) + +2. **Overwrite (Generate Code)** + - `cndi overwrite` reads your `cndi_config.yaml` and `.env` and regenerates + Terraform and Kubernetes manifests in `cndi/`. + +3. **GitOps Pipeline (Provision & Deploy)** + - Push to GitHub β†’ Actions runs `cndi run` β†’ Terraform applies β†’ cluster & + infra up β†’ Argo CD in-cluster pulls manifests β†’ deploys apps. + +4. **Day-2 Operations** + - **Access** via domain/TLS + - **Monitor** with Prometheus, Grafana, & Loki + - **Update** by editing `cndi_config.yaml` β†’ `cndi overwrite` β†’ push β†’ + automated CI/CD + - **Customize** by adding extra Terraform or manifests in config + - **Destroy** everything with `cndi destroy` when done + +## Installation + +**Prerequisites:** + +- GitHub CLI (`gh auth login`) +- Cloud credentials (AWS, GCP, Azure) or none for local `dev` +- (Optional) domain & email for TLS + +Once you have what you need, install CNDI using the guide from the +[Project's README.md](https://cndi.run/gh?utm_source=c2cPE). + +## Quick Start Tutorial + +Deploy **Airflow** on your cloud in minutes: + +1. **Create Project** + ```bash + cndi create my-user/my-airflow-demo -t airflow + ``` + Follow prompts (cloud provider, region, nodes, domain, email…). + +2. **Open GitHub Repo** + ```bash + gh repo view my-user/my-airflow-demo --web + ``` + +3. **Watch CI**\ + In GitHub Actions, see **CNDI Run** deploy infra and cluster. + +4. **Access Airflow** + - Via domain/TLS if configured + - Or use `kubectl port-forward` + +5. **Next Steps** + - Log into Argo CD (`admin` + password from `.env`) + - Monitor via Grafana & Loki + - Scale or add apps by editing config β†’ `cndi overwrite` β†’ push + - Tear down with `cndi destroy` + +## Further Resources + +- **Official Site**: https://cndi.dev +- **GitHub Repo**: https://github.com/polyseam/cndi +- **Discord Community**: https://cndi.run/di?utm_source=c2cPE +- **Related Projects**: + - Argo CD: https://argo-cd.readthedocs.io + - Terraform: https://www.terraform.io + - GitHub Actions: https://github.com/features/actions + - Prometheus: https://prometheus.io + - Grafana: https://grafana.com + - Loki: https://grafana.com/oss/loki + - Cert-Manager: https://cert-manager.io + - Sealed Secrets: https://github.com/bitnami-labs/sealed-secrets + - Checkov: https://www.checkov.io + - Helm: https://helm.sh