Skip to content
Open
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
28 changes: 25 additions & 3 deletions docs/getting-started/infrastructure.md
Original file line number Diff line number Diff line change
Expand Up @@ -29,9 +29,11 @@ components of the stack are supported in each. You can choose to deploy Metaflow
platform](https://docs.outerbounds.com/engineering/deployment/azure-k8s/deployment/).
4. **Google Cloud** on [GKE as a Kubernetes
platform](https://docs.outerbounds.com/engineering/deployment/gcp-k8s/deployment/).
4. [Any **Kubernetes**
cluster](https://docs.outerbounds.com/engineering/deployment/aws-k8s/deployment/)
including on-premise deployments.
5. Any **Kubernetes** cluster including on-premise deployments. For cloud-hosted
clusters, see the [EKS](https://docs.outerbounds.com/engineering/deployment/aws-k8s/deployment/),
[AKS](https://docs.outerbounds.com/engineering/deployment/azure-k8s/deployment/), or
[GKE](https://docs.outerbounds.com/engineering/deployment/gcp-k8s/deployment/) guides.
For on-premise clusters, see the [on-premise setup notes](#on-premise-kubernetes) below.

| Layer | Component | Description | Only Local | AWS | Azure | GCP | K8s |
| ------------- | ------------------------------------------------------------------------------------------------------------ | -------------------------------------------------- | ---------- | --- | ----- | --- | --- |
Expand Down Expand Up @@ -182,4 +184,24 @@ data="/assets/infra-stack-gcp.svg"></object>
This stack incurs a typical maintenance overhead of an GKE-based Kubernetes cluster,
which shouldn't add much burden if your organization uses GKE already.

### On-premise Kubernetes

**Deploy the same stack as EKS, but replace S3 with MinIO**

Metaflow can run on a fully on-premise Kubernetes cluster (such as RKE2, k3s, or
bare-metal kubeadm). The stack is identical to the [Kubernetes-native EKS
stack](https://docs.outerbounds.com/engineering/deployment/aws-k8s/deployment/), with
one substitution: replace AWS S3 with
[MinIO](https://min.io/), an open-source, S3-compatible object store you can
self-host.

Key components for a fully on-prem setup:
- **Compute**: Kubernetes (`@kubernetes` decorator)
- **Datastore**: MinIO (configure `METAFLOW_S3_ENDPOINT_URL` to point to your MinIO instance)
- **Orchestration**: Argo Workflows (self-hosted)
- **Metadata**: Metaflow metadata service (self-hosted via Docker or Helm)

See [issue #682](https://github.com/Netflix/metaflow/issues/682) for community
discussion on on-premise deployments.