Skip to content

Add resource limits to values.yaml for running Texera on limited hardware #4291

@Ma77Ball

Description

@Ma77Ball

Problem

Currently, the default values.yaml does not specify resource requests or limits for most Texera microservices (webserver, workflow-computing-unit-manager, workflow-compiling-service, file-service, config-service, access-control-service), and the PostgreSQL and MinIO resource requests are set relatively high. This makes it difficult to run Texera on resource-constrained environments such as local development clusters (e.g., k3s on a laptop, Minikube, or small cloud VMs).

The values-development.yaml overlay already addresses this by adding conservative resource constraints, but the base values.yaml should document sensible defaults so that new users aren't surprised by unbounded resource consumption.

Proposed Changes

  1. Reduce PostgreSQL resource requests/limits — lower CPU request from 1 to 0.25 and memory from 1Gi to 256Mi (with explicit limits).
  2. Add MinIO resource limits — set memory request and limit to 256Mi.
  3. Add resource requests and limits for all Texera microservices:
Service CPU Request CPU Limit Memory Request Memory Limit
webserver 10m 1000m 256Mi 1Gi
workflow-computing-unit-manager 10m 1000m 256Mi 256Mi
workflow-compiling-service 10m 1000m 256Mi 256Mi
file-service 10m 1000m 256Mi 512Mi
config-service 10m 1000m 256Mi 256Mi
access-control-service 10m 1000m 256Mi 256Mi

Motivation

  • Prevents pods from consuming unbounded resources and starving other services on a shared node.
  • Enables Texera to run on machines with as little as ~8GB RAM.
  • Helps Kubernetes schedule pods more effectively by providing the scheduler with accurate resource expectations.
  • Provides a documented starting point that users can tune for their own hardware.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions