Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
70 changes: 34 additions & 36 deletions AGENTS.md
Original file line number Diff line number Diff line change
Expand Up @@ -75,8 +75,7 @@ make db/teardown # Stop and remove PostgreSQL container

### Code Generation
```bash
make generate # Regenerate Go models from openapi/openapi.yaml
make generate-vendor # Generate using vendor dependencies (offline mode)
make generate-mocks # Generate mock implementations for testing
```

## Project Structure
Expand Down Expand Up @@ -126,25 +125,24 @@ hyperfleet-api/

## Core Components

### 1. API Specification Workflow
### 1. CRD-Driven API

The API is specified using TypeSpec, which compiles to OpenAPI, which then generates Go models:
The API is dynamically generated from Kubernetes Custom Resource Definitions (CRDs):

```
TypeSpec (.tsp files in hyperfleet-api-spec repo)
↓ tsp compile
openapi/openapi.yaml (32KB, uses $ref for DRY)
↓ make generate (openapi-generator-cli in Podman)
pkg/api/openapi/model_*.go (Go structs)
pkg/api/openapi/api/openapi.yaml (44KB, fully resolved, embedded in binary)
charts/crds/*.yaml (CRD definitions)
↓ loaded at startup
pkg/crd/registry.go (CRD registry)
↓ generates
Dynamic routes + OpenAPI spec at runtime
```

**Key Points**:
- TypeSpec definitions are maintained in a separate `hyperfleet-api-spec` repository
- `openapi/openapi.yaml` is the source of truth for this repository (generated from TypeSpec)
- `make generate` uses Podman to run openapi-generator-cli, ensuring consistent versions
- Generated code includes JSON tags, validation, and type definitions
- The fully resolved spec is embedded at compile time via `//go:embed`
- CRD definitions in `charts/crds/` define resource types (Cluster, NodePool, IDP, etc.)
- Routes and OpenAPI spec are generated dynamically at startup
- No code generation required - just modify CRD YAML files
- Local development uses `CRD_PATH` env var to load CRDs from files
- Production loads CRDs from Kubernetes API

### 2. Database Layer

Expand Down Expand Up @@ -401,8 +399,8 @@ All subcommands support these logging flags:
```bash
# Prerequisites: Go 1.24, Podman, PostgreSQL client tools

# Generate OpenAPI code (required before go mod download)
make generate
# Generate mocks for testing
make generate-mocks

# Download Go module dependencies
go mod download
Expand All @@ -419,25 +417,24 @@ make build
# Run migrations
./bin/hyperfleet-api migrate

# Start server (no authentication)
# Start server (no authentication, loads CRDs from local files)
make run-no-auth
```

### Code Generation
### CRD Configuration

When the TypeSpec specification changes:
Resource types are defined by CRDs in `charts/crds/`. To add or modify resource types:

```bash
# Regenerate Go models from openapi/openapi.yaml
make generate

# This will:
# 1. Remove pkg/api/openapi/*
# 2. Build Docker image with openapi-generator-cli
# 3. Generate model_*.go files
# 4. Copy fully resolved openapi.yaml to pkg/api/openapi/api/
# Edit CRD files in charts/crds/
# Restart the server to pick up changes
make run-no-auth
```

The `CRD_PATH` environment variable controls where CRDs are loaded from:
- `make run-no-auth` sets `CRD_PATH=$(PWD)/charts/crds` automatically
- In production, CRDs are loaded from the Kubernetes API

### Testing

**Unit Tests**:
Expand Down Expand Up @@ -712,13 +709,13 @@ The server is configured in cmd/hyperfleet/server/:

**Solution**: Always run `./bin/hyperfleet-api migrate` after pulling code or changing schemas

### 2. Using Wrong OpenAPI File
### 2. CRD Changes Not Reflected

**Problem**: There are two openapi.yaml files:
- `openapi/openapi.yaml` (32KB, source, has $ref)
- `pkg/api/openapi/api/openapi.yaml` (44KB, generated, fully resolved)
**Problem**: Changes to CRD files in `charts/crds/` aren't showing up.

**Rule**: Only edit the source file. The generated file is overwritten by `make generate`.
**Solution**: Restart the server. CRDs are loaded at startup.
- For local dev: `make run-no-auth` loads from `charts/crds/`
- For production: CRDs are loaded from Kubernetes API

### 3. Context Session Access

Expand Down Expand Up @@ -795,6 +792,7 @@ The API is designed to be stateless and horizontally scalable:
Common issues and solutions:

1. **Database connection errors**: Check `make db/setup` was run and container is running
2. **Generated code issues**: Run `make generate` to regenerate from OpenAPI spec
3. **Test failures**: Ensure PostgreSQL container is running and `OCM_ENV` is set
4. **Build errors**: Verify Go version is 1.24+ with `go version`
2. **Missing mocks**: Run `make generate-mocks` to regenerate test mocks
3. **CRDs not loading**: Ensure `CRD_PATH` is set or Kubernetes cluster is accessible
4. **Test failures**: Ensure PostgreSQL container is running and `OCM_ENV` is set
5. **Build errors**: Verify Go version is 1.24+ with `go version`
24 changes: 13 additions & 11 deletions Dockerfile
Original file line number Diff line number Diff line change
@@ -1,10 +1,12 @@
ARG BASE_IMAGE=gcr.io/distroless/static-debian12:nonroot
ARG TARGETARCH=amd64

# OpenAPI generation stage
FROM golang:1.25 AS builder
# Build stage - explicitly use amd64 for cross-compilation from x86 hosts
FROM --platform=linux/amd64 golang:1.25 AS builder

ARG GIT_SHA=unknown
ARG GIT_DIRTY=""
ARG TARGETARCH

WORKDIR /build

Expand All @@ -15,22 +17,22 @@ RUN go mod download
# Copy source code
COPY . .

# Build binary
RUN CGO_ENABLED=0 GOOS=linux make build
# Build binary for target architecture
RUN CGO_ENABLED=0 GOOS=linux GOARCH=${TARGETARCH} make build

# Runtime stage
FROM ${BASE_IMAGE}
# Runtime stage - use target architecture for the base image
ARG BASE_IMAGE
ARG TARGETARCH
FROM --platform=linux/${TARGETARCH} ${BASE_IMAGE}

WORKDIR /app

# Copy binary from builder
COPY --from=builder /build/bin/hyperfleet-api /app/hyperfleet-api

# Copy OpenAPI schema for validation (uses the source spec, not the generated one)
COPY --from=builder /build/openapi/openapi.yaml /app/openapi/openapi.yaml

# Set default schema path (can be overridden by Helm for provider-specific schemas)
ENV OPENAPI_SCHEMA_PATH=/app/openapi/openapi.yaml
# CRD definitions are now loaded from Kubernetes API at runtime
# OpenAPI schema is generated dynamically from CRDs
# For provider-specific schemas, set OPENAPI_SCHEMA_PATH to override

EXPOSE 8000

Expand Down
38 changes: 13 additions & 25 deletions Makefile
Original file line number Diff line number Diff line change
Expand Up @@ -60,9 +60,7 @@ help:
@echo "make run/docs run swagger and host the api spec"
@echo "make test run unit tests"
@echo "make test-integration run integration tests"
@echo "make generate generate openapi modules"
@echo "make generate-mocks generate mock implementations for services"
@echo "make generate-all generate all code (openapi + mocks)"
@echo "make clean delete temporary generated files"
@echo "make image build container image"
@echo "make image-push build and push container image"
Expand All @@ -71,7 +69,6 @@ help:
.PHONY: help

# Encourage consistent tool versions
OPENAPI_GENERATOR_VERSION:=5.4.0
GO_VERSION:=go1.24.

### Constants:
Expand Down Expand Up @@ -135,14 +132,14 @@ lint: $(GOLANGCI_LINT)

# Build binaries
# NOTE it may be necessary to use CGO_ENABLED=0 for backwards compatibility with centos7 if not using centos7
build: check-gopath generate-all
build: check-gopath generate-mocks
@mkdir -p bin
echo "Building version: ${build_version}"
CGO_ENABLED=$(CGO_ENABLED) GOEXPERIMENT=boringcrypto ${GO} build -ldflags="$(ldflags)" -o bin/hyperfleet-api ./cmd/hyperfleet-api
.PHONY: build

# Install
install: check-gopath generate-all
install: check-gopath generate-mocks
CGO_ENABLED=$(CGO_ENABLED) GOEXPERIMENT=boringcrypto ${GO} install -ldflags="$(ldflags)" ./cmd/hyperfleet-api
@ ${GO} version | grep -q "$(GO_VERSION)" || \
( \
Expand Down Expand Up @@ -224,46 +221,38 @@ test-integration: install secrets $(GOTESTSUM)
./test/integration
.PHONY: test-integration

# Regenerate openapi types using oapi-codegen
generate: $(OAPI_CODEGEN)
rm -rf pkg/api/openapi
mkdir -p pkg/api/openapi
$(OAPI_CODEGEN) --config openapi/oapi-codegen.yaml openapi/openapi.yaml
.PHONY: generate

# Generate mock implementations for service interfaces
generate-mocks: $(MOCKGEN)
${GO} generate ./pkg/services/...
.PHONY: generate-mocks

# Generate all code (openapi + mocks)
generate-all: generate generate-mocks
# Generate all code (mocks only - OpenAPI types are now static)
# Note: pkg/api/openapi/openapi.gen.go contains pre-generated types
# OpenAPI spec is now dynamically generated from CRDs at runtime
generate-all: generate-mocks
.PHONY: generate-all

# generate-vendor is now equivalent to generate (oapi-codegen handles dependencies)
generate-vendor: generate
.PHONY: generate-vendor

run: build
./bin/hyperfleet-api migrate
./bin/hyperfleet-api serve
CRD_PATH=$(PWD)/charts/crds ./bin/hyperfleet-api serve
.PHONY: run

run-no-auth: build
./bin/hyperfleet-api migrate
./bin/hyperfleet-api serve --enable-authz=false --enable-jwt=false
CRD_PATH=$(PWD)/charts/crds ./bin/hyperfleet-api serve --enable-authz=false --enable-jwt=false
.PHONY: run-no-auth

# Run Swagger nd host the api docs
# Run Swagger and host the api docs
# Note: With dynamic OpenAPI generation, use the /api/hyperfleet/v1/openapi.html endpoint instead
run/docs:
@echo "Please open http://localhost/"
docker run -d -p 80:8080 -e SWAGGER_JSON=/hyperfleet.yaml -v $(PWD)/openapi/hyperfleet.yaml:/hyperfleet.yaml swaggerapi/swagger-ui
@echo "OpenAPI spec is now dynamically generated from CRDs."
@echo "Start the server and visit: http://localhost:8000/api/hyperfleet/v1/openapi.html"
.PHONY: run/docs

# Delete temporary files
clean:
rm -rf \
bin \
pkg/api/openapi \
data/generated/openapi/*.json \
secrets \
.PHONY: clean
Expand Down Expand Up @@ -330,7 +319,6 @@ endif
# For older engines: use 'docker buildx build' or omit --platform
$(container_tool) build \
--platform linux/amd64 \
--build-arg BASE_IMAGE=alpine:3.21 \
--build-arg GIT_SHA=$(GIT_SHA) \
--build-arg GIT_DIRTY=$(GIT_DIRTY) \
-t quay.io/$(QUAY_USER)/$(IMAGE_NAME):$(DEV_TAG) .
Expand Down
20 changes: 10 additions & 10 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -52,26 +52,29 @@ See [PREREQUISITES.md](PREREQUISITES.md) for installation instructions.
### Installation

```bash
# 1. Generate OpenAPI code and mocks
make generate-all
# 1. Generate mocks for testing
make generate-mocks

# 2. Install dependencies
go mod download

# 3. Build binary
make build

# 4. Setup database
# 4. Initialize secrets
make secrets

# 5. Setup database
make db/setup

# 5. Run migrations
# 6. Run migrations
./bin/hyperfleet-api migrate

# 6. Start service (no auth)
# 7. Start service (no auth)
make run-no-auth
```

**Note**: Generated code is not tracked in git. You must run `make generate-all` after cloning.
**Note**: Mocks are generated from source interfaces. Run `make generate-mocks` after cloning.

### Accessing the API

Expand Down Expand Up @@ -105,7 +108,6 @@ Kubernetes clusters with provider-specific configurations, labels, and adapter-b
Groups of compute nodes within clusters.

**Main endpoints:**
- `GET /api/hyperfleet/v1/nodepools`
- `GET/POST /api/hyperfleet/v1/clusters/{cluster_id}/nodepools`
- `GET /api/hyperfleet/v1/clusters/{cluster_id}/nodepools/{nodepool_id}`
- `GET/POST /api/hyperfleet/v1/clusters/{cluster_id}/nodepools/{nodepool_id}/statuses`
Expand All @@ -131,12 +133,10 @@ curl -G http://localhost:8000/api/hyperfleet/v1/clusters \

```bash
make build # Build binary to bin/
make run-no-auth # Run without authentication
make run-no-auth # Run without authentication (loads CRDs from local files)
make test # Run unit tests
make test-integration # Run integration tests
make generate # Generate OpenAPI models
make generate-mocks # Generate test mocks
make generate-all # Generate OpenAPI models and mocks
make db/setup # Create PostgreSQL container
make image # Build container image
```
Expand Down
4 changes: 3 additions & 1 deletion charts/Chart.yaml
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
apiVersion: v2
name: hyperfleet-api
description: HyperFleet API - Cluster Lifecycle Management Service
description: HyperFleet API - Cluster Lifecycle Management Service with Custom Resource Definitions for Kubernetes-native discoverability
type: application
version: 1.0.0
appVersion: "1.0.0"
Expand All @@ -12,4 +12,6 @@ keywords:
- api
- kubernetes
- cluster-management
- crd
- custom-resource-definition
home: https://github.com/openshift-hyperfleet/hyperfleet-api
Loading