diff --git a/README.md b/README.md index d702701..2fd959c 100644 --- a/README.md +++ b/README.md @@ -50,20 +50,11 @@ cd NetEngine # 2. Install dependencies poetry install -# 3. Verify host prerequisites and requested world subnets -poetry run netengine doctor --spec examples/minimal.yaml - -# 4. Start local Postgres + pgmq (includes pgmq extension pre-installed) -docker compose up -d postgres - -# 5. Apply migrations -poetry run python -m netengine.utils.run_migrations - -# 6. Boot a minimal world -poetry run netengine up examples/minimal.yaml +# 3. Run guided first-time local setup +poetry run netengine setup local examples/minimal.yaml ``` -If you only want host/container checks before configuring Postgres, run `poetry run netengine doctor --skip-db`. Add `--spec examples/minimal.yaml` (or pass the spec as a positional argument) to include subnet-overlap checks for `spec.substrate.networks[*].subnet` before bootstrapping. Check status at any time: +`netengine setup local` runs the pre-Postgres host checks, starts the required compose services, waits for Postgres health, applies migrations, reruns spec-aware doctor checks, and only then calls the existing `netengine up` bootstrap path. If you only want host/container checks before configuring Postgres, run `poetry run netengine doctor --skip-db`. Add `--spec examples/minimal.yaml` (or pass the spec as a positional argument) to include subnet-overlap checks for `spec.substrate.networks[*].subnet` before bootstrapping. Check status at any time: ```bash poetry run netengine status diff --git a/docs/alpha-quickstart.md b/docs/alpha-quickstart.md index 059325d..e2e901a 100644 --- a/docs/alpha-quickstart.md +++ b/docs/alpha-quickstart.md @@ -16,15 +16,11 @@ This guide is the focused alpha operator path. The root `README.md` keeps the hi git clone https://github.com/Forebase/NetEngine.git cd NetEngine poetry install -poetry run netengine doctor --skip-db --spec examples/minimal.yaml -docker compose up -d postgres -poetry run python -m netengine.utils.run_migrations -poetry run netengine doctor --spec examples/minimal.yaml -poetry run netengine up examples/minimal.yaml +poetry run netengine setup local examples/minimal.yaml poetry run netengine status ``` -Use `poetry run netengine doctor --spec examples/minimal.yaml` before bootstrapping a world to check host prerequisites plus Docker subnet conflicts against `spec.substrate.networks[*].subnet`. Use `NETENGINE_MOCK=true` when you want to exercise orchestration and spec validation without creating Docker, DNS, PKI, or identity resources. +Use `poetry run netengine setup local examples/minimal.yaml` for the guided first-time path: it runs host checks that are safe before Postgres starts, starts compose services, waits for database health, applies migrations, performs spec-aware doctor checks, and stops before `netengine up` if required checks fail. Use `NETENGINE_MOCK=true` when you want to exercise orchestration and spec validation without creating Docker, DNS, PKI, or identity resources. ## Pre-release mock smoke test diff --git a/docs/runbook.md b/docs/runbook.md index 811c257..2924b74 100644 --- a/docs/runbook.md +++ b/docs/runbook.md @@ -41,42 +41,20 @@ That's it — all 10 phases run end-to-end and runtime state is written to This brings up the real persistence and identity layers. -### 1. Start backing services - -```bash -poetry run netengine doctor --skip-db -docker compose up -d -poetry run netengine doctor -``` - -This starts: -- `netengine_postgres` on port 5432 (with pgmq extension) -- Keycloak on port 8080 (platform identity, used in Phase 4) - -The doctor command checks Python, Docker/Compose, required ports, writable runtime paths, database connectivity, pgmq, and stale Docker/state conflicts. Wait for postgres to be healthy: - -```bash -docker compose ps # all services should show "healthy" -``` - -### 2. Apply migrations - -```bash -psql postgresql://netengine:dev_password@localhost:5432/netengine \ - -f migrations/001_initial.sql -``` - -Set the database URL for subsequent commands: +### 1. Guided setup and bootstrap ```bash export NETENGINE_DB_URL="postgresql://netengine:dev_password@localhost:5432/netengine" +poetry run netengine setup local examples/minimal.yaml ``` -### 3. Bootstrap the world +The guided setup replaces the former manual sequence of `doctor`, `docker compose up`, database waiting, migrations, another doctor run, and `netengine up`. It runs pre-Postgres host checks first, starts the required compose services, waits for `netengine_postgres` to become healthy, applies migrations, runs spec-aware doctor/database checks, and stops before bootstrapping if required checks fail. -```bash -poetry run netengine up examples/minimal.yaml -``` +The setup workflow provides remediation hints for common blockers: +- **Subnet conflicts:** remove conflicting Docker networks with `docker network rm ` or choose non-overlapping subnets in the world spec. +- **Port conflicts:** stop the process/container using the published port or change the compose/spec port. +- **Docker name conflicts:** run `netengine down`, `docker compose down`, or remove stale containers before retrying. +- **Database failures:** inspect `docker compose ps postgres` and `docker logs netengine_postgres`, verify `NETENGINE_DB_URL`, then rerun `netengine setup local examples/minimal.yaml`. This runs all phases sequentially. Each phase reports `completed successfully` when done. Runtime state is saved after each phase; the run is resumable if diff --git a/netengine/cli/main.py b/netengine/cli/main.py index 758a081..4a180d0 100644 --- a/netengine/cli/main.py +++ b/netengine/cli/main.py @@ -3,7 +3,9 @@ import asyncio import json import os +import subprocess import sys +import time from pathlib import Path from typing import Any @@ -41,6 +43,8 @@ logger = get_logger(__name__) +LOCAL_SETUP_DB_URL = "postgresql://netengine:dev_password@localhost:5432/netengine" + def _parse_set_overrides(set_values: tuple[str, ...]) -> dict[str, Any]: """Convert repeatable dotted key=value CLI overrides into a nested dictionary.""" @@ -53,9 +57,7 @@ def _parse_set_overrides(set_values: tuple[str, ...]) -> dict[str, Any]: parts = key.split(".") if any(part == "" for part in parts): - raise click.BadParameter( - "keys must be non-empty dotted paths", param_hint="--set" - ) + raise click.BadParameter("keys must be non-empty dotted paths", param_hint="--set") value = yaml.safe_load(raw_value) cursor = overrides @@ -86,9 +88,7 @@ def _load_spec_for_cli( ): """Load a spec using the same composition semantics as ``up``.""" overrides = _parse_set_overrides(set_values) - feature_state_kwargs = ( - {} if validate_feature_states else {"validate_feature_states": False} - ) + feature_state_kwargs = {} if validate_feature_states else {"validate_feature_states": False} if environment: return load_spec_with_environment( spec_file, @@ -97,9 +97,7 @@ def _load_spec_for_cli( **feature_state_kwargs, ) if overrides: - return load_spec_with_composition( - spec_file, overrides=overrides, **feature_state_kwargs - ) + return load_spec_with_composition(spec_file, overrides=overrides, **feature_state_kwargs) return load_spec(spec_file, **feature_state_kwargs) @@ -150,8 +148,7 @@ async def _run_migrations(db_url: str) -> MigrationRunResult: for migration in result.results: if migration.status == "applied": logger.info( - f"Applied migration: {migration.filename} " - f"({migration.duration_seconds:.3f}s)" + f"Applied migration: {migration.filename} " f"({migration.duration_seconds:.3f}s)" ) elif migration.status == "skipped": logger.info(f"Skipped migration: {migration.filename} (already applied)") @@ -180,16 +177,12 @@ def _print_migration_status(status: MigrationStatus) -> None: click.echo("Migration status") click.echo(f" Applied: {len(status.applied)}") for record in status.applied: - applied_at = ( - record.applied_at.isoformat() if record.applied_at else "unknown time" - ) + applied_at = record.applied_at.isoformat() if record.applied_at else "unknown time" click.echo(f" ✓ {record.version} {record.name} ({applied_at})") click.echo(f" Pending: {len(status.pending)}") for migration in status.pending: - click.echo( - f" • {migration.version} {migration.name} ({migration.path.name})" - ) + click.echo(f" • {migration.version} {migration.name} ({migration.path.name})") click.echo(f" Failed: {len(status.failed)}") for record in status.failed: @@ -220,9 +213,7 @@ def _print_migration_status(status: MigrationStatus) -> None: } -def _readiness_line( - status: DoctorStatus, name: str, detail: str, hint: str | None = None -) -> None: +def _readiness_line(status: DoctorStatus, name: str, detail: str, hint: str | None = None) -> None: """Print one readiness check line with an optional remediation hint.""" click.echo(f"[{_STATUS_LABELS[status]}] {name}: {detail}") if hint and status in {DoctorStatus.WARN, DoctorStatus.FAIL}: @@ -243,41 +234,43 @@ def _migration_readiness_results(status: MigrationStatus) -> list[DoctorCheckRes "migrations:failed", DoctorStatus.FAIL if status.failed else DoctorStatus.OK, f"{len(status.failed)} failed migration record(s)", - "Inspect netengine_schema_migrations errors and rerun migrations." - if status.failed - else None, + ( + "Inspect netengine_schema_migrations errors and rerun migrations." + if status.failed + else None + ), "migrations", ), DoctorCheckResult( "migrations:checksum-drift", DoctorStatus.FAIL if status.checksum_drifted else DoctorStatus.OK, f"{len(status.checksum_drifted)} drifted checksum(s)", - "Restore the expected migration files or reconcile database history." - if status.checksum_drifted - else None, + ( + "Restore the expected migration files or reconcile database history." + if status.checksum_drifted + else None + ), "migrations", ), DoctorCheckResult( "migrations:pgmq", - DoctorStatus.OK - if status.pgmq_available - and status.pgmq_installed - and not status.missing_queues - else DoctorStatus.FAIL, + ( + DoctorStatus.OK + if status.pgmq_available and status.pgmq_installed and not status.missing_queues + else DoctorStatus.FAIL + ), ( "pgmq available, installed, and queues present" - if status.pgmq_available - and status.pgmq_installed - and not status.missing_queues + if status.pgmq_available and status.pgmq_installed and not status.missing_queues else "pgmq unavailable/absent or queues missing" ), - "Install pgmq and run migrations to create queues." - if not ( - status.pgmq_available - and status.pgmq_installed - and not status.missing_queues - ) - else None, + ( + "Install pgmq and run migrations to create queues." + if not ( + status.pgmq_available and status.pgmq_installed and not status.missing_queues + ) + else None + ), "migrations", ), ] @@ -326,9 +319,11 @@ def _feature_state_readiness_results(spec: Any) -> list[DoctorCheckResult]: "feature-state", status, line, - "Disable this field or use a NetEngine release that supports it." - if status == DoctorStatus.FAIL - else "Review alpha/experimental behavior before relying on it in production.", + ( + "Disable this field or use a NetEngine release that supports it." + if status == DoctorStatus.FAIL + else "Review alpha/experimental behavior before relying on it in production." + ), "spec", required=status == DoctorStatus.FAIL, ) @@ -345,6 +340,95 @@ def _feature_state_readiness_results(spec: Any) -> list[DoctorCheckResult]: return results +def _required_setup_failed(results: list[DoctorCheckResult]) -> bool: + """Return True when required setup checks should stop bootstrapping.""" + return any(result.status == DoctorStatus.FAIL and result.required for result in results) + + +def _print_setup_results(results: list[DoctorCheckResult]) -> None: + for result in results: + _readiness_line(result.status, result.name, result.detail, result.hint) + + +def _setup_host_probes() -> tuple[Any, ...]: + """Doctor probes that are valid before local Postgres has been started.""" + from netengine.diagnostic import preflight as preflight + + return ( + lambda ctx: preflight._check_python(), + lambda ctx: preflight._check_python_dependencies(), + preflight._check_required_commands, + preflight._check_optional_commands, + lambda ctx: preflight._check_step_version(), + lambda ctx: preflight._check_docker_daemon(), + lambda ctx: preflight._check_compose(), + preflight._check_ports, + preflight._check_filesystem, + preflight._check_docker_conflicts, + lambda ctx: preflight._check_docker_subnet_conflicts(ctx), + ) + + +def _run_setup_host_checks( + db_url: str | None, state_file: Path, spec_subnets: tuple[str, ...] +) -> list[DoctorCheckResult]: + """Run pre-Postgres setup checks with database probes intentionally omitted.""" + ctx = build_context( + db_url, + state_file, + skip_db=True, + spec_subnets=spec_subnets, + ) + return run_preflight(ctx, probes=_setup_host_probes()) + + +def _docker_compose_up(services: tuple[str, ...] = ("postgres",)) -> None: + command = ["docker", "compose", "up", "-d", *services] + result = subprocess.run(command, text=True, capture_output=True, check=False) + if result.returncode != 0: + detail = (result.stderr or result.stdout or "docker compose failed").strip() + raise click.ClickException( + "Unable to start required compose services. " + f"Command `{' '.join(command)}` failed: {detail}\n\n" + "Remediation: ensure Docker Desktop/daemon is running, remove stale " + "containers with `netengine down` or `docker compose down`, and retry." + ) + + +def _wait_for_postgres_health(timeout_seconds: float = 120.0, interval: float = 2.0) -> None: + """Wait until the compose Postgres container reports healthy/running.""" + deadline = time.monotonic() + timeout_seconds + last_status = "unknown" + while time.monotonic() < deadline: + result = subprocess.run( + [ + "docker", + "inspect", + "--format", + "{{if .State.Health}}{{.State.Health.Status}}{{else}}{{.State.Status}}{{end}}", + "netengine_postgres", + ], + text=True, + capture_output=True, + check=False, + ) + if result.returncode == 0: + last_status = result.stdout.strip() or "unknown" + if last_status in {"healthy", "running"}: + return + else: + last_status = (result.stderr or result.stdout or "inspect failed").strip() + time.sleep(interval) + + raise click.ClickException( + f"Postgres did not become healthy within {timeout_seconds:.0f}s " + f"(last status: {last_status}).\n\n" + "Remediation: run `docker compose ps postgres` and " + "`docker logs netengine_postgres`; verify port 5432 is free and the " + "postgres_data volume is not from an incompatible database version." + ) + + @click.group() def cli() -> None: """NetEngine — spin up, reload, and tear down authority-autonomous worlds.""" @@ -412,9 +496,7 @@ async def _migrate_status(*, exit_on_unhealthy: bool) -> None: @cli.command("readiness") @click.argument("spec_file", type=click.Path(exists=True)) -@click.option( - "--db-url", default=db_url_from_env, help="PostgreSQL URL for migration checks." -) +@click.option("--db-url", default=db_url_from_env, help="PostgreSQL URL for migration checks.") @click.option( "--state-file", type=click.Path(path_type=Path), @@ -447,9 +529,7 @@ def readiness( set_values: tuple[str, ...], ) -> None: """Validate SPEC_FILE and report host, migration, and feature readiness.""" - asyncio.run( - _readiness(spec_file, db_url, state_file, skip_db, environment, set_values) - ) + asyncio.run(_readiness(spec_file, db_url, state_file, skip_db, environment, set_values)) async def _readiness( @@ -462,9 +542,7 @@ async def _readiness( ) -> None: results: list[DoctorCheckResult] = [] try: - spec = _load_spec_for_cli( - spec_file, environment=environment, set_values=set_values - ) + spec = _load_spec_for_cli(spec_file, environment=environment, set_values=set_values) except SpecLoadError as exc: _readiness_line( DoctorStatus.FAIL, @@ -498,8 +576,7 @@ async def _readiness( click.echo(f"NetEngine readiness summary for {spec.metadata.name}") counts = { - status: sum(1 for result in results if result.status == status) - for status in DoctorStatus + status: sum(1 for result in results if result.status == status) for status in DoctorStatus } click.echo( "Summary: " @@ -511,9 +588,7 @@ async def _readiness( for result in results: _readiness_line(result.status, result.name, result.detail, result.hint) - if any( - result.status == DoctorStatus.FAIL and result.required for result in results - ): + if any(result.status == DoctorStatus.FAIL and result.required for result in results): sys.exit(1) @@ -562,11 +637,7 @@ def validate( ) except SpecLoadError as exc: if output_format == "json": - click.echo( - json.dumps( - {"ok": False, "error": str(exc), "feature_states": []}, indent=2 - ) - ) + click.echo(json.dumps({"ok": False, "error": str(exc), "feature_states": []}, indent=2)) else: click.echo(f"Spec validation failed: {exc}", err=True) sys.exit(1) @@ -587,9 +658,7 @@ def validate( ) else: if unsupported: - click.echo( - "Spec validation failed: Unsupported spec features enabled:", err=True - ) + click.echo("Spec validation failed: Unsupported spec features enabled:", err=True) for item in unsupported: click.echo( f" - {item['path']} is {item['state']} in {item['stage']}: {item['reason']}", @@ -611,6 +680,130 @@ def validate( sys.exit(1) +@cli.command("setup") +@click.argument("mode", type=click.Choice(["local"])) +@click.argument("spec_file", type=click.Path(exists=True)) +@click.option( + "--db-url", default=db_url_from_env, help="PostgreSQL URL for migration and doctor checks." +) +@click.option( + "--state-file", + type=click.Path(path_type=Path), + default=get_state_file, + help="Runtime state file path for host preflight checks.", +) +@click.option( + "--env", + "environment", + help="Merge spec.{ENV}.yaml next to SPEC_FILE before validating.", +) +@click.option( + "--set", + "set_values", + multiple=True, + metavar="KEY=VALUE", + help="Override a spec value; repeat for multiple dotted keys.", +) +@click.option( + "--postgres-timeout", + default=120.0, + show_default=True, + help="Seconds to wait for Postgres health.", +) +def setup( + mode: str, + spec_file: str, + db_url: str | None, + state_file: Path, + environment: str | None, + set_values: tuple[str, ...], + postgres_timeout: float, +) -> None: + """Guided first-time setup for a local NetEngine world.""" + asyncio.run( + _setup_local(spec_file, db_url, state_file, environment, set_values, postgres_timeout) + ) + + +async def _setup_local( + spec_file: str, + db_url: str | None, + state_file: Path, + environment: str | None = None, + set_values: tuple[str, ...] = (), + postgres_timeout: float = 120.0, +) -> None: + try: + spec = _load_spec_for_cli(spec_file, environment=environment, set_values=set_values) + except SpecLoadError as exc: + raise click.ClickException( + f"spec validation failed: {exc}\n\n" + "Remediation: fix the spec validation error and rerun `netengine setup local`." + ) from exc + + spec_subnets = tuple( + str(network.subnet) + for network in spec.substrate.networks.values() + if getattr(network, "subnet", None) + ) + + click.echo(f"NetEngine local setup for {spec.metadata.name}") + click.echo("Step 1/5: host checks before Postgres starts") + host_results = _run_setup_host_checks(db_url, state_file, spec_subnets) + _print_setup_results(host_results) + if _required_setup_failed(host_results): + raise click.ClickException( + "required host checks failed; fix the remediation hints above before running `netengine up`." + ) + + click.echo("Step 2/5: starting compose services") + _docker_compose_up(("postgres",)) + + click.echo("Step 3/5: waiting for Postgres health") + _wait_for_postgres_health(timeout_seconds=postgres_timeout) + + effective_db_url = db_url or _db_url_from_env() or LOCAL_SETUP_DB_URL + + click.echo("Step 4/5: running migrations") + try: + await _run_migrations(effective_db_url) + except Exception as exc: + raise click.ClickException( + f"migrations failed: {exc}\n\n" + "Remediation: verify database credentials, inspect `docker logs netengine_postgres`, " + "then rerun `netengine migrate up`." + ) from exc + + click.echo("Step 5/5: spec-aware doctor checks") + ctx = build_context( + effective_db_url, + state_file, + skip_db=False, + spec_subnets=spec_subnets, + ) + readiness_results = [] + readiness_results.extend(run_preflight(ctx)) + readiness_results.extend(await _check_migration_readiness(effective_db_url)) + readiness_results.extend(_feature_state_readiness_results(spec)) + _print_setup_results(readiness_results) + if _required_setup_failed(readiness_results): + raise click.ClickException( + "required setup checks failed; NetEngine stopped before `netengine up`. " + "Apply the remediation hints above and rerun `netengine setup local`." + ) + + click.echo("Setup checks passed; bootstrapping world with existing `netengine up` path.") + await _up( + spec_file, + up_to=9, + mock=False, + skip_migrations=True, + allow_migration_failure=False, + environment=environment, + set_values=set_values, + ) + + @cli.command() @click.argument("spec_file", type=click.Path(exists=True)) @click.option("--up-to", default=9, help="Stop after this phase number (0-9).") @@ -680,9 +873,7 @@ async def _up( spec = _load_spec_for_cli(spec_file, environment=environment, set_values=set_values) if mock: - click.echo( - "WARNING: running in mock mode — no real infrastructure will be created." - ) + click.echo("WARNING: running in mock mode — no real infrastructure will be created.") if not skip_migrations and not mock: db_url = _db_url_from_env() @@ -726,9 +917,7 @@ def _on_skip(phase_num: int, phase_name: str) -> None: def _on_error(phase_num: int, phase_name: str, exc: Exception) -> None: click.echo( - click.style( - f" ✗ Phase {phase_num}: {phase_name} — {exc}", fg="red", bold=True - ), + click.style(f" ✗ Phase {phase_num}: {phase_name} — {exc}", fg="red", bold=True), err=True, ) @@ -842,9 +1031,7 @@ def reload(spec_file: str) -> None: is_ephemeral = old_spec.metadata.lifecycle.value == "ephemeral" click.echo("Computing diff…") - result = asyncio.run( - apply_reload(old_spec, new_spec, state, is_ephemeral=is_ephemeral) - ) + result = asyncio.run(apply_reload(old_spec, new_spec, state, is_ephemeral=is_ephemeral)) if result.immutability_violations: click.echo("Reload REJECTED — immutable fields changed:", err=True) @@ -882,17 +1069,13 @@ def status() -> None: @cli.command() -@click.option( - "--yes", is_flag=True, help="Skip confirmation prompt for ephemeral worlds." -) +@click.option("--yes", is_flag=True, help="Skip confirmation prompt for ephemeral worlds.") @click.option( "--confirm", default=None, help="For persistent worlds, type the world name exactly.", ) -@click.option( - "--dry-run", is_flag=True, help="Show what would be removed without removing it." -) +@click.option("--dry-run", is_flag=True, help="Show what would be removed without removing it.") def down(yes: bool, confirm: str | None, dry_run: bool) -> None: """Tear down the running world (containers, networks, volumes).""" asyncio.run(_down(yes, confirm, dry_run)) @@ -901,9 +1084,7 @@ def down(yes: bool, confirm: str | None, dry_run: bool) -> None: async def _down(yes: bool, confirm: str | None, dry_run: bool) -> None: state = RuntimeState.load() if state.world_spec and not dry_run: - raw_lifecycle = (state.world_spec.get("metadata") or {}).get( - "lifecycle", "ephemeral" - ) + raw_lifecycle = (state.world_spec.get("metadata") or {}).get("lifecycle", "ephemeral") world_name = (state.world_spec.get("metadata") or {}).get("name", "") if raw_lifecycle == "persistent" and confirm != world_name: raise click.ClickException( @@ -961,9 +1142,7 @@ async def _down(yes: bool, confirm: str | None, dry_run: bool) -> None: errors.append(f"{label}: {exc}") for network in client.networks.list(): - if network.name and any( - network.name.startswith(p) for p in _CONTAINER_PREFIXES - ): + if network.name and any(network.name.startswith(p) for p in _CONTAINER_PREFIXES): label = f"network:{network.name}" if dry_run: click.echo(f" would remove {label}") @@ -1095,16 +1274,12 @@ async def _diagnose(spec_file: str, as_json: bool) -> None: "related_resource": r.related_resource, "related_logs": r.related_logs, "command_to_retry": r.command_to_retry, - "elapsed_ms": round(r.elapsed_ms, 1) - if r.elapsed_ms is not None - else None, + "elapsed_ms": round(r.elapsed_ms, 1) if r.elapsed_ms is not None else None, } for r in results ] click.echo(_json.dumps(payload, indent=2)) - issues = sum( - 1 for r in results if r.status in (ProbeStatus.FAIL, ProbeStatus.WARN) - ) + issues = sum(1 for r in results if r.status in (ProbeStatus.FAIL, ProbeStatus.WARN)) if issues: sys.exit(1) return @@ -1153,12 +1328,8 @@ async def _diagnose(spec_file: str, as_json: bool) -> None: @cli.command() -@click.option( - "--interval", default=30, type=int, help="Poll interval in seconds (default 30)." -) -@click.option( - "--max-retries", default=3, type=int, help="Max self-heal retries per phase." -) +@click.option("--interval", default=30, type=int, help="Poll interval in seconds (default 30).") +@click.option("--max-retries", default=3, type=int, help="Max self-heal retries per phase.") @click.option("--no-auto-heal", is_flag=True, help="Detect drift but don't auto-heal.") def drift_watch(interval: int, max_retries: int, no_auto_heal: bool) -> None: """Watch running world for drift and optionally auto-heal (Ctrl+C to stop).""" @@ -1171,9 +1342,7 @@ async def _drift_watch(interval: int, max_retries: int, no_auto_heal: bool) -> N click.echo("No running world found — use `netengine up` first.", err=True) sys.exit(1) - click.echo( - f"Starting drift detection (interval={interval}s, auto-heal={not no_auto_heal})…" - ) + click.echo(f"Starting drift detection (interval={interval}s, auto-heal={not no_auto_heal})…") click.echo("Press Ctrl+C to stop.\n") orchestrator = Orchestrator(state.world_spec, mock_mode=False) @@ -1285,14 +1454,10 @@ def export_support_bundle(out_path: str) -> None: "substrate_output": _sanitize_export_value(state.substrate_output), "pki_output": _sanitize_export_value(state.pki_output), "dns_output": _sanitize_export_value(state.dns_output), - "identity_platform_output": _sanitize_export_value( - state.identity_platform_output - ), + "identity_platform_output": _sanitize_export_value(state.identity_platform_output), "world_registry_output": _sanitize_export_value(state.world_registry_output), "domain_registry_output": _sanitize_export_value(state.domain_registry_output), - "identity_inworld_output": _sanitize_export_value( - state.identity_inworld_output - ), + "identity_inworld_output": _sanitize_export_value(state.identity_inworld_output), "ands_output": _sanitize_export_value(state.ands_output), "world_services_output": _sanitize_export_value(state.world_services_output), "org_apps_output": _sanitize_export_value(state.org_apps_output), @@ -1326,9 +1491,7 @@ def import_support_bundle(bundle_file: str) -> None: raise click.ClickException("Support bundle is missing a spec object") spec_schema = (spec_data.get("metadata") or {}).get("schema_version") if spec_schema is not None and spec_schema not in SUPPORTED_SPEC_SCHEMA_VERSIONS: - raise click.ClickException( - f"Unsupported spec metadata.schema_version: {spec_schema!r}" - ) + raise click.ClickException(f"Unsupported spec metadata.schema_version: {spec_schema!r}") spec = NetEngineSpec.model_validate(spec_data) imported_state = RuntimeState( world_spec=spec.model_dump(mode="json"), @@ -1397,9 +1560,7 @@ async def _events(queue: str | None, dlq: bool, limit: int) -> None: ) if rows: click.echo( - click.style( - f" {dlq_name} ({len(rows)} message(s)):", bold=True - ) + click.style(f" {dlq_name} ({len(rows)} message(s)):", bold=True) ) for row in rows: import json as _json @@ -1409,18 +1570,14 @@ async def _events(queue: str | None, dlq: bool, limit: int) -> None: event_type = payload.get("event_type", "?") emitted_by = payload.get("emitted_by", "?") retry_count = payload.get("retry_count", 0) - dlq_reason = (payload.get("payload") or {}).get( - "dlq_reason", "" - ) + dlq_reason = (payload.get("payload") or {}).get("dlq_reason", "") click.echo( f" [{row['msg_id']}] {event_type} " f"from={emitted_by} retries={retry_count}" + (f" reason={dlq_reason}" if dlq_reason else "") ) except Exception: - click.echo( - f" [{row['msg_id']}] (unparseable message)" - ) + click.echo(f" [{row['msg_id']}] (unparseable message)") else: click.echo(f" {dlq_name}: empty") except Exception as exc: @@ -1430,9 +1587,7 @@ async def _events(queue: str | None, dlq: bool, limit: int) -> None: for q in queues_to_check: dlq_name = dlq_for(Queue(q)).value try: - depth_row = await conn.fetchrow( - "SELECT count(*) AS depth FROM pgmq.q_$1", q - ) + depth_row = await conn.fetchrow("SELECT count(*) AS depth FROM pgmq.q_$1", q) dlq_row = await conn.fetchrow( "SELECT count(*) AS depth FROM pgmq.q_$1", dlq_name ) @@ -1444,9 +1599,7 @@ async def _events(queue: str | None, dlq: bool, limit: int) -> None: else click.style("!", fg="yellow") ) dlq_status = ( - "" - if dlq_depth == 0 - else click.style(f" DLQ: {dlq_depth}", fg="red") + "" if dlq_depth == 0 else click.style(f" DLQ: {dlq_depth}", fg="red") ) click.echo(f" {status} {q:<30} depth={depth}{dlq_status}") except Exception as exc: @@ -1518,9 +1671,7 @@ def _print_status(state: RuntimeState) -> None: "dev-sandbox: two orgs, all services, dev apps." ), ) -@click.option( - "--output", "-o", default=None, help="Output file path (default: .yaml)." -) +@click.option("--output", "-o", default=None, help="Output file path (default: .yaml).") @click.option( "--yes", "-y", @@ -1567,9 +1718,7 @@ def init( click.confirm(f"{early_path} already exists — overwrite?", abort=True) try: - cfg: WorldConfig = run_wizard( - preset=preset, yes=yes, name=name, lifecycle=lifecycle - ) + cfg: WorldConfig = run_wizard(preset=preset, yes=yes, name=name, lifecycle=lifecycle) except click.Abort: click.echo("\nAborted.", err=True) return @@ -1627,9 +1776,7 @@ def _print_init_summary(cfg: "Any", out_path: Path) -> None: click.echo(f"\n Organisations ({len(cfg.orgs)}):") for org in cfg.orgs: user_count = len(org.users) - click.echo( - f" • {org.name:<20} profile={org.and_profile} users={user_count}" - ) + click.echo(f" • {org.name:<20} profile={org.and_profile} users={user_count}") else: click.echo("\n Organisations: none (add later with `netengine reload`)") diff --git a/tests/test_cli.py b/tests/test_cli.py index 7d8c307..6b6182a 100644 --- a/tests/test_cli.py +++ b/tests/test_cli.py @@ -496,7 +496,10 @@ def test_validate_json_unsupported_active_feature_exits_nonzero(tmp_path: Path) assert result.exit_code == 1 payload = json.loads(result.output) assert payload["ok"] is False - assert payload["feature_states"][0]["path"] == "gateway_portal.real_internet.upstream_resolver_enabled" + assert ( + payload["feature_states"][0]["path"] + == "gateway_portal.real_internet.upstream_resolver_enabled" + ) assert payload["feature_states"][0]["state"] == "unsupported" assert payload["feature_states"][0]["current_value"] is True assert payload["feature_states"][0]["default_value"] is False @@ -513,3 +516,112 @@ def test_validate_explain_includes_dotted_field_paths_and_feature_states(tmp_pat assert "pki.intermediate_ca_enabled" in result.output assert "experimental" in result.output assert "alpha" in result.output + + +def test_setup_local_orchestrates_doctor_compose_migrations_and_bootstrap(tmp_path: Path): + """Setup should coordinate preflight, compose, migrations, readiness, then up.""" + spec_file = Path(__file__).parent.parent / "examples" / "minimal.yaml" + ok = cli_main.DoctorCheckResult("ok", cli_main.DoctorStatus.OK, "ready", group="test") + + with ( + patch("netengine.cli.main._run_setup_host_checks", return_value=[ok]) as host_checks, + patch("netengine.cli.main._docker_compose_up") as compose_up, + patch("netengine.cli.main._wait_for_postgres_health") as wait_postgres, + patch("netengine.cli.main._run_migrations", new_callable=AsyncMock) as migrations, + patch("netengine.cli.main.run_preflight", return_value=[ok]) as readiness, + patch( + "netengine.cli.main._check_migration_readiness", + new_callable=AsyncMock, + return_value=[ok], + ), + patch("netengine.cli.main._feature_state_readiness_results", return_value=[ok]), + patch("netengine.cli.main._up", new_callable=AsyncMock) as up, + ): + result = CliRunner().invoke( + cli_main.cli, + [ + "setup", + "local", + str(spec_file), + "--db-url", + "postgresql://example/db", + "--state-file", + str(tmp_path / "state.json"), + ], + ) + + assert result.exit_code == 0, result.output + assert "Step 1/5: host checks before Postgres starts" in result.output + assert "Setup checks passed" in result.output + host_checks.assert_called_once() + compose_up.assert_called_once_with(("postgres",)) + wait_postgres.assert_called_once_with(timeout_seconds=120.0) + migrations.assert_awaited_once_with("postgresql://example/db") + readiness.assert_called_once() + up.assert_awaited_once() + assert up.call_args.kwargs["skip_migrations"] is True + + +def test_setup_local_stops_before_compose_when_required_host_check_fails(tmp_path: Path): + """Setup should not start infrastructure when required pre-Postgres checks fail.""" + spec_file = Path(__file__).parent.parent / "examples" / "minimal.yaml" + failed = cli_main.DoctorCheckResult( + "Docker container names", + cli_main.DoctorStatus.FAIL, + "netengine_postgres", + "Stop/remove conflicting containers before startup.", + "docker", + required=True, + ) + + with ( + patch("netengine.cli.main._run_setup_host_checks", return_value=[failed]), + patch("netengine.cli.main._docker_compose_up") as compose_up, + patch("netengine.cli.main._up", new_callable=AsyncMock) as up, + ): + result = CliRunner().invoke( + cli_main.cli, + ["setup", "local", str(spec_file), "--state-file", str(tmp_path / "state.json")], + ) + + assert result.exit_code == 1 + assert "Hint: Stop/remove conflicting containers before startup." in result.output + assert "required host checks failed" in result.output + compose_up.assert_not_called() + up.assert_not_awaited() + + +def test_setup_local_stops_before_up_when_spec_aware_check_fails(tmp_path: Path): + """Setup should stop after migrations when spec-aware doctor checks fail.""" + spec_file = Path(__file__).parent.parent / "examples" / "minimal.yaml" + ok = cli_main.DoctorCheckResult("ok", cli_main.DoctorStatus.OK, "ready", group="test") + failed = cli_main.DoctorCheckResult( + "Database connectivity", + cli_main.DoctorStatus.FAIL, + "connection refused", + "Verify database connectivity, credentials, and pgmq/Postgres availability.", + "database", + required=True, + ) + + with ( + patch("netengine.cli.main._run_setup_host_checks", return_value=[ok]), + patch("netengine.cli.main._docker_compose_up"), + patch("netengine.cli.main._wait_for_postgres_health"), + patch("netengine.cli.main._run_migrations", new_callable=AsyncMock), + patch("netengine.cli.main.run_preflight", return_value=[failed]), + patch( + "netengine.cli.main._check_migration_readiness", new_callable=AsyncMock, return_value=[] + ), + patch("netengine.cli.main._feature_state_readiness_results", return_value=[]), + patch("netengine.cli.main._up", new_callable=AsyncMock) as up, + ): + result = CliRunner().invoke( + cli_main.cli, + ["setup", "local", str(spec_file), "--db-url", "postgresql://example/db"], + ) + + assert result.exit_code == 1 + assert "required setup checks failed; NetEngine stopped before `netengine up`" in result.output + assert "Verify database connectivity" in result.output + up.assert_not_awaited() diff --git a/tests/test_doctor.py b/tests/test_doctor.py index 9abcc8f..8a37028 100644 --- a/tests/test_doctor.py +++ b/tests/test_doctor.py @@ -375,7 +375,6 @@ def fake_run(command: list[str], *, timeout: float = 8.0): assert "Stop the process using 5432/tcp" in (result.hint or "") -def test_docker_resource_conflicts_use_short_format_commands(monkeypatch, tmp_path: Path) -> None: def test_docker_resource_conflicts_use_short_format_commands( monkeypatch, tmp_path: Path ) -> None: