fix(bootstrap): resolve DNS failures and add container memory limits

[brianwtaylor]

Supersedes #478

@drew — reworked per your review: the bootstrap crate now sniffs resolvers and passes them as an UPSTREAM_DNS env var. No system files are mounted into the container.

Summary

• Sniff upstream DNS resolvers from the Rust bootstrap crate by reading /run/systemd/resolve/resolv.conf (systemd-resolved hosts only)
• Filter loopback addresses (127.x.x.x, ::1) and pass result to container as UPSTREAM_DNS env var
• Skip DNS sniffing for remote deploys where local resolvers would be wrong
• Entrypoint reads UPSTREAM_DNS first, falls back to /etc/resolv.conf for manual launches
• Add DNS verification logging on failure
• Add --memory flag to openshell gateway start that caps container memory via Docker HostConfig
• Auto-detect 80% of available memory from Docker daemon when unset; supports human-readable sizes (80g, 4096m) and OPENSHELL_MEMORY_LIMIT env var

Closes #437

Changes

crates/openshell-bootstrap/src/docker.rs — Add resolve_upstream_dns() that reads /run/systemd/resolve/resolv.conf, filters loopback addresses, and returns real upstream resolvers. Pass them as UPSTREAM_DNS env var to the cluster container (skipped for remote deploys). Includes unit tests.

deploy/docker/cluster-entrypoint.sh — Add get_upstream_resolvers() that reads UPSTREAM_DNS env var (priority) or falls back to /etc/resolv.conf. When upstream resolvers are found, write them directly to the k3s resolv.conf instead of relying on DNAT proxy. Improve DNS verification logging on failure.

crates/openshell-bootstrap/src/docker.rs — Add parse_memory_limit() for human-readable size parsing and detect_memory_limit() that queries Docker daemon MemTotal. Sets memory_swap = memory to disable swap inside the container.

crates/openshell-cli/src/main.rs / bootstrap.rs — Add --memory CLI flag, wire through to Docker container creation.

Root Cause

Docker's embedded DNS at 127.0.0.11 is only reachable from the container's own network namespace. The existing DNAT rules forward to this loopback address, but k3s pods run in child network namespaces where the forwarded packets are dropped as martian packets. On systemd-resolved hosts, /etc/resolv.conf contains 127.0.0.53 (another loopback), so the fallback also fails silently.

DNS Flow — Before vs After

BEFORE (broken on systemd-resolved hosts):

  Pod → CoreDNS → resolv.conf → iptables DNAT → Docker DNS
        (cache)   127.0.0.11     PREROUTING     127.0.0.11
                        │                            │
                        └──── FAILS ─────────────────┘
                        loopback DNAT from pod namespace
                        dropped as martian packet

AFTER (this PR):

  Pod → CoreDNS → resolv.conf → upstream resolver → response
        (cache)   e.g. 192.168.1.1    (direct UDP)
                   ▲
                   │
              set by Rust bootstrap:
              resolve_upstream_dns()
              reads /run/systemd/resolve/resolv.conf
              passes via UPSTREAM_DNS env var
              entrypoint writes to k3s resolv.conf

NON-SYSTEMD HOSTS (macOS, WSL2, Alpine) — unchanged:

  Pod → CoreDNS → resolv.conf → iptables DNAT → Docker DNS → host
        (cache)   container IP   PREROUTING     127.0.0.11

  /run/systemd/resolve/resolv.conf absent → UPSTREAM_DNS not set →
  entrypoint falls back to existing DNAT proxy path. Zero behavior change.

Testing

Software Versions

Component	Version
OpenShell (release)	v0.0.13
OpenShell (PR branch)	0.0.13-dev.11+gebc1369
Cluster image (PR branch)	openshell/cluster:dev (built from ebc1369)
NemoClaw	0.1.0

Results

• Tested on DGX Spark (Ubuntu 24.04, systemd-resolved, Docker with
  cgroupns=host)
• Verified DNS resolution works from k3s pods after the fix
• Verified no behavior change on macOS (Apple Silicon) and Windows/WSL2 hosts

===VALIDATION TOPOLOGY===

                ┌─────────────────────────┐
                │     Node A (Linux)      │
                │    aarch64 · GPU        │
                │    OpenShell 0.0.13     │
                │                         │
                │  BASELINE + ORCHESTRATOR │
                │  (read-only, runs all   │
                │   tests from here)      │
                └─────┬──────────┬────────┘
                      │          │
        high-speed    │          │ LAN
        interconnect  │          │
                      │          ├──────────────┐
            ┌─────────▼──┐   ┌───▼──────────┐  ┌▼──────────────┐
            │  Node B    │   │  Node C      │  │  Node D       │
            │  Linux     │   │  macOS       │  │  Windows/WSL2 │
            │  aarch64   │   │  Apple Si    │  │  x86_64       │
            │  OS 0.0.13 │   │  OS 0.0.13   │  │  OS 0.0.13   │
            │  -dev.11   │   │  no systemd  │  │  no systemd   │
            │            │   │              │  │               │
            │ TEST TARGET │   │              │  │               │
            │ (DNS-fixed │   │  CONTROL     │  │  CONTROL      │
            │  gateway   │   │  ✓ verified  │  │  ✓ verified   │
            │  deployed) │   │              │  │               │
            └────────────┘   └──────────────┘  └───────────────┘

===WHAT EACH NODE PROVED DURING VALIDATION===

Node A ─── Baseline capture. systemd-resolved active, upstream 192.168.4.1,
stub at 127.0.0.53. All existing containers healthy. Zero drift.

Node B ─── Fix works. Custom binary + image from PR branch deployed.
UPSTREAM_DNS=192.168.4.1 set, written to k3s resolv.conf.
Pod DNS resolution verified. All pods healthy.

Node C ─── No change on macOS. No UPSTREAM_DNS set. DNAT proxy path intact.
Pod DNS resolution verified.

Node D ─── No change on WSL2. No UPSTREAM_DNS set. DNAT proxy path intact.
Pod DNS resolution verified.

Automated Tests

cargo test -p openshell-bootstrap
cargo test -p openshell-cli

Checklist

• Follows Conventional Commits
• Commits are signed off (DCO)
• Addresses review feedback from fix(cluster): resolve DNS failures on systemd-resolved hosts #478 (no system file mounts)

[drew]

This seems not necessary now, right?

[brianwtaylor]

good call. no need for these tests with the logic living properly in rust. apologies for the duplicate PR. lost the first while while deploying these changes in my test environment yesterday.

[brianwtaylor]

explored this change more last night, ran into a memory issue that im working through. would like to get it figured out rather than rushing this change in and introducing a new round of signal noise in the issues section. temporarily moving to draft.

-brian

[drew]

We are going to refactor the bootstrapping workflow to use a microVM instead of Docker based k3s cluster. This should improve some of the reliability on the networking side.

I would hold off on this PR until we get moved over to the microVM based approach. Discussion on the topic is here #558.