Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion README.md
Original file line number Diff line number Diff line change
Expand Up @@ -189,7 +189,7 @@ CLI helper example:
KEY_PAIR_NAME=<lightsail-key-name> ./infra/create-instance.sh
```

See `docs/runbook/H-005-lightsail-provision.md` for the full runbook and safety checks.
See `docs/runbook/lightsail-provision-host-hardening.md` for the full runbook and safety checks.

## Security and Privacy Notes

Expand Down
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
# H-005 Runbook: Lightsail Provision + Host Hardening
# Runbook: Lightsail Provision + Host Hardening

## Purpose
Provision a single Havenhold MVP Lightsail host and apply baseline hardening to satisfy deploy gate `D0`.
Expand All @@ -17,9 +17,9 @@ Provision a single Havenhold MVP Lightsail host and apply baseline hardening to
## Resource Naming
- Instance: `havenhold-app-01`
- Static IP: `havenhold-app-ip`
- Admin user: `havenadmin`
- Admin user: `adminuser` (example; choose your own)

## Provision Path A (Recommended for this ticket): AWS Dashboard UI
## Provision Path A: AWS Dashboard UI
1. In Lightsail (`us-east-1`), create instance:
- Platform: Linux/Unix
- Blueprint: Ubuntu `24.04 LTS`
Expand Down Expand Up @@ -53,20 +53,22 @@ KEY_PAIR_NAME=<lightsail-key-name> SSH_PORT=2222 ./infra/create-instance.sh
Copy and execute once on the VM:

```bash
scp -i <key.pem> ./infra/provision.sh ubuntu@<STATIC_IP>:/tmp/provision.sh
ssh -i <key.pem> ubuntu@<STATIC_IP> 'bash /tmp/provision.sh'
KEY_PATH=/path/to/key.pem
STATIC_IP=203.0.113.10
scp -i "$KEY_PATH" ./infra/provision.sh ubuntu@"$STATIC_IP":/tmp/provision.sh
ssh -i "$KEY_PATH" ubuntu@"$STATIC_IP" 'DEPLOY_USER=adminuser bash /tmp/provision.sh'
```

After script completion, set a local sudo password for the hardened admin user:

```bash
ssh -t -i <key.pem> ubuntu@<STATIC_IP> 'sudo passwd havenadmin'
ssh -t -i "$KEY_PATH" ubuntu@"$STATIC_IP" 'sudo passwd adminuser'
```

Optional custom SSH port on host:

```bash
ssh -i <key.pem> ubuntu@<STATIC_IP> 'SSH_PORT=2222 bash /tmp/provision.sh'
ssh -i "$KEY_PATH" ubuntu@"$STATIC_IP" 'DEPLOY_USER=adminuser SSH_PORT=2222 bash /tmp/provision.sh'
```

## Critical Safety Sequence (Avoid Lockout)
Expand All @@ -75,15 +77,16 @@ ssh -i <key.pem> ubuntu@<STATIC_IP> 'SSH_PORT=2222 bash /tmp/provision.sh'
3. From local machine, open a second SSH session with hardened user:

```bash
ssh -i <key.pem> -p <SSH_PORT> havenadmin@<STATIC_IP>
SSH_PORT=22
ssh -i "$KEY_PATH" -p "$SSH_PORT" adminuser@"$STATIC_IP"
```

4. Only close original session after second session succeeds.

## D0 Verification Checklist
- [ ] Lightsail instance exists in `us-east-1` with static IP attached.
- [ ] Lightsail inbound rules expose only required ports (`22/80/443` or `2222/80/443`).
- [ ] `ssh -p <SSH_PORT> havenadmin@<STATIC_IP>` works with key auth.
- [ ] `ssh -p "$SSH_PORT" adminuser@"$STATIC_IP"` works with key auth.
- [ ] Root SSH login is rejected.
- [ ] Password SSH auth is rejected.
- [ ] UFW enabled with default deny inbound and only required ports allowed.
Expand Down Expand Up @@ -113,7 +116,7 @@ sudo fail2ban-client status sshd
Optional from local machine:

```bash
nmap -p <SSH_PORT>,80,443 <STATIC_IP>
nmap -p "$SSH_PORT",80,443 "$STATIC_IP"
```

## Rollback / Recovery
Expand Down
109 changes: 109 additions & 0 deletions docs/runbook/nginx-reverse-proxy-tls.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,109 @@
# Runbook: Nginx Reverse Proxy + TLS

## Purpose
Install and configure Nginx as a reverse proxy on the Havenhold app host, issue a Let's Encrypt TLS certificate for your production hostname, enforce HTTP → HTTPS redirect, and verify auto-renewal.

## Security Handling
- This runbook is git-tracked and must stay sanitized.
- Do not commit secrets, private keys, account tokens, or operator-only CIDRs if sensitive.
- Store key material in approved team secret storage.
- The `nginx-tls.sh` script is run under `sudo -E`; keep the Lightsail static IP and email address out of version control.

## Prerequisites
- Host hardening baseline is complete — admin user exists, UFW and fail2ban active.
- DNS `A` record for your hostname points to the Lightsail static IP (`havenhold-app-ip`).
- Cloudflare proxy status: **DNS only (grey cloud)** during initial cert issuance (HTTP-01 challenge must reach the host directly).
- Lightsail firewall inbound rules allow `80/tcp` and `443/tcp` from the internet.
- Repo checked out locally with `infra/nginx/havenhold` and `infra/nginx-tls.sh` present.

## Deploy Steps

Copy the site config and setup script to the host, then run the script remotely:

```bash
KEY_PATH=/path/to/key.pem
STATIC_IP=203.0.113.10
ADMIN_USER=adminuser
HOSTNAME=app.example.com
EMAIL=ops@example.com
scp -i "$KEY_PATH" infra/nginx/havenhold "$ADMIN_USER"@"$STATIC_IP":/tmp/havenhold-nginx
scp -i "$KEY_PATH" infra/nginx-tls.sh "$ADMIN_USER"@"$STATIC_IP":/tmp/nginx-tls.sh
ssh -i "$KEY_PATH" "$ADMIN_USER"@"$STATIC_IP" \
"HOSTNAME=$HOSTNAME EMAIL=$EMAIL STATIC_IP=$STATIC_IP sudo -E bash /tmp/nginx-tls.sh"
```

Replace the variable values with your real key path, host IP, admin username, hostname, and contact email before running.

The script is idempotent — it is safe to re-run if a step fails mid-way after fixing the underlying issue.

## D0 Verification Checklist
- [ ] DNS points hostname to correct static IP.
- [ ] Nginx active and healthy (`systemctl status nginx`).
- [ ] HTTPS endpoint responds with valid certificate for hostname.
- [ ] HTTP returns permanent redirect (301) to HTTPS.
- [ ] Renewal automation enabled (`certbot.timer` active).
- [ ] `certbot renew --dry-run` succeeds.
- [ ] Runbook updated with provisioning + renewal details.

## Evidence Commands
Run on host unless noted:

```bash
# Nginx health/config
sudo nginx -t
systemctl status nginx --no-pager

# Redirect behavior (expect 301)
curl -I http://app.example.com

# HTTPS response
curl -I https://app.example.com

# Certificate validity
echo | openssl s_client -servername app.example.com -connect app.example.com:443 2>/dev/null | openssl x509 -noout -issuer -subject -dates
sudo certbot certificates

# Renewal posture
systemctl status certbot.timer --no-pager
sudo certbot renew --dry-run
```

## Renewal Operations

### Check timer status
```bash
systemctl status certbot.timer --no-pager
systemctl list-timers certbot.timer
```

### Force manual renewal
```bash
sudo certbot renew --force-renewal
sudo nginx -t && sudo systemctl reload nginx
```

### Dry-run renewal (non-destructive verification)
```bash
sudo certbot renew --dry-run
```

### Troubleshoot renewal failures
```bash
# Check certbot logs
sudo journalctl -u certbot.timer --no-pager -n 50
sudo journalctl -u certbot.service --no-pager -n 50

# Verify nginx config after renewal attempt
sudo nginx -t
```

## Rollback / Recovery
- **Remove certificate and revert to HTTP-only:** `sudo certbot delete --cert-name app.example.com`, then restore the original `infra/nginx/havenhold` template to `/etc/nginx/sites-available/havenhold` and reload Nginx.
- **Stop Nginx entirely:** `sudo systemctl stop nginx` (takes site offline; use only in emergencies).
- **Restore from snapshot:** Take a Lightsail snapshot before running this script. If configuration is unrecoverable, restore the snapshot from the Lightsail console and re-run only the steps that failed.
- **Certbot-modified config is corrupt:** `sudo certbot rollback` reverts the last Certbot config checkpoint; then re-run from Step 6 of the script after resolving the underlying issue.

## Notes
- The committed `infra/nginx/havenhold` template is HTTP-only. Certbot modifies it in-place to add SSL directives and the redirect block. Do not pre-populate `ssl_` directives — they will conflict with Certbot's edits.
- HSTS (`Strict-Transport-Security`) is commented out in the template. Uncomment only after HTTPS is confirmed stable end-to-end, and be aware it is difficult to reverse once a browser caches the header.
- Add TLS expiry and uptime alerting in your observability workstream; until then, monitor `certbot.timer` via `journalctl`.
4 changes: 2 additions & 2 deletions infra/create-instance.sh
Original file line number Diff line number Diff line change
@@ -1,9 +1,9 @@
#!/usr/bin/env bash
set -euo pipefail

# Optional helper for H-005.
# Optional provisioning helper.
# This script uses AWS CLI to provision a Lightsail instance with a static IP and minimal public ports.
# Keep docs/runbook/H-005-lightsail-provision.md as source of truth.
# Keep the provisioning runbook as the operational source of truth.

INSTANCE_NAME="${INSTANCE_NAME:-havenhold-app-01}"
STATIC_IP_NAME="${STATIC_IP_NAME:-havenhold-app-ip}"
Expand Down
101 changes: 101 additions & 0 deletions infra/nginx-tls.sh
Original file line number Diff line number Diff line change
@@ -0,0 +1,101 @@
#!/usr/bin/env bash
# infra/nginx-tls.sh — idempotent Nginx + Certbot TLS setup
# Operator invocation (from repo root):
# KEY_PATH=/path/to/key.pem
# STATIC_IP=203.0.113.10
# ADMIN_USER=adminuser
# HOSTNAME=app.example.com
# EMAIL=ops@example.com
# scp -i "$KEY_PATH" infra/nginx/havenhold "$ADMIN_USER"@"$STATIC_IP":/tmp/havenhold-nginx
# scp -i "$KEY_PATH" infra/nginx-tls.sh "$ADMIN_USER"@"$STATIC_IP":/tmp/nginx-tls.sh
# ssh -i "$KEY_PATH" "$ADMIN_USER"@"$STATIC_IP" \
# "HOSTNAME=$HOSTNAME EMAIL=$EMAIL STATIC_IP=$STATIC_IP sudo -E bash /tmp/nginx-tls.sh"

set -euo pipefail

HOSTNAME="${HOSTNAME:?HOSTNAME env var required (e.g. app.example.com)}"
EMAIL="${EMAIL:?EMAIL env var required for certificate registration}"
STATIC_IP="${STATIC_IP:?STATIC_IP env var required}"
NGINX_SRC="${NGINX_SRC:-/tmp/havenhold-nginx}"

# ---------------------------------------------------------------------------
# Pre-step: ensure dig is available (not present on all Ubuntu images)
# ---------------------------------------------------------------------------
if ! command -v dig &>/dev/null; then
echo "[nginx-tls] dig not found — installing dnsutils"
apt-get update -q
apt-get install -y dnsutils
fi

# ---------------------------------------------------------------------------
# Step 1: DNS precheck — abort if hostname does not resolve to static IP
# ---------------------------------------------------------------------------
echo "[nginx-tls] Step 1: DNS precheck for ${HOSTNAME} → ${STATIC_IP}"
RESOLVED=$(dig +short "${HOSTNAME}" | tail -n1)
if [ "${RESOLVED}" != "${STATIC_IP}" ]; then
echo "[nginx-tls] ERROR: ${HOSTNAME} resolves to '${RESOLVED}', expected '${STATIC_IP}'. Aborting." >&2
exit 1
fi
echo "[nginx-tls] DNS OK: ${HOSTNAME} → ${RESOLVED}"

# ---------------------------------------------------------------------------
# Step 2: Upstream binding check — warn if port 3001 is not loopback-only
# ---------------------------------------------------------------------------
echo "[nginx-tls] Step 2: Upstream binding check (port 3001)"
if ss -tulpen 2>/dev/null | grep ':3001' | grep -qv '127.0.0.1'; then
echo "[nginx-tls] WARNING: port 3001 appears to be bound to a non-loopback address." >&2
fi

# ---------------------------------------------------------------------------
# Step 3: Install Nginx + Certbot
# ---------------------------------------------------------------------------
echo "[nginx-tls] Step 3: Installing nginx and certbot"
apt-get update -q
apt-get install -y nginx python3-certbot-nginx

# ---------------------------------------------------------------------------
# Step 4: Deploy site config, remove default site
# ---------------------------------------------------------------------------
echo "[nginx-tls] Step 4: Deploying site config"
if [ ! -f "${NGINX_SRC}" ]; then
echo "[nginx-tls] ERROR: site config not found at ${NGINX_SRC}" >&2
exit 1
fi
cp "${NGINX_SRC}" /etc/nginx/sites-available/havenhold
sed -i "s|<HOSTNAME>|${HOSTNAME}|g" /etc/nginx/sites-available/havenhold
ln -sf /etc/nginx/sites-available/havenhold /etc/nginx/sites-enabled/havenhold
rm -f /etc/nginx/sites-enabled/default

# ---------------------------------------------------------------------------
# Step 5: Validate config and start Nginx
# ---------------------------------------------------------------------------
echo "[nginx-tls] Step 5: nginx -t + enable and start"
nginx -t
systemctl enable --now nginx

# ---------------------------------------------------------------------------
# Step 6: Issue certificate, configure HTTPS + redirect via Certbot
# ---------------------------------------------------------------------------
echo "[nginx-tls] Step 6: Certbot certificate issuance and HTTPS configuration"
certbot --nginx \
-d "${HOSTNAME}" \
--non-interactive \
--agree-tos \
-m "${EMAIL}" \
--redirect

# ---------------------------------------------------------------------------
# Step 7: Second validation + reload after Certbot edits
# ---------------------------------------------------------------------------
echo "[nginx-tls] Step 7: Second nginx -t + reload"
nginx -t
systemctl reload nginx

# ---------------------------------------------------------------------------
# Step 8: Enable certbot auto-renewal timer
# ---------------------------------------------------------------------------
echo "[nginx-tls] Step 8: Enable certbot.timer"
systemctl enable certbot.timer
systemctl start certbot.timer

echo "[nginx-tls] Done. Run 'sudo certbot renew --dry-run' to verify renewal posture."
24 changes: 24 additions & 0 deletions infra/nginx/havenhold
Original file line number Diff line number Diff line change
@@ -0,0 +1,24 @@
# /etc/nginx/sites-available/havenhold
# Certbot will modify this file: adding ssl_certificate lines and a redirect block.
# Do not manually add ssl_ directives before running certbot.

server {
listen 80;
server_name <HOSTNAME>;

server_tokens off;

location / {
proxy_pass http://127.0.0.1:3001;
proxy_http_version 1.1;
proxy_set_header Host $host;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header X-Forwarded-Proto $scheme;
proxy_set_header Upgrade $http_upgrade;
proxy_set_header Connection "upgrade";
}
}

# HSTS — uncomment only after HTTPS is confirmed stable end-to-end:
# add_header Strict-Transport-Security "max-age=31536000" always;
4 changes: 2 additions & 2 deletions infra/provision.sh
Original file line number Diff line number Diff line change
@@ -1,10 +1,10 @@
#!/usr/bin/env bash
set -euo pipefail

# H-005 one-time host baseline hardening script for Ubuntu Lightsail.
# One-time host baseline hardening script for Ubuntu Lightsail.
# Run as: bash /tmp/provision.sh (as ubuntu user with sudo access)

DEPLOY_USER="${DEPLOY_USER:-havenadmin}"
DEPLOY_USER="${DEPLOY_USER:-adminuser}"
SSH_PORT="${SSH_PORT:-22}"
ALLOW_UBUNTU_USER="${ALLOW_UBUNTU_USER:-true}"

Expand Down
4 changes: 4 additions & 0 deletions server/.env.example
Original file line number Diff line number Diff line change
Expand Up @@ -10,3 +10,7 @@ DEMO_USER_ID="<seeded-user-id>"
DEMO_MODE=true # true = demo auth middleware mounted (default). false = server refuses to start (no JWT auth yet).
PIPELINE_ENABLED=true # true = AI document pipeline active (default). false = /documents/upload returns 503.
INTEGRATIONS_ENABLED=false # false = no outbound integration calls (default). Reserved for future use.

# Deployment environment and CORS (production host only — omit or set NODE_ENV=development locally)
NODE_ENV=production
CORS_ORIGIN=https://<HOSTNAME>
9 changes: 8 additions & 1 deletion server/src/index.ts
Original file line number Diff line number Diff line change
Expand Up @@ -30,7 +30,14 @@ if (!flags.DEMO_MODE) {
const app = express();
const PORT = process.env.PORT ?? 3001;

app.use(cors({ origin: ['http://localhost:5173', 'http://localhost:8080', 'http://localhost:3000'] }));
const devOrigins = ['http://localhost:5173', 'http://localhost:8080', 'http://localhost:3000'];
const prodOrigins =
process.env.NODE_ENV !== 'development' && process.env.CORS_ORIGIN
? process.env.CORS_ORIGIN.split(',').map(o => o.trim()).filter(Boolean)
: [];
// devOrigins are always included while DEMO_MODE is the only auth layer.
// When real auth lands, restrict to prodOrigins-only in production (remove devOrigins spread).
app.use(cors({ origin: [...devOrigins, ...prodOrigins] }));
app.use(express.json());

// Demo auth: sets req.user on every request.
Expand Down
Loading