Add PostgreSQL setup, backup automation, and recovery runbook

[dashprotocol]

Summary

Implements ticket H-007 by adding PostgreSQL setup, daily backup automation, and restore runbook for Havenhold on Lightsail.

This PR delivers the D0 infra foundation and documents the required H-010 pre-launch restore test sequencing (schema + seed required before restore validation can pass).

Changes

Infra scripts

• infra/postgres-setup.sh

  • Installs PostgreSQL (PGDG fallback path included)
  • Starts/enables service
  • Forces localhost-only binding
  • Manages app pg_hba.conf entries for both 127.0.0.1/32 and ::1/128 (scram-sha-256)
  • Creates/updates app role + DB
  • Creates /var/backups/havenhold with postgres:postgres 750
  • Validates TCP connectivity via 127.0.0.1

• infra/postgres-backup.sh

  • Backup mode: plain SQL pg_dump with non-empty file check and retention prune
  • Cron install mode: installs /usr/local/bin/postgres-backup.sh and writes /etc/cron.d/havenhold-backup
  • Uses CRON_TZ=UTC, explicit postgres run user
  • Idempotent cron updates via desired-content comparison

• infra/postgres-restore.sh

  • Restores latest or specified backup
  • Confirmation gate (unless SKIP_CONFIRM=true)
  • Terminates DB connections, runs dropdb/createdb, restores with ON_ERROR_STOP
  • Prints row counts and fails if Patient < 1

Runbook

• docs/runbook/postgres-backup-and-recovery.md
  • Full operator workflow with copy/paste commands
  • Clear split:
    • D0 (H-007): setup + cron
    • H-010 pre-launch: migrate, seed, backup, restore test, then start app
  • Evidence and recovery guidance included

Gate / AC handling

• D0 gate is satisfied by infra readiness (Postgres + cron + runbook).
• “Restore tested once” AC is executed in H-010 pre-launch sequence before first app start, since it depends on deployed code + migrations + seed data.

Validation

• Script syntax checks passed:
  • bash -n infra/postgres-setup.sh
  • bash -n infra/postgres-backup.sh
  • bash -n infra/postgres-restore.sh
• Manual host execution/operational verification to be captured during rollout runbook steps.