Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion .claude/skills/project-setup/SKILL.md
Original file line number Diff line number Diff line change
Expand Up @@ -55,7 +55,7 @@ The `.env.example` file documents all variables. For local dev, only three need
| Variable | Source |
|----------|--------|
| `SUPABASE_KEY` | Output of `supabase start` (anon key) |
| `SUPABASE_SERVICE_KEY` | Output of `supabase start` (service_role key) |
| `SUPABASE_SECRET_KEY` | Output of `supabase start` (secret key) |
| `HUGGING_FACE_TOKEN` | huggingface.co/settings/tokens (for seeding datasets) |

### Common Local Commands
Expand Down
4 changes: 2 additions & 2 deletions .claude/skills/project-setup/references/runbook.md
Original file line number Diff line number Diff line change
Expand Up @@ -34,7 +34,7 @@ The defaults in `.env.example` work for local development with Supabase. You onl
| Variable | Where to get it | Required? |
|----------|----------------|-----------|
| `SUPABASE_KEY` | Output of `supabase start` (anon key) | Yes |
| `SUPABASE_SERVICE_KEY` | Output of `supabase start` (service_role key) | Yes |
| `SUPABASE_SECRET_KEY` | Output of `supabase start` (secret key) | Yes |
| `HUGGING_FACE_TOKEN` | [huggingface.co/settings/tokens](https://huggingface.co/settings/tokens) | For seeding datasets |
| `ANTHROPIC_API_KEY` | [console.anthropic.com](https://console.anthropic.com) | For `/agent` endpoint only |
| `LOGFIRE_TOKEN` | [logfire.pydantic.dev](https://logfire.pydantic.dev) | Optional (observability) |
Expand All @@ -45,7 +45,7 @@ The defaults in `.env.example` work for local development with Supabase. You onl
supabase start
```

This starts local Postgres (port 54322), PostgREST (port 54321), and storage. Copy the `anon key` and `service_role key` from the output into your `.env`.
This starts local Postgres (port 54322), PostgREST (port 54321), and storage. Copy the `anon key` and `secret key` (formerly service_role key) from the output into your `.env`.

### Step 4: Initialize database

Expand Down
6 changes: 3 additions & 3 deletions .env.example
Original file line number Diff line number Diff line change
Expand Up @@ -13,8 +13,8 @@ SUPABASE_URL=http://127.0.0.1:54321
# Supabase anon/public key (safe to expose in client)
SUPABASE_KEY=your-anon-key

# Supabase service role key (server-side only, never expose)
SUPABASE_SERVICE_KEY=your-service-role-key
# Supabase secret key (server-side only, never expose)
SUPABASE_SECRET_KEY=your-secret-key

# PostgreSQL connection string for direct database access
# For production Supabase: use the "connection string" from Dashboard > Settings > Database
Expand Down Expand Up @@ -80,7 +80,7 @@ MODAL_ENVIRONMENT=main
# DATABASE_URL='postgresql://...' \
# SUPABASE_URL='https://...' \
# SUPABASE_KEY='...' \
# SUPABASE_SERVICE_KEY='...' \
# SUPABASE_SECRET_KEY='...' \
# STORAGE_BUCKET='datasets'
#
# 2. modal secret create anthropic-api-key \
Expand Down
3 changes: 2 additions & 1 deletion .github/scripts/modal-sync-secrets.sh
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
#!/bin/bash
# Sync secrets from GitHub Actions to a Modal environment
# Usage: ./modal-sync-secrets.sh <modal-environment> <logfire-environment>
# Required env vars: SUPABASE_DB_URL, SUPABASE_URL, SUPABASE_KEY, LOGFIRE_TOKEN
# Required env vars: SUPABASE_DB_URL, SUPABASE_URL, SUPABASE_KEY, SUPABASE_SECRET_KEY, LOGFIRE_TOKEN
set -euo pipefail

MODAL_ENV="${1:?Modal environment required (staging or main)}"
Expand All @@ -13,6 +13,7 @@ uv run modal secret create policyengine-db \
"DATABASE_URL=${SUPABASE_DB_URL}" \
"SUPABASE_URL=${SUPABASE_URL}" \
"SUPABASE_KEY=${SUPABASE_KEY}" \
"SUPABASE_SECRET_KEY=${SUPABASE_SECRET_KEY}" \
"STORAGE_BUCKET=${STORAGE_BUCKET:-datasets}" \
--env="$MODAL_ENV" \
--force
Expand Down
6 changes: 3 additions & 3 deletions .github/workflows/db-reset.yml
Original file line number Diff line number Diff line change
Expand Up @@ -84,7 +84,7 @@ jobs:
SUPABASE_DB_URL: ${{ secrets.SUPABASE_POOLER_URL }}
SUPABASE_URL: ${{ secrets.SUPABASE_URL }}
SUPABASE_KEY: ${{ secrets.SUPABASE_KEY }}
SUPABASE_SERVICE_KEY: ${{ secrets.SUPABASE_SERVICE_KEY }}
SUPABASE_SECRET_KEY: ${{ secrets.SUPABASE_SECRET_KEY }}
LOGFIRE_TOKEN: ${{ secrets.LOGFIRE_TOKEN }}
LOGFIRE_ENVIRONMENT: prod
run: |
Expand All @@ -97,7 +97,7 @@ jobs:
SUPABASE_DB_URL: ${{ secrets.SUPABASE_POOLER_URL }}
SUPABASE_URL: ${{ secrets.SUPABASE_URL }}
SUPABASE_KEY: ${{ secrets.SUPABASE_KEY }}
SUPABASE_SERVICE_KEY: ${{ secrets.SUPABASE_SERVICE_KEY }}
SUPABASE_SECRET_KEY: ${{ secrets.SUPABASE_SECRET_KEY }}
STORAGE_BUCKET: ${{ vars.STORAGE_BUCKET }}
LOGFIRE_TOKEN: ${{ secrets.LOGFIRE_TOKEN }}
LOGFIRE_ENVIRONMENT: prod
Expand All @@ -112,7 +112,7 @@ jobs:
SUPABASE_DB_URL: ${{ secrets.SUPABASE_POOLER_URL }}
SUPABASE_URL: ${{ secrets.SUPABASE_URL }}
SUPABASE_KEY: ${{ secrets.SUPABASE_KEY }}
SUPABASE_SERVICE_KEY: ${{ secrets.SUPABASE_SERVICE_KEY }}
SUPABASE_SECRET_KEY: ${{ secrets.SUPABASE_SECRET_KEY }}
HUGGING_FACE_TOKEN: ${{ secrets.HUGGING_FACE_TOKEN }}
STORAGE_BUCKET: ${{ vars.STORAGE_BUCKET }}
LOGFIRE_TOKEN: ${{ secrets.LOGFIRE_TOKEN }}
Expand Down
3 changes: 3 additions & 0 deletions .github/workflows/deploy.yml
Original file line number Diff line number Diff line change
Expand Up @@ -141,6 +141,7 @@ jobs:
env:
TF_VAR_supabase_url: ${{ secrets.SUPABASE_URL }}
TF_VAR_supabase_key: ${{ secrets.SUPABASE_KEY }}
TF_VAR_supabase_secret_key: ${{ secrets.SUPABASE_SECRET_KEY }}
TF_VAR_supabase_db_url: ${{ secrets.SUPABASE_DB_URL }}
TF_VAR_logfire_token: ${{ secrets.LOGFIRE_TOKEN }}
TF_VAR_logfire_environment: prod
Expand Down Expand Up @@ -208,6 +209,7 @@ jobs:
SUPABASE_DB_URL: ${{ secrets.SUPABASE_DB_URL }}
SUPABASE_URL: ${{ secrets.SUPABASE_URL }}
SUPABASE_KEY: ${{ secrets.SUPABASE_KEY }}
SUPABASE_SECRET_KEY: ${{ secrets.SUPABASE_SECRET_KEY }}
LOGFIRE_TOKEN: ${{ secrets.LOGFIRE_TOKEN }}
run: |
chmod +x .github/scripts/*.sh
Expand Down Expand Up @@ -325,6 +327,7 @@ jobs:
SUPABASE_DB_URL: ${{ secrets.SUPABASE_DB_URL }}
SUPABASE_URL: ${{ secrets.SUPABASE_URL }}
SUPABASE_KEY: ${{ secrets.SUPABASE_KEY }}
SUPABASE_SECRET_KEY: ${{ secrets.SUPABASE_SECRET_KEY }}
LOGFIRE_TOKEN: ${{ secrets.LOGFIRE_TOKEN }}
run: |
chmod +x .github/scripts/*.sh
Expand Down
2 changes: 1 addition & 1 deletion README.md
Original file line number Diff line number Diff line change
Expand Up @@ -207,7 +207,7 @@ Copy `.env.example` to `.env` and configure. All variables are documented in `.e
|----------|-------------|----------|
| `SUPABASE_URL` | Supabase API URL | Yes |
| `SUPABASE_KEY` | Supabase anon/public key | Yes |
| `SUPABASE_SERVICE_KEY` | Supabase service role key | Yes |
| `SUPABASE_SECRET_KEY` | Supabase secret key | Yes |
| `SUPABASE_DB_URL` | PostgreSQL connection string | Yes |
| `STORAGE_BUCKET` | Supabase storage bucket name | Yes (default: `datasets`) |
| `HUGGING_FACE_TOKEN` | HuggingFace token for dataset downloads | For seeding |
Expand Down
1 change: 1 addition & 0 deletions changelog.d/rename-secret-key.changed
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
Rename SUPABASE_SERVICE_KEY to SUPABASE_SECRET_KEY across codebase, aligning with Supabase's new key naming. Add secret key to Terraform, deploy.yml, and Modal secrets sync.
4 changes: 2 additions & 2 deletions docker-compose.yml
Original file line number Diff line number Diff line change
Expand Up @@ -7,7 +7,7 @@ services:
environment:
SUPABASE_URL: http://supabase_kong_policyengine-api-v2-alpha:8000
SUPABASE_KEY: ${SUPABASE_KEY}
SUPABASE_SERVICE_KEY: ${SUPABASE_SERVICE_KEY}
SUPABASE_SECRET_KEY: ${SUPABASE_SECRET_KEY}
SUPABASE_DB_URL: postgresql://postgres:postgres@supabase_db_policyengine-api-v2-alpha:5432/postgres
LOGFIRE_TOKEN: ${LOGFIRE_TOKEN}
DEBUG: "false"
Expand All @@ -33,7 +33,7 @@ services:
environment:
SUPABASE_URL: http://supabase_kong_policyengine-api-v2-alpha:8000
SUPABASE_KEY: ${SUPABASE_KEY}
SUPABASE_SERVICE_KEY: ${SUPABASE_SERVICE_KEY}
SUPABASE_SECRET_KEY: ${SUPABASE_SECRET_KEY}
SUPABASE_DB_URL: postgresql://postgres:postgres@supabase_db_policyengine-api-v2-alpha:5432/postgres
LOGFIRE_TOKEN: ${LOGFIRE_TOKEN}
volumes:
Expand Down
4 changes: 2 additions & 2 deletions docs/SETUP_RUNBOOK.md
Original file line number Diff line number Diff line change
Expand Up @@ -34,7 +34,7 @@ The defaults in `.env.example` work for local development with Supabase. You onl
| Variable | Where to get it | Required? |
|----------|----------------|-----------|
| `SUPABASE_KEY` | Output of `supabase start` (anon key) | Yes |
| `SUPABASE_SERVICE_KEY` | Output of `supabase start` (service_role key) | Yes |
| `SUPABASE_SECRET_KEY` | Output of `supabase start` (secret key) | Yes |
| `HUGGING_FACE_TOKEN` | [huggingface.co/settings/tokens](https://huggingface.co/settings/tokens) | For seeding datasets |
| `ANTHROPIC_API_KEY` | [console.anthropic.com](https://console.anthropic.com) | For `/agent` endpoint only |
| `LOGFIRE_TOKEN` | [logfire.pydantic.dev](https://logfire.pydantic.dev) | Optional (observability) |
Expand All @@ -45,7 +45,7 @@ The defaults in `.env.example` work for local development with Supabase. You onl
supabase start
```

This starts local Postgres (port 54322), PostgREST (port 54321), and storage. Copy the `anon key` and `service_role key` from the output into your `.env`.
This starts local Postgres (port 54322), PostgREST (port 54321), and storage. Copy the `anon key` and `secret key` (formerly service_role key) from the output into your `.env`.

### Step 4: Initialize database

Expand Down
2 changes: 1 addition & 1 deletion docs/src/app/setup/page.tsx
Original file line number Diff line number Diff line change
Expand Up @@ -57,7 +57,7 @@ export default function SetupPage() {
<pre className="text-[var(--color-text-secondary)]">{`# Supabase (from \`supabase start\` output)
SUPABASE_URL=http://127.0.0.1:54321
SUPABASE_KEY=eyJ...
SUPABASE_SERVICE_KEY=eyJ...
SUPABASE_SECRET_KEY=eyJ...
SUPABASE_DB_URL=postgresql://postgres:postgres@127.0.0.1:54322/postgres

# Storage
Expand Down
6 changes: 3 additions & 3 deletions scripts/init.py
Original file line number Diff line number Diff line change
Expand Up @@ -24,7 +24,7 @@
from sqlmodel import create_engine

from policyengine_api.config.settings import settings
from policyengine_api.services.storage import get_service_role_client
from policyengine_api.services.storage import get_secret_client

console = Console()

Expand All @@ -36,7 +36,7 @@ def reset_storage_bucket():
console.print("[bold blue]Resetting storage bucket...")

try:
supabase = get_service_role_client()
supabase = get_secret_client()
bucket_name = settings.storage_bucket

# Try to delete the bucket (will fail if it doesn't exist)
Expand Down Expand Up @@ -67,7 +67,7 @@ def ensure_storage_bucket():
console.print("[bold blue]Ensuring storage bucket exists...")

try:
supabase = get_service_role_client()
supabase = get_secret_client()
bucket_name = settings.storage_bucket

# Try to get bucket info
Expand Down
2 changes: 1 addition & 1 deletion src/policyengine_api/api/analysis.py
Original file line number Diff line number Diff line change
Expand Up @@ -724,7 +724,7 @@ def _download_dataset_local(filepath: str) -> str:
if cache_path.exists():
return str(cache_path)

client = create_client(settings.supabase_url, settings.supabase_service_key)
client = create_client(settings.supabase_url, settings.supabase_secret_key)
data = client.storage.from_("datasets").download(filepath)

cache_path.parent.mkdir(parents=True, exist_ok=True)
Expand Down
2 changes: 1 addition & 1 deletion src/policyengine_api/config/settings.py
Original file line number Diff line number Diff line change
Expand Up @@ -18,7 +18,7 @@ class Settings(BaseSettings):
# Supabase
supabase_url: str = "http://localhost:54321"
supabase_key: str = ""
supabase_service_key: str = ""
supabase_secret_key: str = ""
supabase_db_url: str = ""

# Worker
Expand Down
16 changes: 8 additions & 8 deletions src/policyengine_api/modal_app.py
Original file line number Diff line number Diff line change
Expand Up @@ -773,7 +773,7 @@ def simulate_economy_uk(simulation_id: str, traceparent: str | None = None) -> N
database_url = get_database_url()
supabase_url = os.environ["SUPABASE_URL"]
supabase_key = os.environ.get(
"SUPABASE_SERVICE_KEY", os.environ["SUPABASE_KEY"]
"SUPABASE_SECRET_KEY", os.environ["SUPABASE_KEY"]
)
storage_bucket = os.environ.get("STORAGE_BUCKET", "datasets")

Expand Down Expand Up @@ -946,7 +946,7 @@ def simulate_economy_us(simulation_id: str, traceparent: str | None = None) -> N
database_url = get_database_url()
supabase_url = os.environ["SUPABASE_URL"]
supabase_key = os.environ.get(
"SUPABASE_SERVICE_KEY", os.environ["SUPABASE_KEY"]
"SUPABASE_SECRET_KEY", os.environ["SUPABASE_KEY"]
)
storage_bucket = os.environ.get("STORAGE_BUCKET", "datasets")

Expand Down Expand Up @@ -1122,7 +1122,7 @@ def economy_comparison_uk(job_id: str, traceparent: str | None = None) -> None:
database_url = get_database_url()
supabase_url = os.environ["SUPABASE_URL"]
supabase_key = os.environ.get(
"SUPABASE_SERVICE_KEY", os.environ["SUPABASE_KEY"]
"SUPABASE_SECRET_KEY", os.environ["SUPABASE_KEY"]
)
storage_bucket = os.environ.get("STORAGE_BUCKET", "datasets")

Expand Down Expand Up @@ -1805,7 +1805,7 @@ def economy_comparison_us(job_id: str, traceparent: str | None = None) -> None:
database_url = get_database_url()
supabase_url = os.environ["SUPABASE_URL"]
supabase_key = os.environ.get(
"SUPABASE_SERVICE_KEY", os.environ["SUPABASE_KEY"]
"SUPABASE_SECRET_KEY", os.environ["SUPABASE_KEY"]
)
storage_bucket = os.environ.get("STORAGE_BUCKET", "datasets")

Expand Down Expand Up @@ -2757,7 +2757,7 @@ def compute_aggregate_uk(aggregate_id: str, traceparent: str | None = None) -> N
database_url = get_database_url()
supabase_url = os.environ["SUPABASE_URL"]
supabase_key = os.environ.get(
"SUPABASE_SERVICE_KEY", os.environ["SUPABASE_KEY"]
"SUPABASE_SECRET_KEY", os.environ["SUPABASE_KEY"]
)
storage_bucket = os.environ.get("STORAGE_BUCKET", "datasets")

Expand Down Expand Up @@ -2913,7 +2913,7 @@ def compute_aggregate_us(aggregate_id: str, traceparent: str | None = None) -> N
database_url = get_database_url()
supabase_url = os.environ["SUPABASE_URL"]
supabase_key = os.environ.get(
"SUPABASE_SERVICE_KEY", os.environ["SUPABASE_KEY"]
"SUPABASE_SECRET_KEY", os.environ["SUPABASE_KEY"]
)
storage_bucket = os.environ.get("STORAGE_BUCKET", "datasets")

Expand Down Expand Up @@ -3064,7 +3064,7 @@ def compute_change_aggregate_uk(
database_url = get_database_url()
supabase_url = os.environ["SUPABASE_URL"]
supabase_key = os.environ.get(
"SUPABASE_SERVICE_KEY", os.environ["SUPABASE_KEY"]
"SUPABASE_SECRET_KEY", os.environ["SUPABASE_KEY"]
)
storage_bucket = os.environ.get("STORAGE_BUCKET", "datasets")

Expand Down Expand Up @@ -3269,7 +3269,7 @@ def compute_change_aggregate_us(
database_url = get_database_url()
supabase_url = os.environ["SUPABASE_URL"]
supabase_key = os.environ.get(
"SUPABASE_SERVICE_KEY", os.environ["SUPABASE_KEY"]
"SUPABASE_SECRET_KEY", os.environ["SUPABASE_KEY"]
)
storage_bucket = os.environ.get("STORAGE_BUCKET", "datasets")

Expand Down
12 changes: 6 additions & 6 deletions src/policyengine_api/services/storage.py
Original file line number Diff line number Diff line change
Expand Up @@ -19,9 +19,9 @@ def get_supabase_client() -> Client:
return create_client(settings.supabase_url, settings.supabase_key)


def get_service_role_client() -> Client:
"""Get Supabase client with service role key for admin operations."""
return create_client(settings.supabase_url, settings.supabase_service_key)
def get_secret_client() -> Client:
"""Get Supabase client with secret key for admin operations."""
return create_client(settings.supabase_url, settings.supabase_secret_key)


def upload_dataset(file_path: str, object_name: str | None = None) -> str:
Expand Down Expand Up @@ -51,7 +51,7 @@ def upload_dataset(file_path: str, object_name: str | None = None) -> str:


def upload_dataset_for_seeding(file_path: str, object_name: str | None = None) -> str:
"""Upload dataset using service role key (for seeding operations).
"""Upload dataset using secret key (for seeding operations).

Args:
file_path: Local path to dataset file
Expand All @@ -60,12 +60,12 @@ def upload_dataset_for_seeding(file_path: str, object_name: str | None = None) -
Returns:
Object name (key) in storage
"""
supabase = get_service_role_client()
supabase = get_secret_client()

if object_name is None:
object_name = Path(file_path).name

# Upload file using service role client
# Upload file using secret client
with open(file_path, "rb") as f:
supabase.storage.from_(settings.storage_bucket).upload(
object_name,
Expand Down
4 changes: 4 additions & 0 deletions terraform/main.tf
Original file line number Diff line number Diff line change
Expand Up @@ -82,6 +82,10 @@ resource "google_cloud_run_v2_service" "api" {
name = "SUPABASE_KEY"
value = var.supabase_key
}
env {
name = "SUPABASE_SECRET_KEY"
value = var.supabase_secret_key
}
env {
name = "SUPABASE_DB_URL"
value = var.supabase_db_url
Expand Down
6 changes: 6 additions & 0 deletions terraform/variables.tf
Original file line number Diff line number Diff line change
Expand Up @@ -28,6 +28,12 @@ variable "supabase_key" {
sensitive = true
}

variable "supabase_secret_key" {
description = "Supabase secret key (admin operations, bypasses RLS)"
type = string
sensitive = true
}

variable "supabase_db_url" {
description = "Supabase PostgreSQL connection URL"
type = string
Expand Down