Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
9 changes: 7 additions & 2 deletions .dockerignore
Original file line number Diff line number Diff line change
@@ -1,8 +1,13 @@
.git
.gitignore
.next
node_modules
exports
coverage
.env
.env.*
!.env.example
npm-debug.log*
Dockerfile
.dockerignore
.git
.gitignore
README.md
11 changes: 8 additions & 3 deletions .env.example
Original file line number Diff line number Diff line change
@@ -1,15 +1,20 @@
DATABASE_URL="postgresql://hcg:hcg_password@localhost:5432/hidden_cost_game?schema=public"
DATABASE_URL="postgresql://hcg:hcg_password_change_me@localhost:5432/hidden_cost_game?schema=public"
APP_BASE_URL="http://localhost:3000"

ENABLE_SERVER_SUBMISSION="false"
NEXT_PUBLIC_ENABLE_SERVER_SUBMISSION="false"

ADMIN_EXPORT_TOKEN="replace-with-a-long-random-secret"
ADMIN_DASHBOARD_PASSWORD="replace-with-a-long-random-password"
ADMIN_EXPORT_TOKEN="change-me-before-production"
ADMIN_DASHBOARD_PASSWORD="change-me-before-production"

SUBMISSION_RATE_LIMIT_WINDOW_MS="60000"
SUBMISSION_RATE_LIMIT_MAX="20"
MAX_SUBMISSION_BODY_BYTES="250000"

CONSENT_VERSION="pilot-consent-v1"
SCHEMA_VERSION="research-export-v1"

# Docker Compose local defaults. Change all secrets before production.
POSTGRES_USER="hcg"
POSTGRES_PASSWORD="hcg_password_change_me"
POSTGRES_DB="hidden_cost_game"
5 changes: 5 additions & 0 deletions Dockerfile
Original file line number Diff line number Diff line change
@@ -1,10 +1,15 @@
FROM node:20-alpine

WORKDIR /app
ENV NODE_ENV=production
ENV NEXT_TELEMETRY_DISABLED=1

COPY package*.json ./
RUN if [ -f package-lock.json ]; then npm ci; else npm install; fi

COPY prisma ./prisma
RUN npm run db:generate

COPY . .
RUN npm run build

Expand Down
45 changes: 45 additions & 0 deletions docker-compose.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,45 @@
services:
app:
build: .
env_file:
- path: .env
required: false
environment:
NODE_ENV: production
NEXT_TELEMETRY_DISABLED: "1"
APP_BASE_URL: ${APP_BASE_URL:-http://localhost:3000}
DATABASE_URL: postgresql://${POSTGRES_USER:-hcg}:${POSTGRES_PASSWORD:-hcg_password_change_me}@postgres:5432/${POSTGRES_DB:-hidden_cost_game}?schema=public
ENABLE_SERVER_SUBMISSION: ${ENABLE_SERVER_SUBMISSION:-true}
NEXT_PUBLIC_ENABLE_SERVER_SUBMISSION: ${NEXT_PUBLIC_ENABLE_SERVER_SUBMISSION:-true}
ADMIN_EXPORT_TOKEN: ${ADMIN_EXPORT_TOKEN:-change-me-before-production}
ADMIN_DASHBOARD_PASSWORD: ${ADMIN_DASHBOARD_PASSWORD:-change-me-before-production}
SUBMISSION_RATE_LIMIT_WINDOW_MS: ${SUBMISSION_RATE_LIMIT_WINDOW_MS:-60000}
SUBMISSION_RATE_LIMIT_MAX: ${SUBMISSION_RATE_LIMIT_MAX:-20}
MAX_SUBMISSION_BODY_BYTES: ${MAX_SUBMISSION_BODY_BYTES:-250000}
CONSENT_VERSION: ${CONSENT_VERSION:-pilot-consent-v1}
SCHEMA_VERSION: ${SCHEMA_VERSION:-research-export-v1}
ports:
- "127.0.0.1:3000:3000"
depends_on:
postgres:
condition: service_healthy
restart: unless-stopped

postgres:
image: postgres:16-alpine
environment:
POSTGRES_USER: ${POSTGRES_USER:-hcg}
POSTGRES_PASSWORD: ${POSTGRES_PASSWORD:-hcg_password_change_me}
POSTGRES_DB: ${POSTGRES_DB:-hidden_cost_game}
volumes:
- postgres_data:/var/lib/postgresql/data
healthcheck:
test: ["CMD-SHELL", "pg_isready -U $${POSTGRES_USER} -d $${POSTGRES_DB}"]
interval: 10s
timeout: 5s
retries: 5
restart: unless-stopped
# Intentionally no ports: entry. Keep Postgres private to the Docker network in production.

volumes:
postgres_data:
141 changes: 141 additions & 0 deletions docs/DATA_COLLECTION.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,141 @@
# Data Collection Guide

This document answers the operational question: **How do I actually get the data from participants?**

## End-to-end flow

1. A participant opens the game URL, gives consent, completes the game, and completes the pre/post reveal measures.
2. At the end, the participant can optionally submit an anonymous session export.
3. The browser sends the export JSON to `POST /api/submissions` when server submissions are enabled.
4. The API validates the payload and stores it in the `ResearchSubmission` table in Postgres.
5. The researcher opens `/admin` or uses `curl` with `ADMIN_EXPORT_TOKEN`.
6. The researcher downloads CSV or JSON for analysis.

`POST /api/research-submissions` is an alias for the same submission handler.

## Before collecting real data

Set these values in `.env` and restart the app:

```bash
ENABLE_SERVER_SUBMISSION="true"
NEXT_PUBLIC_ENABLE_SERVER_SUBMISSION="true"
ADMIN_EXPORT_TOKEN="replace-with-a-long-random-secret"
ADMIN_DASHBOARD_PASSWORD="replace-with-a-long-random-password"
APP_BASE_URL="https://your-domain.com"
```

Run migrations before collecting data:

```bash
docker compose exec app npm run db:migrate
```

## Confirm submissions are stored

Health check:

```bash
curl http://127.0.0.1:3000/api/health
```

Admin stats:

```bash
curl -H "Authorization: Bearer $ADMIN_EXPORT_TOKEN" \
"$APP_BASE_URL/api/admin/stats"
```

The stats response includes `totalSubmissions`, `completedSubmissions`, treatment-condition counts, and averaged computed metrics.

You can also inspect the database directly from Docker:

```bash
docker compose exec postgres psql -U hcg -d hidden_cost_game -c 'SELECT id, "sessionId", "submittedAt", "completedGameRounds" FROM "ResearchSubmission" ORDER BY submittedAt DESC LIMIT 10;'
```

If you changed `POSTGRES_USER` or `POSTGRES_DB`, update the command.

## Export CSV

From a browser, sign in to `/admin` with `ADMIN_DASHBOARD_PASSWORD` and use the CSV download control.

From the command line:

```bash
curl -H "Authorization: Bearer $ADMIN_EXPORT_TOKEN" \
"$APP_BASE_URL/api/admin/submissions.csv" \
-o submissions.csv
```

The CSV flattens key fields such as assignment condition, game outcomes, computed metrics, pre/post survey values, and participant background fields.

## Export JSON

Use the built-in script:

```bash
docker compose exec app npm run export:submissions
```

Or use `curl`:

```bash
curl -H "Authorization: Bearer $ADMIN_EXPORT_TOKEN" \
"$APP_BASE_URL/api/admin/submissions?limit=500" \
-o submissions.json
```

The JSON export includes database metadata plus each original submission payload. If the response includes `nextCursor`, request the next page with `?cursor=<nextCursor>`.

## Interpret completeness flags

Each stored payload includes a `completeness` object. The most important fields are:

- `isComplete` — `true` means the participant reached the expected complete export state.
- `completedGameRounds` — number of game rounds completed.
- Additional missing-section flags, if present in the payload, identify which survey or game sections were incomplete.

For primary analyses, start with `completeness.isComplete === true`. Keep incomplete submissions for audit and attrition checks unless your study protocol says otherwise.

## Remove test submissions manually

Back up first. Then delete only the rows you have identified as test data.

Create a backup:

```bash
mkdir -p backups
docker compose exec -T postgres pg_dump -U hcg -d hidden_cost_game > backups/before-test-delete-$(date +%Y%m%d-%H%M%S).sql
```

List recent rows:

```bash
docker compose exec postgres psql -U hcg -d hidden_cost_game -c 'SELECT id, "sessionId", "submittedAt" FROM "ResearchSubmission" ORDER BY submittedAt DESC LIMIT 20;'
```

Delete one known test row by id:

```bash
docker compose exec postgres psql -U hcg -d hidden_cost_game -c 'DELETE FROM "ResearchSubmission" WHERE id = '\''paste-test-submission-id-here'\'';'
```

Delete multiple known test rows by id:

```bash
docker compose exec postgres psql -U hcg -d hidden_cost_game -c 'DELETE FROM "ResearchSubmission" WHERE id IN ('\''id-one'\'', '\''id-two'\'');'
```

Avoid broad date-range deletes unless you have already exported and verified the affected rows.

## Backup before deleting anything

Use `pg_dump` before any manual deletion:

```bash
mkdir -p backups
docker compose exec -T postgres pg_dump -U hcg -d hidden_cost_game > backups/hidden_cost_game-$(date +%Y%m%d-%H%M%S).sql
```

Store important backups somewhere other than the VPS, such as encrypted object storage or an institutional backup location.
Loading