Add Google Sheets Apps Script mirror and update webhook payload

README.md

@@ -172,70 +172,18 @@ For Docker Compose, `.env.example` also includes `POSTGRES_USER`, `POSTGRES_PASS
 
 ## Optional Google Sheets mirror
 
-PostgreSQL remains the primary source of truth for completed research submissions. You can also mirror a flattened summary row to a Google Sheet by deploying a Google Apps Script web app and setting `GOOGLE_SHEETS_WEBHOOK_URL`. If the Sheets webhook is missing, slow, invalid, or returns an error, `POST /api/submissions` still returns success to the participant after the PostgreSQL write succeeds; the webhook result is only logged server-side.
+PostgreSQL remains the primary source of truth for completed research submissions. You can also mirror a flattened summary row to a Google Sheet by deploying the ready-to-copy Google Apps Script receiver in [`docs/google-sheets-apps-script.js`](docs/google-sheets-apps-script.js) and setting `GOOGLE_SHEETS_WEBHOOK_URL`. If the Sheets webhook is missing, slow, invalid, or returns an error, `POST /api/submissions` still returns success to the participant after the PostgreSQL write succeeds; the webhook result is only logged server-side.
 
 Setup steps:
 
-1. Create a Google Sheet and add a sheet tab named `Submissions`.
-2. Open **Extensions → Apps Script**, create a web app script, and set a script property named `WEBHOOK_SECRET`.
-3. Deploy the Apps Script as a web app and copy its `/exec` URL into `.env` as `GOOGLE_SHEETS_WEBHOOK_URL`; set the same secret in `GOOGLE_SHEETS_WEBHOOK_SECRET`.
-4. Restart the app so the server process reads the new environment variables.
-5. Submit one complete test session and verify both the app logs and the new Google Sheet row.
-
-A minimal Apps Script receiver that checks the shared secret from the JSON body is shown below. The app also sends an `Authorization: Bearer ...` header, but Apps Script web apps do not always expose request headers consistently, so this example validates `data.secret` and deliberately does not append the secret to the sheet row.
-
-```javascript
-function doPost(e) {
-  const expectedSecret = PropertiesService.getScriptProperties().getProperty("WEBHOOK_SECRET");
-  const data = JSON.parse(e.postData.contents || "{}");
-
-  if (!expectedSecret || data.secret !== expectedSecret) {
-    return ContentService
-      .createTextOutput(JSON.stringify({ ok: false, error: "unauthorized" }))
-      .setMimeType(ContentService.MimeType.JSON);
-  }
-
-  const sheet = SpreadsheetApp.getActiveSpreadsheet().getSheetByName("Submissions");
-  if (!sheet) {
-    return ContentService
-      .createTextOutput(JSON.stringify({ ok: false, error: "missing Submissions sheet" }))
-      .setMimeType(ContentService.MimeType.JSON);
-  }
-
-  sheet.appendRow([
-    new Date(),
-    data.serverSubmissionId,
-    data.submittedAt,
-    data.sessionId,
-    data.schemaVersion,
-    data.exportVersion,
-    data.assignedDisplayedProfile,
-    data.assignedHiddenProfile,
-    data.finalFinancialScore,
-    data.finalHealthScore,
-    data.fullTreatmentChoices,
-    data.partialTreatmentChoices,
-    data.skippedTreatmentChoices,
-    data.responsibilityShift,
-    data.constraintRecognitionShift,
-    data.protestLegitimacyShift,
-    data.ruleCorrectionSupportShift,
-    data.redistributionSupportShift,
-    data.revisionCondition,
-    data.attemptedPreRevealRevision,
-    data.usedRevisionOpportunity,
-    data.revealTimingCondition,
-    data.costVisibilityCondition,
-    data.explanationFrameCondition,
-    data.replayCompleted,
-    data.replayAssignmentCondition
-  ]);
-
-  return ContentService
-    .createTextOutput(JSON.stringify({ ok: true }))
-    .setMimeType(ContentService.MimeType.JSON);
-}
-```
+1. Create a Google Sheet. The script will create a `Submissions` tab if it does not already exist, and it will write the expected header row if row 1 is empty.
+2. Open **Extensions → Apps Script**, paste the contents of [`docs/google-sheets-apps-script.js`](docs/google-sheets-apps-script.js) into `Code.gs`, and save.
+3. In **Project Settings → Script properties**, add `WEBHOOK_SECRET` with the same long random value you will set as `GOOGLE_SHEETS_WEBHOOK_SECRET` in the app.
+4. Deploy the Apps Script as a web app and copy its `/exec` URL into `.env` as `GOOGLE_SHEETS_WEBHOOK_URL`; set the same secret in `GOOGLE_SHEETS_WEBHOOK_SECRET`.
+5. Restart the app so the server process reads the new environment variables.
+6. Submit one complete test session and verify both the app logs and the new Google Sheet row.
+
+The app sends the shared secret in the JSON body as `secret` because Google Apps Script web apps do not always expose request headers consistently. The provided script validates `data.secret` but deliberately excludes it from the `HEADERS` array so the secret is never appended to the sheet.
 
 ## Server pilot mode with Docker
 

docs/VPS_DEPLOYMENT.md

@@ -119,7 +119,7 @@ Required production values:
 
 Optional Google Sheets values:
 
-- `GOOGLE_SHEETS_WEBHOOK_URL`: optional Google Apps Script web app URL for mirroring successful submissions to a Google Sheet.
+- `GOOGLE_SHEETS_WEBHOOK_URL`: optional Google Apps Script web app URL for mirroring successful submissions to a Google Sheet. Use the ready-to-copy receiver in `docs/google-sheets-apps-script.js`.
 - `GOOGLE_SHEETS_WEBHOOK_SECRET`: optional shared secret used by the app and Apps Script webhook. Recommended when the Sheets webhook is enabled.
 
 Save and exit nano with `Ctrl+O`, `Enter`, then `Ctrl+X`.
@@ -326,6 +326,6 @@ After deployment and HTTPS setup, test the full production path:
 5. Enter `ADMIN_EXPORT_TOKEN`.
 6. Export CSV and JSON.
 7. Verify the exported files contain the test submission.
-8. If `GOOGLE_SHEETS_WEBHOOK_URL` is enabled, verify a new row appears in the Google Sheet.
+8. If `GOOGLE_SHEETS_WEBHOOK_URL` is enabled, verify a new row appears in the Google Sheet. The recommended Apps Script receiver is in `docs/google-sheets-apps-script.js`; it creates the `Submissions` headers on first run and rejects requests whose JSON body does not contain the matching `secret`.
 
 Keep the test export and backup notes with your deployment records so future updates can be checked the same way.

docs/google-sheets-apps-script.js

@@ -0,0 +1,120 @@
+/**
+ * Google Apps Script receiver for the Hidden Cost Game Google Sheets mirror.
+ *
+ * Setup:
+ * 1. Create a Google Sheet.
+ * 2. Open Extensions -> Apps Script.
+ * 3. Paste this file into Code.gs.
+ * 4. In Project Settings -> Script properties, add WEBHOOK_SECRET with the
+ *    same value as GOOGLE_SHEETS_WEBHOOK_SECRET in your app deployment.
+ * 5. Deploy as a web app that can receive POST requests.
+ */
+const SHEET_NAME = "Submissions";
+
+const HEADERS = [
+  "receivedAt",
+  "serverSubmissionId",
+  "submittedAt",
+  "sessionId",
+  "schemaVersion",
+  "exportVersion",
+  "assignedDisplayedProfile",
+  "assignedHiddenProfile",
+  "finalFinancialScore",
+  "finalHealthScore",
+  "fullTreatmentChoices",
+  "partialTreatmentChoices",
+  "skippedTreatmentChoices",
+  "responsibilityShift",
+  "constraintRecognitionShift",
+  "protestLegitimacyShift",
+  "ruleCorrectionSupportShift",
+  "redistributionSupportShift",
+  "revisionCondition",
+  "attemptedPreRevealRevision",
+  "usedRevisionOpportunity",
+  "revealTimingCondition",
+  "costVisibilityCondition",
+  "explanationFrameCondition",
+  "replayCompleted",
+  "replayAssignmentCondition",
+  "memoryDistortionMagnitude",
+  "rememberedPrimaryAttributionMatchesOriginal",
+];
+
+function doPost(e) {
+  try {
+    const data = parseJsonBody_(e);
+    const expectedSecret = PropertiesService.getScriptProperties().getProperty("WEBHOOK_SECRET");
+
+    if (!expectedSecret) {
+      return jsonResponse_({ ok: false, error: "WEBHOOK_SECRET script property is not configured" });
+    }
+
+    if (!data || data.secret !== expectedSecret) {
+      return jsonResponse_({ ok: false, error: "unauthorized" });
+    }
+
+    const sheet = getOrCreateSheet_();
+    ensureHeaders_(sheet);
+
+    const row = HEADERS.map((header) => normalizeCellValue_(header === "receivedAt" ? data[header] || new Date().toISOString() : data[header]));
+    sheet.appendRow(row);
+
+    return jsonResponse_({ ok: true });
+  } catch (error) {
+    return jsonResponse_({ ok: false, error: error && error.message ? error.message : "unknown error" });
+  }
+}
+
+function parseJsonBody_(e) {
+  const contents = e && e.postData && typeof e.postData.contents === "string" ? e.postData.contents : "";
+  if (!contents) {
+    throw new Error("empty request body");
+  }
+
+  try {
+    const parsed = JSON.parse(contents);
+    if (!parsed || typeof parsed !== "object" || Array.isArray(parsed)) {
+      throw new Error("JSON body must be an object");
+    }
+    return parsed;
+  } catch (error) {
+    throw new Error("invalid JSON body");
+  }
+}
+
+function getOrCreateSheet_() {
+  const spreadsheet = SpreadsheetApp.getActiveSpreadsheet();
+  if (!spreadsheet) {
+    throw new Error("no active spreadsheet found");
+  }
+
+  return spreadsheet.getSheetByName(SHEET_NAME) || spreadsheet.insertSheet(SHEET_NAME);
+}
+
+function ensureHeaders_(sheet) {
+  const firstRowValues = sheet.getRange(1, 1, 1, HEADERS.length).getValues()[0];
+  const isHeaderRowEmpty = firstRowValues.every((value) => value === "");
+
+  if (isHeaderRowEmpty) {
+    sheet.getRange(1, 1, 1, HEADERS.length).setValues([HEADERS]);
+    sheet.setFrozenRows(1);
+  }
+}
+
+function normalizeCellValue_(value) {
+  if (value === undefined || value === null) {
+    return "";
+  }
+
+  if (typeof value === "object") {
+    return JSON.stringify(value);
+  }
+
+  return value;
+}
+
+function jsonResponse_(body) {
+  return ContentService.createTextOutput(JSON.stringify(body)).setMimeType(ContentService.MimeType.JSON);
+}

lib/googleSheetsWebhook.ts

@@ -1,7 +1,6 @@
 import type { ValidResearchExport } from "@/lib/researchExportSchema";
 
 const WEBHOOK_TIMEOUT_MS = 5_000;
-const MAX_PAYLOAD_JSON_BYTES = 100_000;
 
 type SubmissionWebhookMetadata = {
   serverSubmissionId: string;
@@ -10,7 +9,7 @@ type SubmissionWebhookMetadata = {
 
 type GoogleSheetsSubmissionRow = {
   secret?: string;
-  type: "hidden-cost-game-submission";
+  receivedAt: string;
   serverSubmissionId: string;
   submittedAt: string;
   sessionId: string;
@@ -36,7 +35,8 @@ type GoogleSheetsSubmissionRow = {
   explanationFrameCondition: string | null;
   replayCompleted: boolean;
   replayAssignmentCondition: string | null;
-  payloadJson?: string;
+  memoryDistortionMagnitude: number | null;
+  rememberedPrimaryAttributionMatchesOriginal: boolean | null;
 };
 
 export async function sendSubmissionToGoogleSheets(
@@ -91,7 +91,7 @@ function buildSubmissionRow(
 ): GoogleSheetsSubmissionRow {
   const row: GoogleSheetsSubmissionRow = {
     secret: process.env.GOOGLE_SHEETS_WEBHOOK_SECRET?.trim() || undefined,
-    type: "hidden-cost-game-submission",
+    receivedAt: new Date().toISOString(),
     serverSubmissionId: metadata.serverSubmissionId,
     submittedAt: metadata.submittedAt instanceof Date ? metadata.submittedAt.toISOString() : metadata.submittedAt,
     sessionId: payload.sessionId,
@@ -118,12 +118,10 @@ function buildSubmissionRow(
     explanationFrameCondition: payload.explanationFrameCondition?.condition ?? null,
     replayCompleted: payload.computedMetrics.replayCompleted,
     replayAssignmentCondition: payload.computedMetrics.replayAssignmentCondition ?? payload.replayGame?.assignmentCondition ?? null,
+    memoryDistortionMagnitude: payload.computedMetrics.memoryDistortionMagnitude ?? null,
+    rememberedPrimaryAttributionMatchesOriginal:
+      payload.computedMetrics.rememberedPrimaryAttributionMatchesOriginal ?? null,
   };
 
-  const payloadJson = JSON.stringify(payload);
-  if (Buffer.byteLength(payloadJson, "utf8") <= MAX_PAYLOAD_JSON_BYTES) {
-    row.payloadJson = payloadJson;
-  }
-
   return row;
 }