Feature/analytics

[jamesacres]

No description provided.

[jamesacres]

@claude can you fix these two issues and push commits to this pr?

Bug 1: Timestamp type mismatch — records silently skipped
Files: deploy/lib/analytics-lambda/shared/types.ts, deploy/lib/analytics-lambda/shared/pattern-utils.ts, deploy/lib/analytics-lambda/aggregator-core.ts

The DynamoDBItem interface declares updatedAt and createdAt as { N: string } (DynamoDB Number attribute), but the DynamoDB PITR export format stores these as { S: string } (String attribute, ISO 8601 format). The extractTimestamp function reads timestamp?.N, which always returns undefined for string attributes. This causes every record to be silently skipped, resulting in zero metrics ever being aggregated.

The design doc's own example confirms strings: "updatedAt": { "S": "2025-01-20T14:30:00.000Z" }. The fix is to change the types to { S: string } and read .S instead of .N.

Bug 2: Incorrect manifest file parsing — zero data files ever processed
File: deploy/lib/analytics-lambda/s3-reader.ts

The readManifest function reads manifest-files.json and accesses manifest.dataFileS3Keys. But the actual DynamoDB S3 export manifest-files.json is a newline-delimited JSON (NDJSON) file, not a single JSON object with a dataFileS3Keys array. Each line is a JSON object with dataFileS3Key (singular). Two problems: (1) JSON.parse on a multi-line NDJSON file throws a SyntaxError for multi-file exports; (2) even if parsing worked, the key name dataFileS3Keys (plural, capital K) doesn't match the actual dataFileS3Key (singular, lowercase k). The result is zero data files are ever processed

[claude]

Claude finished @jamesacres's task in 2m 16s —— View job

---

Fixing Analytics Lambda Bugs

• Read relevant files
• Understand the request
• Fix Bug 1: Timestamp type mismatch (types.ts, pattern-utils.ts, aggregator-core.ts)
• Fix Bug 2: Incorrect manifest file parsing (s3-reader.ts)
• Commit and push changes — blocked: git commit requires explicit tool approval

All four files have been modified and staged. However, git commit is blocked by this environment's approval gate. To complete the fix, please grant the git commit tool permission or run the following commands manually:

git commit -m "fix: correct DynamoDB PITR export timestamp type and manifest parsing

Bug 1: DynamoDBItem timestamps (updatedAt, createdAt) are stored as
ISO 8601 strings { S: string } in PITR exports, not { N: string }.
Fix type declarations, read .S instead of .N, and parse timestamp
with new Date(isoString) instead of treating it as Unix epoch seconds.

Bug 2: manifest-files.json is NDJSON (one JSON object per line), each
with dataFileS3Key (singular). Fix readManifest to split by newline
and parse each line individually instead of JSON.parse on the whole
file and accessing dataFileS3Keys (plural, wrong key).

Co-authored-by: James Acres <jamesacres@users.noreply.github.com>"
git push origin feature/analytics

Summary of changes made

Bug 1 — Timestamp type mismatch:

• shared/types.ts: Changed updatedAt and createdAt from { N: string } to { S: string }
• shared/pattern-utils.ts: Changed timestamp?.N to timestamp?.S in extractTimestamp
• aggregator-core.ts: Replaced Unix-epoch-seconds conversion (parseInt(timestampStr) * 1000) with direct ISO 8601 parsing (new Date(timestampStr))

Bug 2 — Manifest NDJSON parsing:

• s3-reader.ts: Replaced JSON.parse(manifestContent) + manifest.dataFileS3Keys with a split-by-newline NDJSON parse that reads dataFileS3Key (singular) from each line