feat(posthog-tools): dashboards-as-code pipeline + developer-funnel dashboard#311
Merged
Conversation
Decomposes the original analytics-foundation workstream into four sub-specs (1A-1D). 1A: PostHog dashboards-as-code pipeline, one sample dashboard, full test coverage, Nx-affected CI gate. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Updates gtm.md §6/§7 and meta-spec §6 to reflect the 4-spec decomposition agreed during Spec 1A brainstorm. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
…ools Devdeps for the Spec 1A dashboards-as-code pipeline. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
namedInputs.taxonomy includes docs/gtm/taxonomy.md so taxonomy edits mark this project as affected. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Commits the generated posthog-api.gen.ts. Regenerate quarterly via nx run posthog-tools:generate-types. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
6 tests covering required key, host default, project id coercion, and rejection of missing/short/non-numeric values. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Schemas validate slug format, required fields, funnel-specific constraints. Fixture validator parses every committed JSON file. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Three-attempt retry with exponential backoff (500ms, 1s, 2s, max 8s) for transient PostHog errors. Lazy singleton via ph(). Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Loads local JSON, validates via zod, matches against remote by posthog_id (or unique name fallback), classifies into create/update/orphan. Ambiguous name matches force create. 6 tests covering match-by-id, name fallback, ambiguous case, orphan detection, invalid JSON error handling. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Apply order: cohorts → insights → dashboards. Creates before updates within a tier. Atomic writeback via temp-file + rename. Partial success tracked. Orphans only deleted with deleteOrphans:true. 6 new tests cover apply order, writeback persistence, slug resolution failure, partial success, dry-run no-write, orphan protection. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
CLI accepts --plan, --apply, --apply --delete-orphans. Adapter wraps openapi-fetch in the SyncClient interface used by tests. Direct-run guard prevents tests from triggering main(). Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
The prior filter (kind !== 'cohort' || path) accepted every create because cohorts also have a path. Replaced with a failure-set check so the "commit the writeback" hint only lists items that actually succeeded. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
7 tests cover sparkline edge cases (empty, all-zero, normalization), delta-cell formatting (new, percent, negative), and markdown structure. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
generateReport() pulls dashboards tagged 'gtm', queries each tile's insight, folds daily counts into 4 weekly buckets. Output goes to docs/gtm/reports/<date>-weekly[-N].md (never overwrites). Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
The round-trip proof for Spec 1A. Four insights: pageviews-by-landing, install-command-clicks, cockpit-recipe-completion (zero until Spec 1C), six-signal-activation-funnel (zero until Spec 1C). Empty cohorts/.gitkeep since cohorts come after data flows. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Every event referenced in any insight JSON must appear in docs/gtm/taxonomy.md. This test stays green permanently and catches event renames or insight drift. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
npm run posthog:sync/apply/report/generate-types as thin Nx aliases. .env.example documents POSTHOG_PERSONAL_API_KEY, POSTHOG_HOST, POSTHOG_PROJECT_ID. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
New posthog-sync-plan job runs `nx run posthog-tools:sync:plan` on PRs that affect posthog-tools (per nx show projects --affected). Soft-skips when POSTHOG_PERSONAL_API_KEY secret is absent (fork PRs). Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Drops speculative JSON Schema references (zod-only), updates CLI commands to nx run posthog-tools:*, documents POSTHOG_PERSONAL_API_KEY, adds rename procedure, links to the spec. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
- schema.spec.ts: replace node:fs/promises.glob (Node 22+) with readdir filter. CI runs Node 20; the original test would have failed. - report.ts: add expectOk() helper that throws on PostHog HTTP errors instead of silently returning empty results. A 401 now fails the weekly snapshot rather than producing a misleading zero-row report. Fixes flagged in final code review. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
First --apply against PostHog project 406826 succeeded. Records the PostHog-assigned ids so future syncs match by posthog_id, not by name. applied: 5, failed: 0 Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
PostHog's API doesn't accept the tiles field on dashboard create/update — it's read-only. Insights are associated with dashboards via the insight's `dashboards: number[]` field, set via PATCH. Changes: - Drop tiles from dashboard create/update body (stripTiles helper). - Keep tile-slug resolution as early validation (unknown slug still throws). - Add a wiring pass after all tiers: for each local dashboard, PATCH each referenced insight with its full desired dashboards list. Idempotent. - 2 new tests pin the new behavior (35 total). Discovered when first --apply succeeded structurally but PostHog showed tile count of 0 on the new dashboard. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
…hape
PostHog's API expects insight queries inside a nested `filters` object,
not flat at the top level. Our zod schemas describe local intent
(slug/kind/events/breakdown/date_from); PostHog needs
filters: { insight: 'TRENDS'|'FUNNELS', events: [...], breakdown,
breakdown_type, date_from, interval, funnel_window_interval, ... }.
Without this mapping, PostHog silently discards our query body and
stores blank-filter placeholder insights — the dashboard renders but
every tile is empty.
Adds toPostHogInsight() and toPostHogDashboard() with 3 new mapper
tests (38 total). Replaces the spread-of-item.local in createInsight /
updateInsight / createDashboard / updateDashboard.
Discovered when API verification showed tile filters all reduced to
{ insight: 'TRENDS', date_from: null, date_to: null } after first
--apply.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
PostHog rejected our legacy filters body with:
"Creating or updating insights with legacy filters is not available
for this user."
Modern PostHog projects require the InsightVizNode / HogQL query
schema:
query: {
kind: 'InsightVizNode',
source: {
kind: 'TrendsQuery' | 'FunnelsQuery',
series: [{ kind: 'EventsNode', event, math, ... }],
dateRange: { date_from },
interval, breakdownFilter, funnelsFilter, ...
}
}
Rewrote toPostHogInsight() accordingly. Updated the two mapper tests
to assert against the new schema shape.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
TypeScript incremental cache, generated by tsc --noEmit during local test runs. Not portable across machines. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
|
The latest updates on your projects. Learn more about Vercel for GitHub.
|
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Implements Spec 1A (analytics-foundation sub-spec A): the PostHog dashboards-as-code pipeline. Round-trip verified end-to-end against PostHog project 406826.
posthog-toolsattools/posthog/openapi-fetch+ 107k LOC of generated types from PostHog's OpenAPI specsyncCLI with--plan(read-only diff) and--apply(idempotent upsert + writeback + insight→dashboard wiring pass) and--apply --delete-orphans(explicit deletion only)reportCLI: pulls dashboards taggedgtm, renders weekly markdown snapshot with sparklines, fails loud on transport errorsInsightLocalzod schema to PostHog's modernquery/HogQL body shape (InsightVizNode+TrendsQuery/FunnelsQuery); legacyfiltersshape is rejected by modern PostHog accountsdeveloper-funnel+ 4 insights — round-trip verified to render with correct query shapes on production PostHogenv.spec.ts,schema.spec.ts,sync.spec.ts,report.spec.ts,taxonomy.spec.tsdocs/gtm/taxonomy.md. Stays green on every CI run; catches event renames or insight drift.posthog-sync-planwith Nx-affected gating + soft-skip on missing secret (works for fork PRs)gtm.md §6/§7+ meta-spec §6 now reflect 1A–1D sub-specsSpec: docs/superpowers/specs/gtm/2026-05-14-analytics-foundation-1a-dashboards-as-code-design.md
Plan: docs/superpowers/plans/gtm/2026-05-14-analytics-foundation-1a-dashboards-as-code.md
End-to-end verification
Manually ran the full pipeline against PostHog project 406826 ("Angular Agent Framework"):
```
$ npm run posthog:sync # --plan
[create] dashboard developer-funnel
[create] insight cockpit-recipe-completion
[create] insight install-command-clicks
[create] insight pageviews-by-landing
[create] insight six-signal-activation-funnel
[orphan] ... (PostHog default-onboarding artifacts, correctly NOT auto-deleted)
$ npm run posthog:apply
applied: 5, failed: 0
Writeback complete.
```
API verification of the resulting dashboard (https://us.posthog.com/project/406826/dashboard/1582272):
```
Dashboard: GTM · Developer funnel | tiles: 4
cockpit:transport_connected, cockpit:chat_first_message, cockpit:thread_persisted,
cockpit:interrupt_handled, cockpit:generative_component_rendered
```
Bugs caught + fixed during end-to-end verification
The "test thoroughly" loop caught three real bugs that would have shipped broken otherwise:
tileson dashboard create/update. Fixed by adding a wiring pass that PATCHes each insight with its desireddashboards: [<id>]list.toPostHogInsight()/toPostHogDashboard()mappers producingInsightVizNode/TrendsQuery/FunnelsQuery.node:fs/promises.globrequires Node 22; CI runs Node 20. Replaced withreaddir + filter.Test Plan
posthog-sync-plan(Nx-affected gate green)npm run posthog:syncshows current PostHog state (5 already-synced, 8 orphan)npm run posthog:applyis idempotent — re-running produces `applied: 5, failed: 0` with no side effectsGTM · Developer funneldashboard renders 4 tilesnx run posthog-tools:test)Known limitations (deferred to follow-ups, not blocking)
POSTHOG_PERSONAL_API_KEYin CI — production hardening TODO documented intools/posthog/README.mdto migrate to a read-only Personal API Key for CItools/posthog/cohorts/ships empty with.gitkeep— cohorts come after data flowsCleanup needed in PostHog after merge
The PostHog project has 8 orphan dashboards/insights/cohorts from default onboarding ("My App Dashboard", "Daily active users", "Internal / Test users", etc.). These are correctly not auto-deleted. Decide whether to:
npm run posthog:apply -- -- --delete-orphans(deletes them explicitly), or🤖 Generated with Claude Code