Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
36 commits
Select commit Hold shift + click to select a range
018baa6
Add join-e2e package scaffold
conatus Mar 18, 2026
58a25e7
Add E2E seed scripts
conatus Mar 18, 2026
765406b
Add Phase 1 rendering tests
conatus Mar 18, 2026
3d489f8
Add Phase 2 form progression tests
conatus Mar 18, 2026
a6f344c
Add Phase 3 free membership happy path tests
conatus Mar 18, 2026
f6cfe6d
Add GitHub Actions E2E workflow
conatus Mar 18, 2026
7e5960a
Add build:e2e script to join-flow
conatus Mar 18, 2026
15d54ec
Update join-flow lock files after adding build:e2e script
conatus Mar 18, 2026
f0ab6dd
Add package-lock.json for join-e2e
conatus Mar 18, 2026
603077a
Fix 2.2 validation assertions
conatus Mar 18, 2026
9dcea1b
Simplify 2.2i country test to field-presence check
conatus Mar 18, 2026
c2fb813
Add e2e seed pages for donation upsell and supporter mode scenarios
conatus Mar 28, 2026
b286144
Add shared e2e test helpers (mockRestEndpoints, injectEnvOverrides, c…
conatus Mar 28, 2026
e95ecf4
Add Phase 4 donation upsell e2e tests (PR #59 sections 2, 3, 4)
conatus Mar 28, 2026
c12ac13
Add Phase 5 supporter mode monthly e2e tests (PR #59 sections 5, 7)
conatus Mar 28, 2026
22b1043
Add Phase 6 supporter mode one-off e2e tests (PR #59 sections 8, 8a)
conatus Mar 28, 2026
d0e416c
Add Phase 7 supporter mode edge cases and product naming e2e tests (P…
conatus Mar 28, 2026
bb615d2
Add PHPUnit tests for Mailchimp failure non-fatal behaviour in JoinSe…
conatus Mar 28, 2026
1c479e3
Replace PR references in e2e test comments with self-contained descri…
conatus Mar 28, 2026
eb7352d
Fix wp-env core to use wordpress.org zip URL
conatus Mar 28, 2026
4a943b7
Fix getProvidedStateFromQueryParams clobbering session state via sche…
conatus Mar 28, 2026
4a7c44a
Fix DONATION_SUPPORTER_MODE clobbering one-off recurDonation on Strip…
conatus Mar 28, 2026
c551a1c
Fix strict mode violations: qualify tier button selectors with type="…
conatus Mar 28, 2026
22f3525
Remove phase numbering from e2e test file headers
conatus Mar 28, 2026
2a9ce45
Remove numeric section labels from e2e test describe and test names
conatus Mar 28, 2026
9f3cadd
Add e2e testing plan and link from README
conatus Mar 29, 2026
f0bb07e
Remove numeric prefixes from e2e spec filenames
conatus Mar 29, 2026
0a50825
Remove section number comments from supporter-mode-edge-cases spec
conatus Mar 29, 2026
ae2b640
Merge branch 'master' into feature/join-e2e-join-124-tests
conatus Mar 29, 2026
aa7dd3f
Rename build:e2e to test:build and stop running CI on feature branch …
conatus Mar 30, 2026
3309531
Ignore playwright-report and test-results directories
conatus Mar 30, 2026
9b4f66a
Update WordPress test environment to 6.9.4
conatus Mar 30, 2026
bd25bbb
Update e2e testing docs: fix name, expand env injection note, add WP …
conatus Mar 30, 2026
312092e
Replace waitForTimeout race condition in captureJoinBodyViaStripeRedi…
conatus Mar 30, 2026
e103300
Remove duplicate CONTINUE and mockRestEndpoints from form-progression…
conatus Mar 30, 2026
4ca62a6
Add guard assertions and fix weak assertion in captureJoinBodyViaStri…
conatus Mar 30, 2026
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
53 changes: 53 additions & 0 deletions .github/workflows/e2e.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,53 @@
name: E2E Tests

on:
push:
branches:
- master
pull_request:

jobs:
e2e:
runs-on: ubuntu-latest

steps:
- uses: actions/checkout@v4

- uses: actions/setup-node@v4
with:
node-version: '20'

- name: Install root deps
run: yarn install

- name: Build join-flow bundle (E2E mode)
working-directory: packages/join-flow
run: npm run test:build

- name: Install E2E deps
working-directory: packages/join-e2e
run: npm ci

- name: Install Playwright browsers
working-directory: packages/join-e2e
run: npx playwright install --with-deps chromium

- name: Start wp-env
working-directory: packages/join-e2e
run: npx wp-env start

- name: Seed WordPress test environment
working-directory: packages/join-e2e
run: |
npx wp-env run tests-cli wp eval-file /var/www/html/wp-content/e2e-scripts/setup.php
npx wp-env run tests-cli wp rewrite flush --hard

- name: Run E2E tests
working-directory: packages/join-e2e
run: npm test -- --reporter=list,html

- uses: actions/upload-artifact@v4
if: failure()
with:
name: playwright-report
path: packages/join-e2e/playwright-report/
270 changes: 270 additions & 0 deletions docs/e2e-testing.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,270 @@
# End-to-End Testing Plan

## Overview

Join ships with a Playwright end-to-end test suite that exercises the React join flow inside a real WordPress environment. Tests run against a wp-env (Docker WordPress) instance seeded with purpose-built test pages.

### Test approach

The suite validates the join form's UI behaviour and the data contracts it sends to the backend. It does not exercise live payment provider APIs. Instead, it uses helpers that simulate the outcome of payment flows (redirect-based success) and captures the `/join` request body to assert correctness.

This is a deliberate trade-off: payment provider SDKs require live credentials, load scripts from CDNs, and introduce external redirect flows that are fragile in CI. By mocking the REST layer and simulating payment completion, the tests stay fast, deterministic, and credential-free while still covering the most important contract: what the frontend sends to the backend.
Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This should also note that the patching the Javascript environment script block means that the WordPress database is not truly in the correct format. To do this, we would need to simulate operation of the backend, or injection of the result of these changes into the database. Which would be good for true end to end tests, but is good enough for now.


### Infrastructure

| Component | Location | Purpose |
|-----------|----------|---------|
| Test suite | `packages/join-e2e/tests/` | Playwright spec files |
| Helpers | `packages/join-e2e/tests/helpers.ts` | Shared utilities for mocking, environment injection, and payment simulation |
| Seed script | `packages/join-e2e/scripts/setup.php` | Creates test pages with specific block configurations in WordPress |
| Playwright config | `packages/join-e2e/playwright.config.ts` | Single Chromium project, serial execution, base URL `localhost:8889` |

### Key helpers

**`mockRestEndpoints(page)`** intercepts `/wp-json/join/v1/step` and `/wp-json/join/v1/join` with empty 200 responses so network errors never block form progression.

**`injectEnvOverrides(page, urlPattern, overrides)`** intercepts the page HTML response and patches the `<script id="env">` JSON block before the browser receives it. This lets tests toggle feature flags (e.g. `USE_STRIPE`, `STRIPE_DIRECT_DEBIT_ONLY`) without changing WordPress plugin settings.

Note: this approach only affects what the frontend receives. The WordPress database and PHP backend still hold the real plugin settings. Any backend behaviour that reads those settings directly (e.g. server-side feature gating, REST endpoint logic) will not be affected by overrides applied here. For true end-to-end coverage of backend-gated behaviour, the settings would need to be written to the database directly, for example via `wp option update` in the seed script.

**`captureJoinBodyViaStripeRedirect(page, pageUrl)`** simulates a completed Stripe payment by injecting a fake `stripePaymentIntentId` into `sessionStorage` and reloading with `?stripe_success=true`. The app detects this and jumps to the confirm stage. The helper intercepts the resulting `/join` POST and returns its body for assertions.

### Test data

Building with `USE_TEST_DATA=true` pre-fills all personal detail fields (name, email, phone, address). This lets tests skip manual field entry when the details page is not the focus.

### Seed pages

The setup script (`scripts/setup.php`) creates these WordPress pages:

| Slug | Configuration |
|------|---------------|
| `e2e-standard-join` | Standard membership, GBP 5/month |
| `e2e-free-join` | Free membership, GBP 0/month |
| `e2e-donation-upsell` | Standard membership with `ask_for_additional_donation` enabled |
| `e2e-supporter` | Supporter mode with 3 tiers (GBP 5, 10, 20/month) |
| `e2e-supporter-custom` | Supporter mode with custom amount allowed |
| `e2e-supporter-no-plans` | Supporter mode with no plans (triggers warning UI) |


## What is covered

The suite currently contains 52 passing tests across 7 spec files.

| Spec file | Area | Tests | What it verifies |
|-----------|------|------:|------------------|
| `render.spec.ts` | Rendering | 3 | Form container present, env script injected, first name input visible, no console errors, progress breadcrumb renders |
| `form-progression.spec.ts` | Form progression | 7 | Test data pre-fill, required field validation (name, email, phone, address), email format validation, country defaults to GB, details-to-plan advancement, plan selection, `/step` request body |
| `free-membership.spec.ts` | Free membership | 4 | Payment stages skipped for GBP 0 plans, confirm stage reached from plan, confirm page content, `/join` request body |
| `donation-upsell.spec.ts` | Donation upsell | 6 | Upsell page appears when enabled, "Not right now" skips to payment, tier selection advances, `recurDonation` flag toggling, supporter mode takes precedence over upsell |
| `supporter-mode-monthly.spec.ts` | Supporter monthly | 9 | Donation page is first step, breadcrumb text, monthly toggle default, tier selection updates CTA text, donation-to-details progression, `/join` body fields, custom amount input |
| `supporter-mode-oneoff.spec.ts` | Supporter one-off | 6 | One-off tab enabled under correct env flags, CTA text changes, `/join` body contains `recurDonation=false`, `paymentMethod=creditCard` forced, custom one-off amounts |
| `supporter-mode-edge-cases.spec.ts` | Edge cases | 7 | One-off disabled under `STRIPE_DIRECT_DEBIT_ONLY`, explanatory note shown, monthly still works with DD-only, "no amounts configured" warning, standard vs supporter `/join` body differences |

### Coverage strengths

- **Form navigation logic** is well covered across standard, free, supporter, and upsell flows.
- **Data contracts** (`/step` and `/join` request bodies) are asserted for all major flows.
- **Feature flag behaviour** is tested by injecting env overrides, covering combinations of `USE_STRIPE`, `STRIPE_DIRECT_DEBIT`, `STRIPE_DIRECT_DEBIT_ONLY`, and `ASK_FOR_ADDITIONAL_DONATION`.
- **Validation** for required fields and email format is exercised.


## What is not covered

### Payment UI rendering

The `captureJoinBodyViaStripeRedirect` helper bypasses the payment stage entirely. If Stripe Elements stopped rendering, the payment method selector broke, or the intent creation API call failed, the current tests would not detect it.

### GoCardless flow

Zero coverage. GoCardless is the primary payment method for Direct Debit members. The hosted redirect flow, API flow, and mandate creation are all untested. There is no equivalent of `captureJoinBodyViaStripeRedirect` for GoCardless.

### Chargebee flow

Zero coverage. Neither the hosted pages flow nor the API flow is tested.

### Update flow (`IS_UPDATE_FLOW=true`)

No tests. When `IS_UPDATE_FLOW=true`, personal details are skipped and the form starts at the plan stage, with the member's email pre-filled via URL parameter. This distinct entry point has no coverage.

### Payment method selection page

The `payment-method` step (choosing between Direct Debit and Card when multiple providers are enabled) is never exercised. Tests either mock everything or bypass payment.

### Optional data collection flags

The following flags control whether additional fields appear on the details page, but none are tested:

- `COLLECT_DATE_OF_BIRTH`
- `COLLECT_COUNTY`
- `COLLECT_HEAR_ABOUT_US`
- `COLLECT_PHONE_AND_EMAIL_CONTACT_CONSENT`

### Custom fields

`CUSTOM_FIELDS` allows arbitrary text, checkbox, number, select, and radio fields on the details page. Not tested.

### Address lookup

`USE_POSTCODE_LOOKUP` enables a postcode-to-address autocomplete. Not tested.

### `/join` response handling

Tests verify what is sent to `/join` but not what happens with the response. Success redirects, error message rendering, and retry behaviour are all unverified.

### Success redirect

The `SUCCESS_REDIRECT` / `joined_page` block setting controls post-join navigation. Not tested.

### Error states

Network errors, server validation errors, Stripe card declines, and GoCardless mandate failures have no coverage.

### Skip payment button

`INCLUDE_SKIP_PAYMENT_BUTTON` renders a "skip to thank you" button. Not tested.

### Backend integrations (out of scope)

Mailchimp, Action Network, Zetkin, Auth0, and lapsing/unlapsing webhooks are backend concerns. Mailchimp is covered by PHPUnit (`JoinServiceMailchimpTest.php`). These are not appropriate for frontend e2e tests.


## Future coverage workstreams

Ordered roughly by value and feasibility. Each item is designed to be tackled independently.

### 1. Update flow

**Value:** High. This is a distinct user-facing flow with no coverage at all.

**What is needed:**
- [ ] Seed a new test page (`e2e-update-flow`) via `setup.php`
- [ ] Use `injectEnvOverrides` to set `IS_UPDATE_FLOW=true`
- [ ] Pass an `email` URL parameter to pre-fill the member email
- [ ] Verify the form starts at the plan step (details page skipped)
- [ ] Verify the pre-filled email appears on the confirm page
- [ ] Assert the `/join` request body includes the URL-provided email

### 2. Optional data collection fields

**Value:** Medium. These are simple show/hide toggles but affect data sent to the backend.

**What is needed:**
- [ ] Use `injectEnvOverrides` to enable each flag individually
- [ ] Verify the corresponding input fields appear on the details page
- [ ] Fill in values and confirm they are included in the `/step` request body
- [ ] Verify fields are absent when the flag is disabled

Flags to cover: `COLLECT_DATE_OF_BIRTH`, `COLLECT_COUNTY`, `COLLECT_HEAR_ABOUT_US`, `COLLECT_PHONE_AND_EMAIL_CONTACT_CONSENT`.

### 3. Payment method selection page

**Value:** Medium. This is the gate between choosing Direct Debit vs Card when multiple providers are active.

**What is needed:**
- [ ] Use `injectEnvOverrides` to enable both `USE_GOCARDLESS=true` and `USE_STRIPE=true`
- [ ] Verify the payment-method step appears between donation/plan and payment-details
- [ ] Verify selecting "Direct Debit" and "Card" each advance to the correct next stage
- [ ] Assert the selected `paymentMethod` value is stored in session state

### 4. GoCardless redirect simulation

**Value:** High, but requires more infrastructure.

**What is needed:**
- [ ] Write a `captureJoinBodyViaGoCardlessRedirect` helper that injects a fake `billing_request_id` into session state and reloads with `?gocardless_success=true`
- [ ] Verify the confirm page renders and the `/join` request body includes GoCardless-specific fields
- [ ] Determine the exact query parameters and session keys the app expects on GoCardless return (check the `useGoCardless` hook or equivalent)

### 5. Chargebee redirect simulation

**Value:** Medium. Similar pattern to GoCardless.

**What is needed:**
- [ ] Write a `captureJoinBodyViaChargebeeRedirect` helper that simulates the Chargebee hosted page return
- [ ] Determine the expected query parameters on return (`?chargebee_success=true` or similar)
- [ ] Use `injectEnvOverrides` to set `USE_CHARGEBEE=true` (and optionally `USE_CHARGEBEE_HOSTED_PAGES=true`)
- [ ] Assert `/join` body includes Chargebee-specific fields

### 6. `/join` response handling and error states

**Value:** High for user experience confidence. Moderate implementation effort.

**What is needed:**
- [ ] Mock `/join` to return an error response (e.g. `{ success: false, message: "Email already registered" }`)
- [ ] Verify the error message is displayed to the user
- [ ] Mock `/join` to return a network error (use `route.abort()`)
- [ ] Verify appropriate error UI is shown
- [ ] Mock `/join` to return success and verify redirect to `SUCCESS_REDIRECT` URL

### 7. Success redirect

**Value:** Medium. Validates the post-join experience.

**What is needed:**
- [ ] Seed a test page with a `joined_page` / `SUCCESS_REDIRECT` pointing to a known URL
- [ ] Complete a flow (e.g. free membership) and verify `page.url()` matches the redirect target
- [ ] Test with and without a configured redirect

### 8. Custom fields

**Value:** Low-medium. Affects organisations that configure custom fields.

**What is needed:**
- [ ] Use `injectEnvOverrides` to inject a `CUSTOM_FIELDS` configuration with at least one of each type: text, checkbox, number, select, radio
- [ ] Verify each field renders on the details page
- [ ] Fill in values and verify they appear in the `/step` or `/join` request body
- [ ] Verify validation (e.g. required custom fields block progression)

### 9. Address lookup

**Value:** Low. Nice to have, but the feature depends on a third-party postcode API.

**What is needed:**
- [ ] Mock the postcode lookup API endpoint
- [ ] Use `injectEnvOverrides` to set `USE_POSTCODE_LOOKUP=true`
- [ ] Type a postcode and verify the autocomplete dropdown appears with mocked results
- [ ] Select an address and verify the address fields are populated

### 10. WordPress version compatibility

**Value:** Medium. The test suite currently pins WordPress to a specific version. Regressions caused by WordPress core updates would not be caught until the pin is moved.

**What is needed:**
- [ ] Configure the CI matrix to run the suite against the current stable WordPress release and the previous major version
- [ ] Update `.wp-env.json` when a new WordPress major is released and confirm the suite passes
- [ ] Consider running nightly or weekly against WordPress trunk to catch regressions before stable release

### 11. Skip payment button

**Value:** Low. Single feature flag toggle.

**What is needed:**
- [ ] Use `injectEnvOverrides` to set `INCLUDE_SKIP_PAYMENT_BUTTON=true`
- [ ] Verify the skip button appears on the payment page
- [ ] Click it and verify the user reaches the confirm page
- [ ] Assert the `/join` body reflects that payment was skipped


## Running the tests

```bash
# Start the wp-env environment
yarn wp-env start

# Seed the test pages
yarn wp-env run tests-cli wp eval-file /var/www/html/wp-content/e2e-scripts/setup.php

# Build the frontend with test data pre-fill
USE_TEST_DATA=true yarn build

# Run the suite
cd packages/join-e2e
npx playwright test

# Run a specific spec
npx playwright test tests/free-membership.spec.ts

# View the HTML report after a run
npx playwright show-report
```
Loading
Loading