Skip to content

feat: multi step api runner#997

Open
shahzad31 wants to merge 13 commits into
mainfrom
api-journey
Open

feat: multi step api runner#997
shahzad31 wants to merge 13 commits into
mainfrom
api-journey

Conversation

@shahzad31

@shahzad31 shahzad31 commented Feb 5, 2025

Copy link
Copy Markdown
Contributor

Summary

Adds apiJourney(...) — runs API-only checks without launching a browser. Same DSL shape as journey(...), but the callback receives an isolated APIRequestContext instead of a Page/Browser driver. Pushes to Kibana as an HTTP-type monitor.

import { apiJourney, monitor, step, expect } from '@elastic/synthetics';

apiJourney('orders API health', ({ request, params }) => {
  monitor.use({ schedule: 1, locations: ['us_east'] });

  let token: string;
  step('obtain OAuth2 token', async () => {
    const r = await request.post(`${params.authUrl}/oauth2/token`, {
      form: { grant_type: 'client_credentials', client_id: params.clientId, client_secret: params.clientSecret },
    });
    expect(r.status()).toBe(200);
    token = (await r.json()).access_token;
  });

  step('check /orders', async () => {
    const r = await request.get(`${params.apiUrl}/orders`, {
      headers: { Authorization: `Bearer ${token}` },
    });
    expect(r.status()).toBe(200);
  });
});

Requires Playwright ≥ 1.61. TLS and remote-address capture use the native APIResponse.securityDetails() / APIResponse.serverAddr() APIs added in microsoft/playwright#40932.

Closes

What's in the PR

  • APIJourney DSL — subclass of Journey, monitor type 'http', full monitor.use(...) / params / expect / .skip / .only support.
  • No browser when not needed — runner skips launchBrowser if no browser journeys are registered.
  • APINetworkManager — intercepts APIRequestContext.fetch (the single funnel) and captures URL, method, status, headers (redacted), body bytes, timings, and step attribution.
  • Native TLS / remote address — reads APIResponse.securityDetails() (cert issuer / subject / validFrom / validTo / protocol) and APIResponse.serverAddr() (remote IP / port) straight off the response from the actual request, mirroring the browser network path. HTTPS and plain HTTP surface server.ip / server.port; cert info follows redirects to the final hop. No extra socket or parallel handshake.
  • Playwright 1.61 bump + codegen vendoring — 1.61 bundles its server code into coreBundle.js and no longer exports the recorder's codegen classes, so the minimal slice SyntheticsGenerator extends (_asLocator / _generateActionCall / JavaScriptFormatter) is vendored into src/formatter/codegen.ts, reusing Playwright's still-exported iso helpers. Recorder snapshots are unchanged.
  • PluginManager — common NetworkPlugin interface so API network data isn't dropped at journey-end.
  • JSONReporter — emits journey.type: 'api', omits browser-only fields, surfaces tls.* and server.ip / server.port.

Tests

All green across:

  • dsl/api-journey, core/api-journey-register — DSL + factory
  • plugins/api-network, plugins/plugin-manager — capture, native TLS / remote-addr, orchestration
  • core/api-runner — e2e, no-browser, failure path, empty-step grouping, cookie isolation, HTTPS with TLS / IP / body-bytes
  • reporters/jsontype: 'api' emission, server / TLS fields
  • formatter/javascript — recorder codegen snapshots unchanged against the vendored generator

Manual verification

Ran the built CLI against api.github.com, httpbin.org, and a local self-signed HTTPS server. Confirmed: no browser spawned, full tls.* / server.* / body.bytes populated, cookies isolated, redaction still works.

Docs

  • README updated with browser vs. API journey sections.
  • examples/todos/api.journey.ts runnable example added.

Follow-ups (separate PRs)

  1. Kibana UI — handle journey.type: 'api' in waterfall, monitor list, data stream. Open question from [proposal] Lightweight API journeys DSL #900 needs an answer.
  2. mTLS client cert — to fully solve elastic/enhancements#25818 (Atos France).
  3. Finer timing breakdown (dns, connect, ssl, wait, receive) — APIResponse doesn't expose per-phase timings; needs Node http.Agent hooks (or a future Playwright API).
  4. HAR export integration — Playwright already records API timings to HAR (microsoft/playwright#32613); could complement the response fields.

Comment thread src/helpers.ts Outdated
Comment thread src/dsl/api-journey.ts Outdated
};
export type APIJourneyCallback = (options: APIJourneyCallbackOpts) => void;

export class APIJourney {

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ideally I would like us to have a base Journey class that both BrowserJourney and APIJourney extends from and the runner should be a single entity that decides how to run each of them. I am not in favor of having multiple runners.

@shahzad31 shahzad31 changed the title DRAFT: multi step api runner. feat: multi step api runner. Feb 10, 2025
@shahzad31 shahzad31 changed the title feat: multi step api runner. feat: multi step api runner Feb 13, 2025

@vigneshshanmugam vigneshshanmugam left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Did a first pass.

Comment thread src/common_types.ts Outdated
Comment thread src/core/runner.ts
Comment thread src/dsl/api-journey.ts Outdated
Comment thread src/plugins/api-network.ts Outdated
Comment thread src/plugins/plugin-manager.ts Outdated
…ting it

Playwright 1.61 bundles its server code into coreBundle.js and no longer
ships the codegen classes as importable modules, so the recorder's
`playwright-core/lib/server/codegen/javascript` deep import no longer
resolves. Vendor the minimal slice SyntheticsGenerator extends
(_asLocator + _generateActionCall + JavaScriptFormatter) into
src/formatter/codegen.ts, reusing the still-exported `iso` helpers
(asLocator / formatObject / escapeWithQuotes) rather than vendoring the
heavy locator logic. Formatter snapshots are unchanged.

Also bump playwright/-chromium/-core to 1.61.0 (required for the native
APIResponse TLS APIs) and refresh the device-descriptor Chrome UA in the
options test that 1.61's bundled descriptors updated.

Co-authored-by: Cursor <cursoragent@cursor.com>
shahzad31 and others added 12 commits June 15, 2026 18:07
Introduces apiJourney() DSL plus APIDriver, APINetworkManager, and the
type-aware Runner/Gatherer/PluginManager branching needed to run API-only
journeys without launching Chromium.

Co-authored-by: Cursor <cursoragent@cursor.com>
Builds on the rebased api-journey commit:

- PluginManager.output() now surfaces APINetworkManager results
  (previously dropped silently because of an instanceof NetworkManager
  check), and onStep() uses a proper union narrow instead of a lying
  cast. The browser/api branching is hidden behind a shared NetworkPlugin
  contract so the manager is transparent to journey type.
- Runner only launches Chromium when at least one browser journey is
  scheduled; pure API suites skip launch entirely.
- APIJourney now overrides _updateMonitor so 'synthetics push' registers
  it as an HTTP monitor instead of mislabeling it as browser. Journey
  base class gained a protected _setMonitor helper to make subclassing
  safe.
- apiJourney.skip / apiJourney.only are now wired up.
- APINetworkManager rewritten: only patches request.fetch (Playwright's
  helpers funnel through it, so the previous double-patch was
  double-counting requests), restores the prototype method on stop,
  handles fetch(Request, opts), wraps in try/finally so failed requests
  still leave a valid entry, drops dead Page/Frame barriers, surfaces
  status/headers/url/statusText.
- APIJourney class trimmed: dead #cb / #driver fields removed; subclass
  now carries the http monitor override and forwards string|options
  upstream.
- Reporter payload no longer carries browserDelay / browserconsole for
  API journeys; common_types updated accordingly.
- Public API exports APIJourney / APIJourneyCallback /
  APIJourneyCallbackOpts / APIJourneyOptions /
  APIJourneyWithAnnotations.
- Tests added: dsl/api-journey, plugins/api-network round-trip against a
  local HTTP server, plugins/plugin-manager API-driver coverage,
  core/api-runner end-to-end with browser launch spy, core/api-journey-register
  factory + skip/only wiring.

Co-authored-by: Cursor <cursoragent@cursor.com>
Playwright's APIResponse doesn't expose securityDetails(), serverAddr(),
or request/response timings (upstream microsoft/playwright#32647 and
microsoft/playwright#34938 were both declined; see microsoft/playwright#40905
for the current ask).

To still surface this for HTTPS API monitoring use cases (cert expiry,
remote-address alerting, response size tracking), open a side-channel
tls.connect() in parallel with each request and fold the result into
the NetworkInfo entry:

- securityDetails: issuer, subjectName, protocol, validFrom, validTo
- remoteIPAddress / remotePort
- coarse dns / connect / ssl timings

Probes are cached per host:port within a journey and silently skipped
for HTTP or on failure (timeout, refused, untrusted) so they never
affect the actual request.

Also derive request and response body bytes (Content-Length preferred,
buffer fallback) and emit server.ip / server.port in the JSON reporter.

Co-authored-by: Cursor <cursoragent@cursor.com>
- Empty step with no requests must not break network-event step
  attribution: the next step's requests must still be assigned to
  that step's reference identity (required for waterfall grouping).
- API journeys must have isolated APIRequestContexts: a cookie set in
  journey A must not leak into journey B.
- HTTPS journey verifies the full pipeline emits TLS securityDetails,
  remote address, and response body bytes.

Co-authored-by: Cursor <cursoragent@cursor.com>
Splits the README usage section into "Browser journeys" and "API
journeys (no browser)" so users discover apiJourney() without having
to dig through the Elastic docs site. Adds a runnable example under
examples/todos/api.journey.ts showing OAuth-style multi-step API
checks.

Co-authored-by: Cursor <cursoragent@cursor.com>
- Replace non-null assertions with explicit narrowing in api-tls and
  api-runner tests so the `@typescript-eslint/no-non-null-assertion`
  rule is satisfied.
- Apply prettier formatting to api-tls.ts and json.test.ts.

Co-authored-by: Cursor <cursoragent@cursor.com>
Heartbeat hands API monitor scripts to `elastic-synthetics` as inline
source via `--inline`. When that source uses ESM (e.g. `import` /
top-level await) — which is the natural shape for `apiJourney()` and
`step()` imports — Node's `vm.runInContext` path falls over with
`SyntaxError: Cannot use import statement outside a module`, silently
dropping the run.

Detect ESM-shaped inline source and execute it via a temporary `.mjs`
module file resolved with a `Module._resolveFilename` alias so
`@elastic/synthetics` keeps resolving to the agent-installed copy. The
CommonJS fast path is unchanged.

Co-authored-by: Cursor <cursoragent@cursor.com>
The new ESM inline loader path compiles the source as a regular module
rather than running it through the new Function(...) wrapper, so `params`
is no longer injected as an implicit local. The test needs to pull it
out of the apiJourney callback args like the sibling browser test does.

Co-authored-by: Cursor <cursoragent@cursor.com>
Address review findings from end-to-end review of the API journey work:

- src/reporters/json.ts: `formatTLS` previously called
  `new Date(undefined * 1000).toISOString()` when a TLS probe resolved
  with `protocol` set but cert dates missing (malformed `valid_from` /
  `valid_to`). That throws `RangeError: Invalid time value` and sinks
  the entire `journey/end` document. Route the dates through a
  defensive `epochToIso` helper that returns `undefined` for non-finite
  inputs, and add a regression test.

- src/plugins/api-tls.ts: For IP-literal hosts the `lookup` event
  never fires, so `dnsEnd` stayed at its `-1` sentinel and cascaded
  into `connect: -1`. Treat the missing DNS phase as `dns: 0` and
  measure `connect` from `dnsStart`, so the timing breakdown stays
  meaningful. Add a probe test that exercises this path.

- src/plugins/api-network.ts: `_currentStep` was typed `Partial<Step>`
  but initialised to `null`, contradicting the shared `NetworkPlugin`
  shape. Widen the field type to `Partial<Step> | null`.

- src/loader.ts: Drop the `journey(` / `apiJourney(` heuristic from
  `isModuleInlineSource`. The regex matched inside string literals
  and comments, silently routing legacy inline scripts through the
  ESM loader and stripping the implicit `step` / `page` / `params`
  injection. Key off `import` / `export` only — that's the contract
  documented in the README. Also register a best-effort `process.exit`
  cleanup hook for the materialised `mkdtempSync` directory so
  long-lived hosts don't accumulate tmp dirs.

- __tests__/plugins/api-network.test.ts: Add a guard that exercises
  every `APIRequestContext` helper (`get`/`post`/`put`/`patch`/
  `delete`/`head`/`fetch`) so a future Playwright that bypasses
  `this.fetch` on any of them stops being a silent capture loss.

- __tests__/core/api-runner.test.ts: Add a mixed-mode test verifying
  that a suite with both a `journey()` and an `apiJourney()` launches
  Chromium exactly once and routes each journey through the right
  driver type.

Co-authored-by: Cursor <cursoragent@cursor.com>
Mirror the existing browser-journey scaffold examples with API-journey
counterparts so users adopting `npx @elastic/synthetics <dir>` see
both monitoring shapes side by side.

- templates/journeys/api-example.journey.ts: minimal `apiJourney` with
  two GET steps and status assertions, structurally parallel to the
  existing `example.journey.ts`.
- templates/journeys/advanced-api-example.journey.ts and its
  `advanced-api-example-helpers.ts`: multi-step API journey
  demonstrating the recommended shape — small reusable step builders,
  shared state populated by earlier steps and consumed by later ones,
  with a thunk-based id deletion to make the registration vs.
  execution timing explicit.
- templates/synthetics.config.ts: adds `params.apiUrl` defaulting to
  jsonplaceholder.typicode.com so the API examples run out of the
  box; users override per-environment for their own service.
- templates/README.md: distinguishes browser vs. API journey examples
  and explains when to reach for each.

Co-authored-by: Cursor <cursoragent@cursor.com>
`CLIMock.output()` returns only the last stdout chunk, and the
existing test fed that into `JSON.parse`. On Linux CI a single
chunk can hold multiple NDJSON events from the json reporter, which
makes the parse throw `SyntaxError: Unexpected non-whitespace
character after JSON at position N`. The test was therefore order-
and flush-dependent and recently started flaking.

Switch to the `cli.buffer()` accumulator (which join+split-by-line
correctly reconstructs NDJSON regardless of chunk boundaries) and
locate the `journey/start` event explicitly instead of assuming the
last chunk holds exactly one event. A defensive `tryParse` swallows
the rare partial-line case where the listener detaches mid-event,
so the lookup keeps working without resorting to longer waits.

Co-authored-by: Cursor <cursoragent@cursor.com>
…t 1.61

Playwright 1.61 (microsoft/playwright#40932) exposes
APIResponse.securityDetails() and APIResponse.serverAddr(), returning the
same shapes the browser network path already consumes. Drop the
tls.connect() side-channel (api-tls.ts), its per-origin cache, and the
probe-fold blocks in APINetworkManager in favour of reading cert info and
remote address straight off the response used by the actual request.

This gives true per-request fidelity (final hop on redirects), removes the
extra parallel TLS handshake, and now also reports server.ip/port over
plain HTTP. The synthesized dns/connect/ssl timings (which came from a
separate socket) are gone. A small normalizeTLSProtocol keeps the
"TLSv1.3" -> "TLS 1.3" shape the JSON reporter expects.

Co-authored-by: Cursor <cursoragent@cursor.com>
@shahzad31 shahzad31 changed the base branch from main to chore/bump-playwright-1.61 June 15, 2026 16:15
@shahzad31 shahzad31 changed the base branch from chore/bump-playwright-1.61 to main June 16, 2026 10:35
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants