Skip to content

Auto-instrumentation via node hook breaks OpenAI helper APIs (parse/withResponse/stream helpers) #1591

@lforst

Description

@lforst

Summary

Under node --import braintrust/hook.mjs, several OpenAI SDK helper APIs stop working even though the equivalent lower-level calls still work.

What is broken

These helper paths fail in auto-instrumented mode:

  • chat.completions.parse(...)
  • .withResponse()-based request helpers
  • sync stream helper paths like chat.completions.stream(...)

In practice, the auto-instrumented e2e scenario had to fall back to plain create(...) / streamed create(...) calls to keep coverage running.

Why this matters

This means the auto-instrumented path is not API-compatible with normal SDK usage for some common OpenAI helpers. A user can write valid OpenAI code that works normally, then see it break only when enabling Braintrust auto-instrumentation.

Repro

The aligned e2e coverage in e2e/scenarios/openai-auto-instrumentation-node-hook reproduces this. One concrete failure we hit was:

  • _thenUnwrap is not a function from chat.completions.parse(...) under the node hook

Expected

The node-hook auto-instrumented path should preserve normal behavior for OpenAI helper APIs, not require falling back to lower-level request methods.

Metadata

Metadata

Labels

No labels
No labels

Type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions