Skip to content

OpenAI: chat.completions.parse() (non-beta) not instrumented #106

@AbhiPrasad

Description

Gap

The non-beta client.chat.completions.parse() method is not instrumented. This is the GA method for structured outputs (available since openai v1.40.0, August 2024). Only client.beta.chat.completions.parse() is wrapped via BetaCompletionsV1Wrapper.

CompletionsV1Wrapper (line 776 of oai.py) only defines create(). When users call client.chat.completions.parse(), it falls through via NamedWrapper.__getattr__ to the original unwrapped method — no span is created.

Note: client.responses.parse() IS correctly wrapped via ResponsesV1Wrapper.parse().

What is missing

  • client.chat.completions.parse() (sync) — needs wrapping in CompletionsV1Wrapper
  • client.chat.completions.parse() (async) — needs wrapping in AsyncCompletionsV1Wrapper

Braintrust docs status

supported — the OpenAI integration page at braintrust.dev/docs/integrations/ai-providers/openai states structured outputs via chat.completions.parse() are supported, but the Python SDK implementation only covers the beta path.

Upstream sources

Local files inspected

  • py/src/braintrust/oai.py:
    • CompletionsV1Wrapper (line 776) — only wraps create()
    • BetaCompletionsV1Wrapper (line 887) — wraps parse()
    • ResponsesV1Wrapper (line 847) — wraps both create() and parse()
    • NamedWrapper.__getattr__ (line 24) — delegates unknown attrs to wrapped object (unwrapped)
  • py/src/braintrust/wrappers/test_openai.py — no tests for chat.completions.parse, only responses.parse and beta.chat.completions.parse
  • Grep for chat.completions.parse across the repo: zero results

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions