-
Notifications
You must be signed in to change notification settings - Fork 1
Description
Gap
The non-beta client.chat.completions.parse() method is not instrumented. This is the GA method for structured outputs (available since openai v1.40.0, August 2024). Only client.beta.chat.completions.parse() is wrapped via BetaCompletionsV1Wrapper.
CompletionsV1Wrapper (line 776 of oai.py) only defines create(). When users call client.chat.completions.parse(), it falls through via NamedWrapper.__getattr__ to the original unwrapped method — no span is created.
Note: client.responses.parse() IS correctly wrapped via ResponsesV1Wrapper.parse().
What is missing
client.chat.completions.parse()(sync) — needs wrapping inCompletionsV1Wrapperclient.chat.completions.parse()(async) — needs wrapping inAsyncCompletionsV1Wrapper
Braintrust docs status
supported — the OpenAI integration page at braintrust.dev/docs/integrations/ai-providers/openai states structured outputs via chat.completions.parse() are supported, but the Python SDK implementation only covers the beta path.
Upstream sources
- OpenAI structured outputs guide: https://platform.openai.com/docs/guides/structured-outputs
openai.resources.chat.completions.Completions.parsemethod (GA since openai-python v1.40.0)
Local files inspected
py/src/braintrust/oai.py:CompletionsV1Wrapper(line 776) — only wrapscreate()BetaCompletionsV1Wrapper(line 887) — wrapsparse()✓ResponsesV1Wrapper(line 847) — wraps bothcreate()andparse()✓NamedWrapper.__getattr__(line 24) — delegates unknown attrs to wrapped object (unwrapped)
py/src/braintrust/wrappers/test_openai.py— no tests forchat.completions.parse, onlyresponses.parseandbeta.chat.completions.parse- Grep for
chat.completions.parseacross the repo: zero results