[Bug]: LangChain instrumentation mis-attributes response model into gen_ai.request.model; gen_ai.response.model never set

[JacksonWeber]

Component

general

Description

The MS JS distro (0.1.0-beta) LangChainTraceInstrumentor mis-attributes the LLM response model into the gen_ai.request.model attribute and never sets gen_ai.response.model. The actual request deployment alias is also lost in some flows. The bug reproduces across three LangChain code paths and also affects span naming.

Affected scenarios

1. LangChain Node.js – AzureChatOpenAI – Chat Completions API (CAPI)

• Span name: chat gpt-3.5-turbo
• gen_ai.request.model: o4-mini-2025-04-16 ← wrong (this is the response model)
• gen_ai.response.model: (empty)
• Prompt tokens: 18 / Completion tokens: 0
• Root cause: The instrumentor reads LLMResult.llmOutput.model_name (the response model) and writes it to gen_ai.request.model, conflating the two attributes. The actual request deployment alias is dropped entirely. The span name falls back to LangChain''s default model field (gpt-3.5-turbo) because no model kwarg is passed to AzureChatOpenAI.

2. LangChain Node.js – ChatOpenAI (Foundry) – Chat Completions API (CAPI)

• Span name: chat deployment-o4-mini
• gen_ai.request.model: o4-mini-2025-04-16 ← wrong (this is the response model)
• gen_ai.response.model: (empty)
• Prompt tokens: 5 / Completion tokens: 0
• Root cause: Same as Initial Microsoft Distro Scaffolding #1 — the response model is mis-attributed into gen_ai.request.model. Here the span name picks up the deployment string (deployment-o4-mini) because it is passed as the model kwarg.

3. LangChain Node.js – ChatOpenAI (Foundry, useResponsesApi) – Responses API (RAPI)

• Span name: chat deployment-o4-mini
• gen_ai.request.model: deployment-o4-mini
• gen_ai.response.model: (empty)
• Prompt tokens: 4 / Completion tokens: 0
• Root cause: The LangChain callback path doesn''t extract a model from the Responses-API result object (the field shape differs from Chat Completions), so it falls back to the request deployment for gen_ai.request.model and leaves gen_ai.response.model empty.

Expected Behavior

• gen_ai.request.model should be the model/deployment requested by the caller (the deployment alias for Azure/Foundry, or the model kwarg passed to the client).
• gen_ai.response.model should be populated from LLMResult.llmOutput.model_name (Chat Completions) or the equivalent field in the Responses-API result object.
• Span name should follow GenAI semantic conventions and use the request model.
• Across all three scenarios above, the request and response model attributes should be distinct and both populated.

Steps to Reproduce

1. Install @microsoft/opentelemetry-distro-javascript (or the appropriate LangChain instrumentation package) at version 0.1.0-beta.
2. Configure the distro to instrument LangChain.
3. Reproduce each of the three scenarios:
   a. Build a AzureChatOpenAI client (CAPI) without passing a model kwarg, and invoke it.
   b. Build a ChatOpenAI client pointed at Foundry (CAPI) with the deployment passed as the model kwarg, and invoke it.
   c. Build a ChatOpenAI client pointed at Foundry with useResponsesApi: true (RAPI), and invoke it.
4. Inspect the resulting spans:
   • gen_ai.request.model contains the response model (Initial Microsoft Distro Scaffolding #1, Add planning and other md files #2) or the request deployment with no response model set (Migrate AzMon Distro #3).
   • gen_ai.response.model is empty in all three cases.

Environment

• OS: any
• Node.js: 20.x
• Package version: @microsoft/opentelemetry-distro-javascript 0.1.0-beta
• LangChain: Node.js
• Runtime: standalone Node.js

Additional Context

Suggested fix

1. Stop writing LLMResult.llmOutput.model_name to gen_ai.request.model. Route it to gen_ai.response.model instead.
2. Source gen_ai.request.model from the invocation params / client configuration (e.g., the deployment alias for Azure/Foundry, or the model kwarg).
3. Add a Responses-API-specific extractor for the response model so gen_ai.response.model is populated when useResponsesApi is enabled.
4. Add tests covering all three paths above (AzureChatOpenAI CAPI, ChatOpenAI Foundry CAPI, ChatOpenAI Foundry RAPI).

[github-actions]

cc @hectorhdzg @JacksonWeber @lzchen @MSNev @rads-1996 @jeremydvoss

[fpfp100]

The reason that gen_ai.response.model is not set because it is not in A365 Observability schema. It is by design based on the schema. Attributes which are not in the schema will be discarded by ingestion service.

The reason that response model is used to set request model attribute is because nodejs langchain bug which does not set gen_ai.request.model properly for AzureChatOpenAI. Using gen_ai.response.model generates the name which is very closed to the actual request model e.g gen_ai.request.model: o4-mini-2025-04-16 instead of always "gpt-3.5-turbo".

The suggestion for #1, #2 does not work for AzureChatOpenAI as mentioned above and the current implementation is a work around. Based on the investigation on the bug, it seems the right fix is to keep the current workaround for AzureChatOpenAI client. For ChatOpenAI client, fix the setting gen_ai.request.model as suggested in the bug.
For #3, not required for A365 Observability schema, it is a no op, but might need for Azure foundry scenario.