diff --git a/CHANGELOG.md b/CHANGELOG.md index 2e22443d..860be0df 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -5,73 +5,8 @@ All notable changes to the Agent365 TypeScript SDK will be documented in this fi The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.0.0/), and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html). -## [0.2.0] - 2026-04-15 - -### Breaking Changes (`@microsoft/agents-a365-observability`) - -- **New permission required: `Agent365.Observability.OtelWrite`** — The observability exporter now requires this scope as both a delegated and application permission on your agent blueprint. See [Upgrade Instructions](#upgrade-instructions-observability-permission-for-existing-agents) below. - ---- - - - -### Upgrade Instructions: Observability Permission for Existing Agents - -Existing agent blueprints need `Agent365.Observability.OtelWrite` granted as both a **delegated permission** and an **application permission**. Choose either option below. - -#### Option A — Agent 365 CLI (requires both config files) - -Requires `a365.config.json` and `a365.generated.config.json` in your config directory, a Global Administrator account, and [Agent 365 CLI v1.1.139-preview](https://www.nuget.org/packages/Microsoft.Agents.A365.DevTools.Cli/1.1.139-preview) or later. - -```bash -a365 setup admin --config-dir "" -``` - -This grants all missing permissions including the new Observability scopes. - -#### Option B — Entra Portal (no config files required) - -Requires Global Administrator access to the blueprint app registration. - -1. Go to **Entra portal** > **App registrations** > select your Blueprint app -2. Go to **API permissions** > **Add a permission** > **APIs my organization uses** > search for `9b975845-388f-4429-889e-eab1ef63949c` -3. Select **Delegated permissions** > check `Agent365.Observability.OtelWrite` > **Add permissions** -4. Repeat step 2–3, this time select **Application permissions** > check `Agent365.Observability.OtelWrite` > **Add permissions** -5. Click **Grant admin consent** and confirm - -Both `Agent365.Observability.OtelWrite` (Delegated) and `Agent365.Observability.OtelWrite` (Application) should show `Granted` status. - ---- - ## [Unreleased] -### Breaking Changes (`@microsoft/agents-a365-tooling`) - -- **`listToolServers(agenticAppId, authToken)` throws for V2 MCP servers** — The deprecated - two-argument overload now throws a hard error if the gateway returns any server whose - `audience` field does not match the shared ATG app ID (i.e. a V2 server). The legacy - signature cannot perform per-audience OBO because it has no `Authorization` object or - `authHandlerName`. Agents whose blueprints only have V1 permissions are unaffected. - - **Migration** — switch to the preferred overload which handles both V1 and V2 automatically: - ```typescript - // Before (deprecated) - const servers = await service.listToolServers(agenticAppId, authToken); - - // After - const servers = await service.listToolServers(turnContext, authorization, 'graph'); - // authToken is optional; omit it and the SDK auto-generates it via token exchange. - ``` - -### Added (`@microsoft/agents-a365-tooling`) - -- **V1/V2 per-audience token acquisition** — `resolveTokenScopeForServer` now supports explicit `scope` field for V2 MCP servers. When a V2 server provides a `scope` value, the token is requested as `{audience}/{scope}`; otherwise falls back to `{audience}/.default` (pre-consented permissions cover both cases). -- **`publisher` field preserved end-to-end** — `MCPServerConfig.publisher` is now carried through both gateway and manifest normalization and is available to callers of `listToolServers`. - -### Fixed (`@microsoft/agents-a365-tooling-extensions-openai`, `@microsoft/agents-a365-tooling-extensions-langchain`) - -- **Per-audience Authorization headers now correctly applied** — OpenAI and LangChain extensions now merge the per-server `Authorization: Bearer` token from `server.headers` (set by `attachPerAudienceTokens`) with base request headers, instead of applying a single shared discovery token to all MCP servers. This ensures V2 servers receive their own correctly-scoped token. - ### Breaking Changes (`@microsoft/agents-a365-observability`) - **`InvokeAgentDetails` renamed to `InvokeAgentScopeDetails`** — Now contains only scope-level config (`endpoint`). Agent identity (`AgentDetails`) is a separate parameter. `sessionId` moved to `Request`. @@ -101,9 +36,8 @@ Both `Agent365.Observability.OtelWrite` (Delegated) and `Agent365.Observability. - **`OutputResponse.messages` type changed from `string[]` to `OutputMessagesParam`** — The `OutputMessagesParam` union type (`string[] | OutputMessages`) allows passing either plain strings or a versioned `OutputMessages` wrapper (`{ version, messages: OutputMessage[] }`) with `finish_reason`, multi-modal parts, etc. Existing code passing `string[]` continues to work (auto-converted to OTEL format internally), preserving backward compatibility. - **`recordInputMessages()` / `recordOutputMessages()` parameter type widened** — Methods now accept `InputMessagesParam` (`string[] | InputMessages`) and `OutputMessagesParam` (`string[] | OutputMessages`). `InputMessages` is a versioned wrapper `{ version, messages: ChatMessage[] }` and `OutputMessages` is a versioned wrapper `{ version, messages: OutputMessage[] }`. Plain `string[]` input is auto-wrapped to OTEL gen-ai format. -### Added +### Added (`@microsoft/agents-a365-observability`) -#### `@microsoft/agents-a365-observability` - **OTEL Gen-AI Message Format types** — New types aligned with [OpenTelemetry Gen-AI semantic conventions](https://opentelemetry.io/docs/specs/semconv/gen-ai/): `MessageRole`, `FinishReason`, `Modality`, `ChatMessage`, `OutputMessage`, `InputMessages`, `OutputMessages`, and discriminated `MessagePart` union (`TextPart`, `ToolCallRequestPart`, `ToolCallResponsePart`, `ReasoningPart`, `BlobPart`, `FilePart`, `UriPart`, `ServerToolCallPart`, `ServerToolCallResponsePart`, `GenericPart`). - **`SpanDetails`** — New interface grouping `parentContext`, `startTime`, `endTime`, `spanKind` for scope construction. - **`CallerDetails`** — New interface wrapping `userDetails` and `callerAgentDetails` for `InvokeAgentScope`. @@ -111,15 +45,6 @@ Both `Agent365.Observability.OtelWrite` (Delegated) and `Agent365.Observability. - **`OpenTelemetryScope.recordCancellation()`** — Records a cancellation event on the span with `error.type = 'TaskCanceledException'`. - **`OpenTelemetryConstants.ERROR_TYPE_CANCELLED`** — Constant for the cancellation error type value. - **`ObservabilityBuilder.withServiceNamespace()`** — Configures the `service.namespace` resource attribute. -- **Span links support** — All scope classes (`InvokeAgentScope`, `InferenceScope`, `ExecuteToolScope`, `OutputScope`) now support span links via `SpanDetails.spanLinks` (passed through the existing `spanDetails?` argument) to establish causal relationships to other spans (e.g. linking a batch operation to individual trigger spans). -- **`BaggageBuilder.invokeAgentServer(address, port?)`** — Fluent setter for server address and port baggage values. Port is only recorded when different from 443 (default HTTPS). Clears stale port entries when port is omitted or 443. -- **Agent365ExporterOptions** — New `httpRequestTimeoutMilliseconds` option (default 30s) controls the per-HTTP-request timeout for backend calls. This is distinct from `exporterTimeoutMilliseconds` which controls the overall BatchSpanProcessor export deadline. - -#### `@microsoft/agents-a365-observability-hosting` -- **OutputScope** — Tracing scope for outgoing agent messages with caller details, conversation ID, channel information, and parent span linking. -- **BaggageMiddleware** — Middleware for automatic OpenTelemetry baggage propagation from TurnContext. -- **OutputLoggingMiddleware** — Middleware that creates OutputScope spans for outgoing messages with lazy parent span linking via `A365_PARENT_SPAN_KEY`. -- **ObservabilityHostingManager** — Manager for configuring hosting-layer observability middleware with `ObservabilityHostingOptions`. ### Breaking Changes (`@microsoft/agents-a365-observability-hosting`) @@ -135,36 +60,46 @@ Both `Agent365.Observability.OtelWrite` (Delegated) and `Agent365.Observability. - **`ScopeUtils.populateExecuteToolScopeFromTurnContext(details, turnContext, authToken, ...)`** — New required `authToken: string` parameter. - **`ScopeUtils.buildInvokeAgentDetails()`** — Now accepts `AgentDetails` (was `InvokeAgentDetails`) and returns flat `AgentDetails` instead of the old `InvokeAgentDetails` wrapper. -### Fixed - -#### `@microsoft/agents-a365-observability` -- **Agent365ExporterOptions** — `exporterTimeoutMilliseconds` default increased from 30s to 90s to allow sufficient time for retries across multiple identity groups within a single export cycle. - -### Changed +### Added -#### `@microsoft/agents-a365-observability` -- **InferenceScope.recordInputMessages() / recordOutputMessages()** — Now use JSON array format instead of comma-separated strings. -- **InvokeAgentScope.recordInputMessages() / recordOutputMessages()** — Now use JSON array format instead of comma-separated strings. -- **Environment variables** — Remove ENABLE_A365_OBSERVABILITY or ENABLE_OBSERVABILITY. No longer need to use environment variable for recordAttributes, setTagMaybe, and addBaggage. -- **EnhancedAgentDetails** — Merged `EnhancedAgentDetails` into `AgentDetails` to unify agent detail typing across scopes and middleware. +- **Span links support** — All scope classes (`InvokeAgentScope`, `InferenceScope`, `ExecuteToolScope`, `OutputScope`) now support span links via `SpanDetails.spanLinks` (passed through the existing `spanDetails?` argument) to establish causal relationships to other spans (e.g. linking a batch operation to individual trigger spans). +- **`BaggageBuilder.invokeAgentServer(address, port?)`** — Fluent setter for server address and port baggage values. Port is only recorded when different from 443 (default HTTPS). Clears stale port entries when port is omitted or 443. +- **`OpenAIAgentsInstrumentationConfig.isContentRecordingEnabled`** — Optional `boolean` to enable content recording in OpenAI trace processor. +- **`LangChainTraceInstrumentor.instrument(module, options?)`** — New optional `{ isContentRecordingEnabled?: boolean }` parameter to enable content recording in LangChain tracer. +- **`truncateValue`** / **`MAX_ATTRIBUTE_LENGTH`** — Exported utilities for attribute value truncation (8192 char limit). +- **OutputScope**: Tracing scope for outgoing agent messages with caller details, conversation ID, channel information, and parent span linking. +- **BaggageMiddleware**: Middleware for automatic OpenTelemetry baggage propagation from TurnContext. +- **OutputLoggingMiddleware**: Middleware that creates OutputScope spans for outgoing messages with lazy parent span linking via `A365_PARENT_SPAN_KEY`. +- **ObservabilityHostingManager**: Manager for configuring hosting-layer observability middleware with `ObservabilityHostingOptions`. +- **Agent365ExporterOptions**: New `httpRequestTimeoutMilliseconds` option (default 30s) controls the per-HTTP-request timeout for backend calls. This is distinct from `exporterTimeoutMilliseconds` which controls the overall BatchSpanProcessor export deadline. -#### `@microsoft/agents-a365-observability-hosting` -- **ObservabilityHostingManager** — `enableBaggage` option now defaults to `false` (was `true`). Callers must explicitly set `enableBaggage: true` to register the BaggageMiddleware. -- **ScopeUtils.deriveAgentDetails** — Now resolves `agentId` via `activity.getAgenticInstanceId()` for embodied (agentic) agents only. For non-embodied agents, `agentId` is `undefined` since the token's app ID cannot reliably be attributed to the agent. -- **ScopeUtils.deriveAgentDetails** — Now resolves `agentBlueprintId` from the JWT `xms_par_app_azp` claim via `RuntimeUtility.getAgentIdFromToken()` instead of reading `recipient.agenticAppBlueprintId`. -- **ScopeUtils.deriveAgentDetails** — Now resolves `agentEmail` via `activity.getAgenticUser()` instead of `recipient.agenticUserId`. -- **ScopeUtils.deriveTenantDetails** — Now uses `activity.getAgenticTenantId()` instead of `recipient.tenantId`. -- **getTargetAgentBaggagePairs** — Now uses `activity.getAgenticInstanceId()` instead of `recipient.agenticAppId`. -- **getTenantIdPair** — Now uses `activity.getAgenticTenantId()` instead of manual `channelData` parsing. +### Fixed +- **Agent365ExporterOptions**: `exporterTimeoutMilliseconds` default increased from 30s to 90s to allow sufficient time for retries across multiple identity groups within a single export cycle. ---- +### Changed +- **ObservabilityHostingManager**: `enableBaggage` option now defaults to `false` (was `true`). Callers must explicitly set `enableBaggage: true` to register the BaggageMiddleware. +- `ScopeUtils.deriveAgentDetails` now resolves `agentId` via `activity.getAgenticInstanceId()` for embodied (agentic) agents only. For non-embodied agents, `agentId` is `undefined` since the token's app ID cannot reliably be attributed to the agent. +- `ScopeUtils.deriveAgentDetails` now resolves `agentBlueprintId` from the JWT `xms_par_app_azp` claim via `RuntimeUtility.getAgentIdFromToken()` instead of reading `recipient.agenticAppBlueprintId`. +- `ScopeUtils.deriveAgentDetails` now resolves `agentEmail` via `activity.getAgenticUser()` instead of `recipient.agenticUserId`. +- `ScopeUtils.deriveTenantDetails` now uses `activity.getAgenticTenantId()` instead of `recipient.tenantId`. +- `getTargetAgentBaggagePairs` now uses `activity.getAgenticInstanceId()` instead of `recipient.agenticAppId`. +- `getTenantIdPair` now uses `activity.getAgenticTenantId()` instead of manual `channelData` parsing. +- `InferenceScope.recordInputMessages()` / `recordOutputMessages()` now use JSON array format instead of comma-separated strings. +- `InvokeAgentScope.recordInputMessages()` / `recordOutputMessages()` now use JSON array format instead of comma-separated strings. + +## [1.1.0] - 2025-12-09 +### Changed +- Remove ENABLE_A365_OBSERVABILITY or ENABLE_OBSERVABILITY. No longer need to use environment variable for recordAttributes, setTagMaybe, and addBaggage. +- Merged `EnhancedAgentDetails` into `AgentDetails` to unify agent detail typing across scopes and middleware. ### Deprecated - `EnhancedAgentDetails` is now an alias of `AgentDetails` and marked as deprecated. Existing imports continue to work without breaking changes; migrate to `AgentDetails` when convenient. +### Notes +- This release is non-breaking. A minor version bump reflects additive API changes and deprecation guidance. -## [0.1.0] - 2025-01-03 +## [1.0.0] - 2025-01-03 ### Added - Initial release of Agent365 TypeScript SDK diff --git a/packages/agents-a365-observability-extensions-langchain/src/LangChainTraceInstrumentor.ts b/packages/agents-a365-observability-extensions-langchain/src/LangChainTraceInstrumentor.ts index b0f840cf..2fedc96b 100644 --- a/packages/agents-a365-observability-extensions-langchain/src/LangChainTraceInstrumentor.ts +++ b/packages/agents-a365-observability-extensions-langchain/src/LangChainTraceInstrumentor.ts @@ -20,8 +20,9 @@ class LangChainTraceInstrumentorImpl extends InstrumentationBase ) { - args[0] = addTracerToHandlers(instrumentor.otelTracer, args[0]); + args[0] = addTracerToHandlers(instrumentor.otelTracer, args[0], { isContentRecordingEnabled: instrumentor.isContentRecordingEnabled }); logger.info("[LangChainTraceInstrumentor] _configureSync wrapped to add LangChainTracer"); return original.apply(this, args); }; @@ -152,9 +154,10 @@ export class LangChainTraceInstrumentor { /** * Initialize and auto-instrument for LangChain * @param module The CallbackManager module to instrument + * @param options Optional configuration options */ - static instrument(module: CallbackManagerModuleType): void { - LangChainTraceInstrumentorImpl.getInstance().manuallyInstrumentImpl(module); + static instrument(module: CallbackManagerModuleType, options?: { isContentRecordingEnabled?: boolean }): void { + LangChainTraceInstrumentorImpl.getInstance(options).manuallyInstrumentImpl(module); } /** @@ -188,20 +191,21 @@ export class LangChainTraceInstrumentor { export function addTracerToHandlers( tracer: Tracer, handlers: CallbackManagerModule.Callbacks | undefined, + options?: { isContentRecordingEnabled?: boolean } ): CallbackManagerModule.Callbacks { if (handlers == null) { - return [new LangChainTracer(tracer)]; + return [new LangChainTracer(tracer, options)]; } if (Array.isArray(handlers)) { if (!handlers.some((h) => h instanceof LangChainTracer)) { - handlers.push(new LangChainTracer(tracer)); + handlers.push(new LangChainTracer(tracer, options)); } return handlers; } if (!handlers.inheritableHandlers.some((h) => h instanceof LangChainTracer)) { - handlers.addHandler(new LangChainTracer(tracer), true); + handlers.addHandler(new LangChainTracer(tracer, options), true); } return handlers; } diff --git a/packages/agents-a365-observability-extensions-langchain/src/Utils.ts b/packages/agents-a365-observability-extensions-langchain/src/Utils.ts index a6a82956..1f14db1a 100644 --- a/packages/agents-a365-observability-extensions-langchain/src/Utils.ts +++ b/packages/agents-a365-observability-extensions-langchain/src/Utils.ts @@ -3,20 +3,7 @@ import { Run } from "@langchain/core/tracers/base"; import { Span } from "@opentelemetry/api"; -import { - OpenTelemetryConstants, - serializeMessages, - safeSerializeToJson, - A365_MESSAGE_SCHEMA_VERSION, - MessageRole, -} from "@microsoft/agents-a365-observability"; -import type { - ChatMessage, - OutputMessage, - InputMessages, - OutputMessages, - MessagePart, -} from "@microsoft/agents-a365-observability"; +import { OpenTelemetryConstants, truncateValue } from "@microsoft/agents-a365-observability"; // Type guards export function isString(value: unknown): value is string { @@ -61,32 +48,13 @@ export function setToolAttributes(run: Run, span: Span) { return; } - if (isString(run.name)) { + if (isString(run.name)) { span.setAttribute(OpenTelemetryConstants.GEN_AI_TOOL_NAME_KEY, run.name); } - if (run.inputs) { - const argsValue = run.inputs?.input ?? run.inputs; - span.setAttribute(OpenTelemetryConstants.GEN_AI_TOOL_ARGS_KEY, safeSerializeToJson( - typeof argsValue === 'object' ? argsValue as Record : String(argsValue), 'arguments' - )); - } - - // Tool result: v0 uses output.kwargs.content, v1 returns output as a plain string or has content directly - const toolResult = - run.outputs?.output?.kwargs?.content ?? - (isString(run.outputs?.output) ? run.outputs.output : null) ?? - run.outputs?.output?.content; - if (toolResult != null) { - span.setAttribute(OpenTelemetryConstants.GEN_AI_TOOL_CALL_RESULT_KEY, safeSerializeToJson( - typeof toolResult === 'object' ? toolResult as Record : String(toolResult), 'result' - )); - } - - span.setAttribute(OpenTelemetryConstants.GEN_AI_TOOL_TYPE_KEY, "extension"); - - // Tool call ID: v0 uses output.tool_call_id, v1 may have it on inputs - const toolCallId = run.outputs?.output?.tool_call_id ?? run.inputs?.tool_call_id; - if (toolCallId) span.setAttribute(OpenTelemetryConstants.GEN_AI_TOOL_CALL_ID_KEY, toolCallId); + if (run.inputs) span.setAttribute(OpenTelemetryConstants.GEN_AI_TOOL_ARGS_KEY, truncateValue(JSON.stringify(run.inputs?.input ?? run.inputs))); + if (run.outputs?.output?.kwargs?.content) span.setAttribute(OpenTelemetryConstants.GEN_AI_TOOL_CALL_RESULT_KEY, truncateValue(JSON.stringify(run.outputs?.output?.kwargs?.content))); + span.setAttribute(OpenTelemetryConstants.GEN_AI_TOOL_TYPE_KEY, "extension"); + if (run.outputs?.output?.tool_call_id) span.setAttribute(OpenTelemetryConstants.GEN_AI_TOOL_CALL_ID_KEY, run.outputs?.output?.tool_call_id); } export function setInputMessagesAttribute(run: Run, span: Span) { @@ -95,216 +63,59 @@ export function setInputMessagesAttribute(run: Run, span: Span) { return; } - // LangChain may provide messages as a direct array or as a single nested array. - // Normalize both shapes so agent/inference inputs are consistently processed. - const preprocess = getScopeType(run) !== "unknown" && messages.length > 0 && Array.isArray(messages[0]) - ? messages[0] as unknown[] - : messages; - const chatMessages: ChatMessage[] = []; - - for (const msg of preprocess) { - if (!msg || typeof msg !== "object") continue; - const msgObj = msg as Record; - const parts = buildPartsFromMessage(msgObj); - if (parts.length === 0) continue; - - const msgType = getMessageType(msgObj); - const role = mapLangChainRole(msgType); - chatMessages.push({ role, parts }); - } + const preprocess = getScopeType(run) === "inference" && messages.length > 0 ? messages[0] : messages; + const processed = preprocess?.map((msg: Record) => { + const content = extractMessageContent(msg); + if (!content) return null; - if (chatMessages.length > 0) { - const wrapper: InputMessages = { version: A365_MESSAGE_SCHEMA_VERSION, messages: chatMessages }; - span.setAttribute(OpenTelemetryConstants.GEN_AI_INPUT_MESSAGES_KEY, serializeMessages(wrapper)); - } -} + const msgType = getMessageType(msg); + if (shouldIncludeInputMessage(msgType)) { + return content; + } + return null; + }) + .filter(Boolean); -// Helper: Extract string content from a message (used for fallback text extraction and system instructions) -function extractStringContent(msg: Record): string | null { - const raw = extractRawContent(msg); - return isString(raw) ? raw : null; + if (processed.length > 0) { + span.setAttribute(OpenTelemetryConstants.GEN_AI_INPUT_MESSAGES_KEY, truncateValue(JSON.stringify(processed))); + } } -// Helper: Extract raw content (string or content block array) from various message formats -function extractRawContent(msg: Record): string | unknown[] | null { - // Simple format: {role: "user", content: string | array} - if (msg.content !== undefined && msg.content !== null) { - if (isString(msg.content)) return msg.content; - if (Array.isArray(msg.content)) return msg.content; +// Helper: Extract message content from various formats +function extractMessageContent(msg: Record): string | null { + // Simple format: {role: "user", content} + if (isString(msg.content)) { + return msg.content; } // LangChain format: {lc_type: "human", lc_kwargs: {content}} if (msg.lc_kwargs && typeof msg.lc_kwargs === "object" && !Array.isArray(msg.lc_kwargs)) { const kwargs = msg.lc_kwargs as Record; if (isString(kwargs.content)) return kwargs.content; - if (Array.isArray(kwargs.content)) return kwargs.content; } - // LangChain v1 serialized class instance format: { lc: 1, type: "constructor", kwargs: {...} } - // `lc: 1` is the LangChain serialization version marker indicating a v1 schema. - // `type: "constructor"` means the object was serialized as a class instance (e.g. HumanMessage, AIMessage) - // that can be reconstructed via its constructor arguments stored in `kwargs`. + // New LangChain format: {lc: 1, type: "constructor", kwargs: {content}} if (msg.lc === 1 && msg.type === "constructor" && msg.kwargs && typeof msg.kwargs === "object" && !Array.isArray(msg.kwargs)) { const kwargs = msg.kwargs as Record; if (isString(kwargs.content)) return kwargs.content; - if (Array.isArray(kwargs.content)) return kwargs.content; } return null; } -// Helper: Map LangChain message type to MessageRole -function mapLangChainRole(msgType: string): MessageRole | string { - switch (msgType) { - case "user": - case "human": - return MessageRole.USER; - case "assistant": - case "ai": - return MessageRole.ASSISTANT; - case "system": - return MessageRole.SYSTEM; - case "tool": - return MessageRole.TOOL; - default: - return msgType; - } -} - -// Helper: Build MessagePart[] from a LangChain message -function buildPartsFromMessage(msg: Record): MessagePart[] { - const parts: MessagePart[] = []; - const rawContent = extractRawContent(msg); - - const addUnknownBlockPart = (blockType: string, block: Record) => { - try { - parts.push({ type: blockType, content: JSON.stringify(block) } as MessagePart); - } catch { - parts.push({ type: blockType, content: "[unserializable]" } as MessagePart); - } - }; - - const addPartFromContentBlock = (block: unknown) => { - if (!block || typeof block !== "object") return; - - const contentBlock = block as Record; - const blockType = contentBlock.type as string | undefined; - if (!blockType) return; - - if (blockType === "text" && isString(contentBlock.text)) { - parts.push({ type: "text", content: contentBlock.text }); - return; - } - - if (blockType === "reasoning" && isString(contentBlock.reasoning)) { - parts.push({ type: "reasoning", content: contentBlock.reasoning }); - return; - } - - if (blockType === "tool_call") { - parts.push({ - type: "tool_call", - name: String(contentBlock.name ?? ""), - id: contentBlock.id != null ? String(contentBlock.id) : undefined, - arguments: contentBlock.args && typeof contentBlock.args === "object" ? contentBlock.args as Record : undefined, - }); - return; - } - - addUnknownBlockPart(blockType, contentBlock); - }; - - if (isString(rawContent)) { - parts.push({ type: "text", content: rawContent }); - } else if (Array.isArray(rawContent)) { - for (const block of rawContent) { - addPartFromContentBlock(block); - } - } - - // Extract tool_calls from the message (AI messages may have a separate tool_calls array) - // Deduplicate by ID to avoid duplicates when tool_calls appear in both content blocks and tool_calls array - const seenToolCallIds = new Set(); - for (const part of parts) { - if (part.type !== "tool_call") continue; - const partId = (part as Record).id; - if (isString(partId)) { - seenToolCallIds.add(partId); - } - } - - for (const toolCall of extractToolCalls(msg)) { - const toolCallId = (toolCall as Record).id; - if (isString(toolCallId) && seenToolCallIds.has(toolCallId)) { - continue; - } - if (isString(toolCallId)) { - seenToolCallIds.add(toolCallId); - } - parts.push(toolCall); - } - - // Fallback: if no parts were built, use text extraction - if (parts.length === 0) { - const textContent = extractStringContent(msg); - if (textContent) { - parts.push({ type: "text", content: textContent }); - } - } - - return parts; -} - -// Helper: Extract tool_calls from a LangChain message -function extractToolCalls(msg: Record): MessagePart[] { - const parts: MessagePart[] = []; - - // Standard format: message.tool_calls[] — check direct, lc_kwargs, and kwargs paths - const directToolCalls = - getNestedValue(msg, "tool_calls") ?? - getNestedValue(msg, "lc_kwargs", "tool_calls") ?? - getNestedValue(msg, "kwargs", "tool_calls"); - if (Array.isArray(directToolCalls)) { - for (const tc of directToolCalls) { - if (!tc || typeof tc !== "object") continue; - const call = tc as Record; - parts.push({ - type: "tool_call", - name: String(call.name ?? ""), - id: call.id != null ? String(call.id) : undefined, - arguments: call.args && typeof call.args === "object" ? call.args as Record : undefined, - }); - } - } - - return parts; -} - -// Helper: Safely get a nested value from a message object -function getNestedValue(obj: Record, ...keys: string[]): unknown { - let current: unknown = obj; - for (const key of keys) { - if (!current || typeof current !== "object" || Array.isArray(current)) return undefined; - current = (current as Record)[key]; - } - return current; -} - // Helper: Determine message type function getMessageType(msg: Record): string { // Simple format if (isString(msg.role)) return msg.role; // LangChain old format if (isString(msg.lc_type)) return msg.lc_type; - // Skip v1 constructor type marker — fall through to id array check - if (isString(msg.type) && msg.type !== "constructor") return msg.type; - // LangChain v1 format - check id array for message type (e.g., ["langchain_core", "messages", "HumanMessage"]) + if (isString(msg.type)) return msg.type; + // LangChain new format - check id array for message type if (Array.isArray(msg.id)) { const lastId = msg.id[msg.id.length - 1]; if (isString(lastId)) { if (lastId.includes("Human")) return "human"; if (lastId.includes("AI")) return "ai"; if (lastId.includes("System")) return "system"; - if (lastId.includes("Tool")) return "tool"; } } return "unknown"; @@ -322,6 +133,12 @@ function getScopeType(run: Run): "agent" | "tool" | "inference" | "unknown" { return "unknown"; } +// Helper: Check if input message should be included based on scope and message type +function shouldIncludeInputMessage(msgType: string): boolean { + // For input messages: all scopes want user/human messages only + return msgType === "user" || msgType === "human"; +} + // Helper: Check if output message should be included based on scope and message type function shouldIncludeOutputMessage(scopeType: string, msgType: string): boolean { if (scopeType === "agent" || scopeType === "inference") { @@ -342,25 +159,19 @@ export function setOutputMessagesAttribute(run: Run, span: Span) { } const scopeType = getScopeType(run); - const outputMessages: OutputMessage[] = []; - - // Helper: process a single message object into an OutputMessage - const processMessage = (msg: Record) => { - const msgType = getMessageType(msg); - if (!shouldIncludeOutputMessage(scopeType, msgType)) return; - - const parts = buildPartsFromMessage(msg); - if (parts.length === 0) return; - - const role = mapLangChainRole(msgType); - outputMessages.push({ role, parts }); - }; + const messages: string[] = []; // Direct messages array (used in agent/chain outputs) if (Array.isArray(outputs.messages)) { - for (const msg of outputs.messages as Record[]) { - processMessage(msg); - } + outputs.messages.forEach((msg: Record) => { + const content = extractMessageContent(msg); + if (!content) return; + + const msgType = getMessageType(msg); + if (shouldIncludeOutputMessage(scopeType, msgType)) { + messages.push(content); + } + }); } // LangChain generations format (used in LLM/inference outputs) @@ -370,14 +181,20 @@ export function setOutputMessagesAttribute(run: Run, span: Span) { gen.forEach((item: Record) => { // Try message property if (item.message && typeof item.message === "object" && !Array.isArray(item.message)) { - processMessage(item.message as Record); + const msg = item.message as Record; + const content = extractMessageContent(msg); + if (!content) { + return; + } + + const msgType = getMessageType(msg); + if (shouldIncludeOutputMessage(scopeType, msgType)) { + messages.push(content); + } } // Try direct text property (for generation items) else if (isString(item.text) && scopeType === "inference") { - outputMessages.push({ - role: MessageRole.ASSISTANT, - parts: [{ type: "text", content: item.text }], - }); + messages.push(item.text); } }); } @@ -386,29 +203,29 @@ export function setOutputMessagesAttribute(run: Run, span: Span) { // Check for direct message object (some models return this) if (outputs.message && typeof outputs.message === "object" && !Array.isArray(outputs.message)) { - processMessage(outputs.message as Record); + const msg = outputs.message as Record; + const content = extractMessageContent(msg); + if (content) { + const msgType = getMessageType(msg); + if (shouldIncludeOutputMessage(scopeType, msgType)) { + messages.push(content); + } + } } - if (outputMessages.length > 0) { - const wrapper: OutputMessages = { version: A365_MESSAGE_SCHEMA_VERSION, messages: outputMessages }; - span.setAttribute(OpenTelemetryConstants.GEN_AI_OUTPUT_MESSAGES_KEY, serializeMessages(wrapper)); + if (messages.length > 0) { + span.setAttribute(OpenTelemetryConstants.GEN_AI_OUTPUT_MESSAGES_KEY, truncateValue(JSON.stringify(messages))); } } // Model - Helper to extract model name from run export function getModel(run: Run): string | undefined { - return [ - // v1: response_metadata directly on message - run.outputs?.generations?.[0]?.[0]?.message?.response_metadata?.model_name, - // v0: response_metadata nested under kwargs - run.outputs?.generations?.[0]?.[0]?.message?.kwargs?.response_metadata?.model_name, - // Metadata paths (both v0 and v1) - run.extra?.metadata?.ls_model_name, - run.extra?.invocation_params?.model, - run.extra?.invocation_params?.model_name, - ] - .map((v) => (v != null ? String(v).trim() : "")) - .find((v) => v.length > 0); + return [run.outputs?.generations?.[0]?.[0]?.message?.kwargs?.response_metadata?.model_name, + run.extra?.metadata?.ls_model_name, + run.extra?.invocation_params?.model, + run.extra?.invocation_params?.model_name] + .map((v) => (v != null ? String(v).trim() : "")) + .find((v) => v.length > 0); } // Model - Set model attribute on span @@ -446,35 +263,25 @@ export function setSystemInstructionsAttribute(run: Run, span: Span) { } const prompts = Array.isArray(inputs.prompts) ? inputs.prompts.map(p => String(p ?? "").trim()).filter(Boolean).join("\n") : ""; - if (prompts) return span.setAttribute(OpenTelemetryConstants.GEN_AI_SYSTEM_INSTRUCTIONS_KEY, prompts); - - // Check both flat and nested message arrays - const rawMessages = Array.isArray(inputs.messages) ? inputs.messages : []; - const flatMessages = rawMessages.length > 0 && Array.isArray(rawMessages[0]) ? rawMessages[0] as unknown[] : rawMessages; - const systemText = flatMessages - .filter((m: unknown) => { - if (!m || typeof m !== "object") return false; - const msgType = getMessageType(m as Record); - return msgType === "system"; - }) - .map((m: unknown) => extractStringContent(m as Record) ?? "") - .map((s: string) => s.trim()) + if (prompts) return span.setAttribute(OpenTelemetryConstants.GEN_AI_SYSTEM_INSTRUCTIONS_KEY, truncateValue(prompts)); + + const messages = Array.isArray(inputs.messages) ? inputs.messages : []; + const systemText = messages + .filter((m: Record) => m.lc_type === "system") + .map((m: Record) => String((m.lc_kwargs as Record | undefined)?.content ?? "").trim()) .filter(Boolean) .join("\n"); - if (systemText) span.setAttribute(OpenTelemetryConstants.GEN_AI_SYSTEM_INSTRUCTIONS_KEY, systemText); + if (systemText) span.setAttribute(OpenTelemetryConstants.GEN_AI_SYSTEM_INSTRUCTIONS_KEY, truncateValue(systemText)); } // Tokens (input and output) export function setTokenAttributes(run: Run, span: Span) { // Try multiple paths to find usage metadata (LLM direct/kwargs/response_metadata, agent calls, and chain/model_request outputs) - // v1: usage_metadata is often on the last AI message in outputs.messages - const lastMsg = Array.isArray(run.outputs?.messages) ? run.outputs.messages[run.outputs.messages.length - 1] : undefined; - const usage = + const usage = run.outputs?.generations?.[0]?.[0]?.message?.usage_metadata || run.outputs?.generations?.[0]?.[0]?.message?.kwargs?.usage_metadata || - run.outputs?.generations?.[0]?.[0]?.message?.response_metadata?.tokenUsage || run.outputs?.generations?.[0]?.[0]?.message?.kwargs?.response_metadata?.tokenUsage || - lastMsg?.usage_metadata || + run.outputs?.messages?.[1]?.usage_metadata || run.outputs?.message?.response_metadata?.usage || run.outputs?.message?.response_metadata?.tokenUsage || run.outputs?.messages @@ -486,15 +293,11 @@ export function setTokenAttributes(run: Run, span: Span) { } const usageObj = usage as Record; - // Support both usage_metadata shape (input_tokens/output_tokens) and - // tokenUsage shape (promptTokens/completionTokens) from LangChain OpenAI provider - const inputTokens = usageObj.input_tokens ?? usageObj.promptTokens; - const outputTokens = usageObj.output_tokens ?? usageObj.completionTokens; - if (typeof inputTokens === "number") { - span.setAttribute(OpenTelemetryConstants.GEN_AI_USAGE_INPUT_TOKENS_KEY, inputTokens); + if (typeof usageObj.input_tokens === "number") { + span.setAttribute(OpenTelemetryConstants.GEN_AI_USAGE_INPUT_TOKENS_KEY, usageObj.input_tokens); } - if (typeof outputTokens === "number") { - span.setAttribute(OpenTelemetryConstants.GEN_AI_USAGE_OUTPUT_TOKENS_KEY, outputTokens); + if (typeof usageObj.output_tokens === "number") { + span.setAttribute(OpenTelemetryConstants.GEN_AI_USAGE_OUTPUT_TOKENS_KEY, usageObj.output_tokens); } } diff --git a/packages/agents-a365-observability-extensions-langchain/src/tracer.ts b/packages/agents-a365-observability-extensions-langchain/src/tracer.ts index f6a28241..f4175ac3 100644 --- a/packages/agents-a365-observability-extensions-langchain/src/tracer.ts +++ b/packages/agents-a365-observability-extensions-langchain/src/tracer.ts @@ -4,7 +4,7 @@ import { context, trace, Span, SpanKind, SpanStatusCode, Tracer } from "@opentelemetry/api"; import { BaseTracer, Run } from "@langchain/core/tracers/base"; import { isTracingSuppressed } from "@opentelemetry/core"; -import { logger, OpenTelemetryConstants } from "@microsoft/agents-a365-observability"; +import { logger, OpenTelemetryConstants, truncateValue } from "@microsoft/agents-a365-observability"; import * as Utils from "./Utils"; type RunWithSpan = { run: Run; span: Span; startTime: number; lastAccessTime: number }; @@ -12,13 +12,15 @@ type RunWithSpan = { run: Run; span: Span; startTime: number; lastAccessTime: nu export class LangChainTracer extends BaseTracer { private static readonly MAX_RUNS = 10_000; private tracer: Tracer; + private isContentRecordingEnabled: boolean; private runs = new Map(); private parentByRunId = new Map(); - constructor(tracer: Tracer) { + constructor(tracer: Tracer, options?: { isContentRecordingEnabled?: boolean }) { super(); this.tracer = tracer; + this.isContentRecordingEnabled = options?.isContentRecordingEnabled ?? false; } name = "OpenTelemetryLangChainTracer"; @@ -106,7 +108,7 @@ export class LangChainTracer extends BaseTracer { if (run.error) { span.setStatus({ code: SpanStatusCode.ERROR }); - span.setAttribute(OpenTelemetryConstants.ERROR_MESSAGE_KEY, String(run.error)); + span.setAttribute(OpenTelemetryConstants.ERROR_MESSAGE_KEY, truncateValue(String(run.error))); } else { span.setStatus({ code: SpanStatusCode.OK }); @@ -120,11 +122,14 @@ export class LangChainTracer extends BaseTracer { Utils.setSessionIdAttribute(run, span); Utils.setTokenAttributes(run, span); - // Content attributes — always recorded (aligned with Python/.NET SDKs) - Utils.setToolAttributes(run, span); - Utils.setInputMessagesAttribute(run, span); - Utils.setOutputMessagesAttribute(run, span); - Utils.setSystemInstructionsAttribute(run, span); + // Content attributes gated by content recording setting + const contentRecording = this.isContentRecordingEnabled; + if (contentRecording) { + Utils.setToolAttributes(run, span); + Utils.setInputMessagesAttribute(run, span); + Utils.setOutputMessagesAttribute(run, span); + Utils.setSystemInstructionsAttribute(run, span); + } } catch (error) { logger.error(`[LangChainTracer] Error setting span attributes for run ${run.name}: ${error instanceof Error ? error.message : String(error)}`); diff --git a/packages/agents-a365-observability-extensions-openai/src/OpenAIAgentsTraceInstrumentor.ts b/packages/agents-a365-observability-extensions-openai/src/OpenAIAgentsTraceInstrumentor.ts index 968a34af..0fe4e2bf 100644 --- a/packages/agents-a365-observability-extensions-openai/src/OpenAIAgentsTraceInstrumentor.ts +++ b/packages/agents-a365-observability-extensions-openai/src/OpenAIAgentsTraceInstrumentor.ts @@ -27,6 +27,11 @@ export interface OpenAIAgentsInstrumentationConfig extends InstrumentationConfig * Defaults to false. */ suppressInvokeAgentInput?: boolean; + /** + * Whether to enable content recording (input/output messages, tool args, etc.). + * @default false + */ + isContentRecordingEnabled?: boolean; } /** @@ -101,6 +106,7 @@ export class OpenAIAgentsTraceInstrumentor extends InstrumentationBase = new Map(); private readonly otelSpans: Map = new Map(); private readonly tokens: Map = new Map(); @@ -50,9 +50,23 @@ export class OpenAIAgentsTraceProcessor implements TracingProcessor { ['generation' + Constants.GEN_AI_REQUEST_CONTENT_KEY, OpenTelemetryConstants.GEN_AI_INPUT_MESSAGES_KEY], ]); - constructor(tracer: OtelTracer, options?: { suppressInvokeAgentInput?: boolean }) { + constructor(tracer: OtelTracer, options?: { suppressInvokeAgentInput?: boolean; isContentRecordingEnabled?: boolean }) { this.tracer = tracer; this.suppressInvokeAgentInput = options?.suppressInvokeAgentInput ?? false; + this.isContentRecordingEnabled = options?.isContentRecordingEnabled ?? false; + } + + private static readonly CONTENT_KEYS = new Set([ + OpenTelemetryConstants.GEN_AI_INPUT_MESSAGES_KEY, + OpenTelemetryConstants.GEN_AI_OUTPUT_MESSAGES_KEY, + OpenTelemetryConstants.GEN_AI_TOOL_ARGS_KEY, + OpenTelemetryConstants.GEN_AI_TOOL_CALL_RESULT_KEY, + Constants.GEN_AI_REQUEST_CONTENT_KEY, + Constants.GEN_AI_RESPONSE_CONTENT_KEY, + ]); + + private static isContentKey(key: string): boolean { + return OpenAIAgentsTraceProcessor.CONTENT_KEYS.has(key); } private getNewKey(spanType: string, key: string): string | null { @@ -194,18 +208,19 @@ export class OpenAIAgentsTraceProcessor implements TracingProcessor { */ private processSpanData(otelSpan: OtelSpan, data: SpanData, traceId: string): void { const type = data.type; + const contentRecording = this.isContentRecordingEnabled; switch (type) { case 'response': - this.processResponseSpanData(otelSpan, data); + this.processResponseSpanData(otelSpan, data, contentRecording); break; case 'generation': - this.processGenerationSpanData(otelSpan, data, traceId); + this.processGenerationSpanData(otelSpan, data, traceId, contentRecording); break; case 'function': - this.processFunctionSpanData(otelSpan, data, traceId); + this.processFunctionSpanData(otelSpan, data, traceId, contentRecording); break; case 'mcp_tools': @@ -225,7 +240,7 @@ export class OpenAIAgentsTraceProcessor implements TracingProcessor { /** * Process response span data */ - private processResponseSpanData(otelSpan: OtelSpan, data: SpanData): void { + private processResponseSpanData(otelSpan: OtelSpan, data: SpanData, contentRecording: boolean): void { const responseData = data as Record; // Handle both formats: _response/_input (actual format) and response/input (legacy format) const responseObj = responseData._response || responseData.response; @@ -233,18 +248,15 @@ export class OpenAIAgentsTraceProcessor implements TracingProcessor { if (responseObj) { const resp = responseObj as Record; - // Store the output field as structured OutputMessages (always use versioned envelope) - if (resp.output != null) { - if (Array.isArray(resp.output)) { - const structured = buildStructuredOutputMessages(resp.output as Array>); + // Store the output field for GEN_AI_RESPONSE_CONTENT_KEY + if (resp.output && contentRecording) { + if (typeof resp.output === 'string') { + otelSpan.setAttribute(OpenTelemetryConstants.GEN_AI_OUTPUT_MESSAGES_KEY, truncateValue(resp.output)); + } else { otelSpan.setAttribute( OpenTelemetryConstants.GEN_AI_OUTPUT_MESSAGES_KEY, - serializeMessages(structured) + truncateValue(this.buildOutputMessages(resp.output as Array<{ role: string; content: Array<{ type: string; text: string }> }>)) ); - } else { - // String or non-array object — wrap as raw content - const structured = wrapRawContentAsOutputMessages(resp.output); - otelSpan.setAttribute(OpenTelemetryConstants.GEN_AI_OUTPUT_MESSAGES_KEY, serializeMessages(structured)); } } @@ -261,37 +273,61 @@ export class OpenAIAgentsTraceProcessor implements TracingProcessor { otelSpan.updateName(`${InferenceOperationType.CHAT} ${modelName}`); } - if (inputObj != null && !this.suppressInvokeAgentInput) { + if (inputObj && !this.suppressInvokeAgentInput && contentRecording) { if (typeof inputObj === 'string') { try { const parsed = JSON.parse(inputObj as string); if (Array.isArray(parsed)) { - const structured = buildStructuredInputMessages(parsed); otelSpan.setAttribute( OpenTelemetryConstants.GEN_AI_INPUT_MESSAGES_KEY, - serializeMessages(structured) + truncateValue(this.buildInputMessages(parsed)) ); return; } } catch { - // If parsing fails, wrap raw string in versioned envelope + // If parsing fails, fall back to raw string behavior } - const wrappedInput = wrapRawContentAsInputMessages(inputObj); - otelSpan.setAttribute(OpenTelemetryConstants.GEN_AI_INPUT_MESSAGES_KEY, serializeMessages(wrappedInput)); + otelSpan.setAttribute(OpenTelemetryConstants.GEN_AI_INPUT_MESSAGES_KEY, truncateValue(inputObj as string)); } else if (Array.isArray(inputObj)) { - const structured = buildStructuredInputMessages(inputObj as Array<{ role: string; content: string | unknown[] | unknown }>); + // build the input messages from array otelSpan.setAttribute( OpenTelemetryConstants.GEN_AI_INPUT_MESSAGES_KEY, - serializeMessages(structured) + truncateValue(this.buildInputMessages(inputObj)) ); } } } + private buildInputMessages(arr: Array<{ role: string; content: string }>): string { + const userTexts = arr + .filter((m) => m && m.role === 'user' && typeof m.content === 'string') + .map((m) => m.content); + + return JSON.stringify(userTexts.length ? userTexts : arr); + } + + private buildOutputMessages(arr: Array<{ role: string; content: Array<{ type: string; text: string }> }>): string { + const userTexts: string[] = []; + + for (const { content } of arr) { + if (!Array.isArray(content)) { + continue; + } + + for (const { type, text } of content) { + if (type === 'output_text' && typeof text === 'string') { + userTexts.push(text); + } + } + } + + return JSON.stringify(userTexts.length ? userTexts : arr); + } + /** * Process generation span data */ - private processGenerationSpanData(otelSpan: OtelSpan, data: SpanData, traceId: string): void { + private processGenerationSpanData(otelSpan: OtelSpan, data: SpanData, traceId: string, contentRecording: boolean): void { const attrs = Utils.getAttributesFromGenerationSpanData(data); Object.entries(attrs).forEach(([key, value]) => { const shouldExcludeKey = key === Constants.GEN_AI_EXECUTION_PAYLOAD_KEY; @@ -299,7 +335,9 @@ export class OpenAIAgentsTraceProcessor implements TracingProcessor { const newKey = this.getNewKey(data.type, key); const resolvedKey = newKey || key; if (resolvedKey !== OpenTelemetryConstants.GEN_AI_INPUT_MESSAGES_KEY || !this.suppressInvokeAgentInput) { - otelSpan.setAttribute(resolvedKey, value as string | number | boolean); + if (!OpenAIAgentsTraceProcessor.isContentKey(resolvedKey) || contentRecording) { + otelSpan.setAttribute(resolvedKey, value as string | number | boolean); + } } } }); @@ -317,17 +355,19 @@ export class OpenAIAgentsTraceProcessor implements TracingProcessor { /** * Process function/tool span data */ - private processFunctionSpanData(otelSpan: OtelSpan, data: SpanData, traceId: string): void { + private processFunctionSpanData(otelSpan: OtelSpan, data: SpanData, traceId: string, contentRecording: boolean): void { const functionData = data as Record; const attrs = Utils.getAttributesFromFunctionSpanData(data); Object.entries(attrs).forEach(([key, value]) => { if (value !== null && value !== undefined) { const newKey = this.getNewKey(data.type, key); const resolvedKey = newKey || key; - otelSpan.setAttribute(resolvedKey, value as string | number | boolean); + if (!OpenAIAgentsTraceProcessor.isContentKey(resolvedKey) || contentRecording) { + otelSpan.setAttribute(resolvedKey, value as string | number | boolean); + } } + otelSpan.setAttribute(OpenTelemetryConstants.GEN_AI_TOOL_TYPE_KEY, 'function'); }); - otelSpan.setAttribute(OpenTelemetryConstants.GEN_AI_TOOL_TYPE_KEY, 'function'); this.stampCustomParent(otelSpan, traceId); // Use function name from data instead of span name diff --git a/packages/agents-a365-observability-extensions-openai/src/Utils.ts b/packages/agents-a365-observability-extensions-openai/src/Utils.ts index 43eb1260..5049e591 100644 --- a/packages/agents-a365-observability-extensions-openai/src/Utils.ts +++ b/packages/agents-a365-observability-extensions-openai/src/Utils.ts @@ -3,17 +3,7 @@ // ------------------------------------------------------------------------------ import { SpanStatusCode } from '@opentelemetry/api'; -import { - OpenTelemetryConstants, - serializeMessages, - safeSerializeToJson, - A365_MESSAGE_SCHEMA_VERSION, - MessageRole, - InputMessages, - OutputMessages, - MAX_SPAN_SIZE_BYTES, -} from '@microsoft/agents-a365-observability'; -import type { ChatMessage, OutputMessage, MessagePart } from '@microsoft/agents-a365-observability'; +import { OpenTelemetryConstants, truncateValue } from '@microsoft/agents-a365-observability'; import * as Constants from './Constants'; import { Span as AgentsSpan, SpanData } from '@openai/agents-core/dist/tracing/spans'; @@ -24,9 +14,9 @@ import { Span as AgentsSpan, SpanData } from '@openai/agents-core/dist/tracing/s */ export function safeJsonDumps(obj: unknown): string { try { - return JSON.stringify(obj); + return truncateValue(JSON.stringify(obj)); } catch { - return String(obj); + return truncateValue(String(obj)); } } @@ -98,11 +88,11 @@ export function getAttributesFromGenerationSpanData(data: SpanData): Record : String(funcData.input), - 'arguments' - ); + if (funcData.input) { + attributes[Constants.GEN_AI_REQUEST_CONTENT_KEY] = + typeof funcData.input === 'string' ? truncateValue(funcData.input) : safeJsonDumps(funcData.input); } if (funcData.output !== undefined && funcData.output !== null) { - attributes[Constants.GEN_AI_RESPONSE_CONTENT_KEY] = safeSerializeToJson( - typeof funcData.output === 'object' ? funcData.output as Record : String(funcData.output), - 'result' - ); + const output = typeof funcData.output === 'object' ? safeJsonDumps(funcData.output) : truncateValue(String(funcData.output)); + attributes[Constants.GEN_AI_RESPONSE_CONTENT_KEY] = output; } return attributes; @@ -187,223 +173,31 @@ export function getAttributesFromResponse(response: unknown): Record): { code: SpanStatusCode; message?: string } { - if (span.error) { - const message = span.error.message || span.error.data || 'Unknown error'; - return { - code: SpanStatusCode.ERROR, - message: String(message), - }; - } - - return { code: SpanStatusCode.OK }; -} - -// --------------------------------------------------------------------------- -// Structured message builders (OTEL gen-ai message format) -// --------------------------------------------------------------------------- - -type OpenAIInputMessage = { role: string; content: string | unknown[] | unknown }; -type OpenAIOutputItem = { role?: string; content?: unknown[]; type?: string; text?: string; [key: string]: unknown }; - -/** - * Map an OpenAI role string to a MessageRole value. - */ -function mapOpenAIRole(role: string): MessageRole | string { - switch (role) { - case 'user': - return MessageRole.USER; - case 'assistant': - return MessageRole.ASSISTANT; - case 'system': - return MessageRole.SYSTEM; - case 'tool': - return MessageRole.TOOL; - default: - return role; - } -} - -function getModalityFromMimeType(mimeType: unknown): string { - return String(mimeType ?? 'file').split('/')[0] || 'file'; -} - -function mapGenericBlock(blockType: string | undefined, block: Record): MessagePart { - return { type: blockType ?? 'unknown', content: safeJsonDumps(block) } as MessagePart; -} - -function parseToolCallArguments(args: unknown): Record | undefined { - if (typeof args === 'string') { - try { - return JSON.parse(args) as Record; - } catch { - return { raw: args }; - } - } - - if (args && typeof args === 'object') { - return args as Record; - } - - return undefined; -} - -function getToolCallId(block: Record): string | undefined { - if (block.call_id != null) return String(block.call_id); - if (block.id != null) return String(block.id); - return undefined; -} - -function wrapRawContentAsMessages(raw: unknown, role: MessageRole): InputMessages | OutputMessages { - const content = typeof raw === 'string' ? raw : safeJsonDumps(raw); - return { - version: A365_MESSAGE_SCHEMA_VERSION, - messages: [{ role, parts: [{ type: 'text', content }] }], - }; -} +export function getAttributesFromInput(input: unknown): Record { + const attributes: Record = {}; -/** - * Map an OpenAI input content block to a MessagePart. - */ -function mapInputContentBlock(block: Record): MessagePart { - const blockType = block.type as string | undefined; - switch (blockType) { - case 'input_text': - return { type: 'text', content: String(block.text ?? '') }; - case 'input_image': - return { type: 'blob', modality: 'image', ...stripBinaryFields(block) } as MessagePart; - case 'input_file': - return { - type: 'file' as string, - modality: getModalityFromMimeType(block.mime_type), - ...stripBinaryFields(block), - } as MessagePart; - default: - return mapGenericBlock(blockType, block); + if (typeof input === 'string') { + attributes[Constants.GEN_AI_REQUEST_CONTENT_KEY] = input; + } else if (Array.isArray(input)) { + attributes[Constants.GEN_AI_REQUEST_CONTENT_KEY] = safeJsonDumps(input); } -} -/** - * Strip large binary fields from a content block for telemetry. - */ -function stripBinaryFields(block: Record): Record { - const result: Record = {}; - for (const [key, value] of Object.entries(block)) { - if (key === 'type') continue; - if (typeof value === 'string' && value.length > MAX_SPAN_SIZE_BYTES) { - result[key] = '[truncated]'; - } else { - result[key] = value; - } - } - return result; + return attributes; } /** - * Map an OpenAI output content block to a MessagePart. + * Get span status from OpenAI Agents SDK span */ -function mapOutputContentBlock(block: Record): MessagePart { - const blockType = block.type as string | undefined; - switch (blockType) { - case 'output_text': - return { type: 'text', content: String(block.text ?? '') }; - case 'refusal': - return { type: 'text', content: String(block.refusal ?? '') }; - case 'tool_call': - case 'function_call': { - const parsedArgs = parseToolCallArguments(block.arguments ?? block.args); +export function getSpanStatus(span: AgentsSpan): { code: SpanStatusCode; message?: string } { + if (span.error) { + const message = span.error.message || span.error.data || 'Unknown error'; return { - type: 'tool_call', - name: String(block.name ?? block.function ?? ''), - id: getToolCallId(block), - arguments: parsedArgs, + code: SpanStatusCode.ERROR, + message: String(message), }; } - case 'reasoning': - return { type: 'reasoning', content: String(block.text ?? block.content ?? '') }; - default: - return mapGenericBlock(blockType, block); - } -} - -/** - * Build structured InputMessages from an OpenAI _input message array. - * Includes all roles (system, user, assistant, tool). - */ -export function buildStructuredInputMessages( - arr: OpenAIInputMessage[] -): InputMessages { - const messages: ChatMessage[] = []; - - for (const msg of arr) { - if (!msg || typeof msg !== 'object') continue; - - const role = mapOpenAIRole(msg.role ?? 'user'); - let parts: MessagePart[]; - - if (typeof msg.content === 'string') { - parts = [{ type: 'text', content: msg.content }]; - } else if (Array.isArray(msg.content)) { - parts = (msg.content as Record[]).map(mapInputContentBlock); - } else { - parts = [{ type: 'text', content: safeJsonDumps(msg.content) }]; - } - - messages.push({ role, parts }); - } - - return { version: A365_MESSAGE_SCHEMA_VERSION, messages }; -} - -/** - * Build structured OutputMessages from an OpenAI response.output array. - */ -export function buildStructuredOutputMessages( - arr: OpenAIOutputItem[] -): OutputMessages { - const messages: OutputMessage[] = []; - - for (const item of arr) { - if (!item || typeof item !== 'object') continue; - - const role = mapOpenAIRole(item.role ?? 'assistant'); - - // Items with a content array (standard response format) - if (Array.isArray(item.content)) { - const parts = (item.content as Record[]).map(mapOutputContentBlock); - messages.push({ role, parts }); - continue; - } - - // Items that are themselves content blocks (e.g., type: 'message' with text) - if (item.type && typeof item.type === 'string') { - const parts = [mapOutputContentBlock(item as Record)]; - messages.push({ role, parts }); - continue; - } - - // Fallback: stringify the item - messages.push({ - role, - parts: [{ type: 'text', content: safeJsonDumps(item) }], - }); - } - - return { version: A365_MESSAGE_SCHEMA_VERSION, messages }; -} - -/** - * Wrap opaque raw content as InputMessages (for generation span data). - */ -export function wrapRawContentAsInputMessages(raw: unknown): InputMessages { - return wrapRawContentAsMessages(raw, MessageRole.USER) as InputMessages; -} -/** - * Wrap opaque raw content as OutputMessages (for generation span data). - */ -export function wrapRawContentAsOutputMessages(raw: unknown): OutputMessages { - return wrapRawContentAsMessages(raw, MessageRole.ASSISTANT) as OutputMessages; + return { code: SpanStatusCode.OK }; } diff --git a/packages/agents-a365-observability-hosting/src/utils/TurnContextUtils.ts b/packages/agents-a365-observability-hosting/src/utils/TurnContextUtils.ts index acc7285b..b6fc652b 100644 --- a/packages/agents-a365-observability-hosting/src/utils/TurnContextUtils.ts +++ b/packages/agents-a365-observability-hosting/src/utils/TurnContextUtils.ts @@ -29,8 +29,9 @@ export function getCallerBaggagePairs(turnContext: TurnContext): Array<[string, const from = turnContext.activity.from; const upn = from.agenticUserId; + const userId = from.aadObjectId || from.agenticUserId || from.id; const pairs: Array<[string, string | undefined]> = [ - [OpenTelemetryConstants.USER_ID_KEY, from.aadObjectId], + [OpenTelemetryConstants.USER_ID_KEY, userId], [OpenTelemetryConstants.USER_NAME_KEY, from.name], [OpenTelemetryConstants.USER_EMAIL_KEY, upn], [OpenTelemetryConstants.GEN_AI_CALLER_AGENT_APPLICATION_ID_KEY, from.agenticAppBlueprintId] diff --git a/packages/agents-a365-observability/src/index.ts b/packages/agents-a365-observability/src/index.ts index 6931e03f..d8fdb96e 100644 --- a/packages/agents-a365-observability/src/index.ts +++ b/packages/agents-a365-observability/src/index.ts @@ -80,13 +80,10 @@ export { InferenceScope } from './tracing/scopes/InferenceScope'; export { OutputScope } from './tracing/scopes/OutputScope'; export { logger, setLogger, getLogger, resetLogger, formatError } from './utils/logging'; export type { ILogger } from './utils/logging'; -export { safeSerializeToJson } from './tracing/util'; - -// Message utilities -export { serializeMessages, normalizeInputMessages, normalizeOutputMessages } from './tracing/message-utils'; +export { truncateValue, MAX_ATTRIBUTE_LENGTH } from './tracing/util'; // Exporter utilities -export { isPerRequestExportEnabled, MAX_SPAN_SIZE_BYTES } from './tracing/exporter/utils'; +export { isPerRequestExportEnabled } from './tracing/exporter/utils'; // Configuration export * from './configuration'; diff --git a/packages/agents-a365-observability/src/tracing/exporter/Agent365Exporter.ts b/packages/agents-a365-observability/src/tracing/exporter/Agent365Exporter.ts index 0fc42e56..0e23de28 100644 --- a/packages/agents-a365-observability/src/tracing/exporter/Agent365Exporter.ts +++ b/packages/agents-a365-observability/src/tracing/exporter/Agent365Exporter.ts @@ -81,8 +81,8 @@ interface OTLPStatus { * Observability span exporter for Agent365: * - Partitions spans by (tenantId, agentId) * - Builds OTLP-like JSON: resourceSpans -> scopeSpans -> spans - * - POSTs per group to https://{endpoint}/observability/tenants/{tenantId}/otlp/agents/{agentId}/traces?api-version=1 - * or, when useS2SEndpoint is true, https://{endpoint}/observabilityService/tenants/{tenantId}/otlp/agents/{agentId}/traces?api-version=1 + * - POSTs per group to https://{endpoint}/observability/tenants/{tenantId}/agents/{agentId}/traces?api-version=1 + * or, when useS2SEndpoint is true, https://{endpoint}/observabilityService/tenants/{tenantId}/agents/{agentId}/traces?api-version=1 * - Adds Bearer token via token_resolver(agentId, tenantId) */ export class Agent365Exporter implements SpanExporter { @@ -170,8 +170,10 @@ export class Agent365Exporter implements SpanExporter { const payload = this.buildExportRequest(spans); const body = JSON.stringify(payload); // Select endpoint path based on S2S flag (includes tenantId in path) - const servicePrefix = this.options.useS2SEndpoint ? '/observabilityService' : '/observability'; - const endpointRelativePath = `${servicePrefix}/tenants/${encodeURIComponent(tenantId)}/otlp/agents/${encodeURIComponent(agentId)}/traces`; + const endpointRelativePath = + this.options.useS2SEndpoint + ? `/observabilityService/tenants/${encodeURIComponent(tenantId)}/agents/${encodeURIComponent(agentId)}/traces` + : `/observability/tenants/${encodeURIComponent(tenantId)}/agents/${encodeURIComponent(agentId)}/traces`; let url: string; const domainOverride = getAgent365ObservabilityDomainOverride(this.configProvider); diff --git a/packages/agents-a365-observability/src/tracing/exporter/Agent365ExporterOptions.ts b/packages/agents-a365-observability/src/tracing/exporter/Agent365ExporterOptions.ts index f99f9d09..45523e8c 100644 --- a/packages/agents-a365-observability/src/tracing/exporter/Agent365ExporterOptions.ts +++ b/packages/agents-a365-observability/src/tracing/exporter/Agent365ExporterOptions.ts @@ -21,7 +21,7 @@ export type TokenResolver = (agentId: string, tenantId: string) => string | null * @property {ClusterCategory | string} clusterCategory Environment / cluster category (e.g. ClusterCategory.preprod, ClusterCategory.prod, default to ClusterCategory.prod). * @property {TokenResolver} [tokenResolver] Optional delegate to obtain an auth token. If omitted the exporter will * fall back to reading the cached token (AgenticTokenCacheInstance.getObservabilityToken). - * @property {boolean} [useS2SEndpoint] When true, exporter will POST to the S2S path (/observabilityService/tenants/{tenantId}/otlp/agents/{agentId}/traces). + * @property {boolean} [useS2SEndpoint] When true, exporter will POST to the S2S path (/observabilityService/tenants/{tenantId}/agents/{agentId}/traces). * @property {number} maxQueueSize Maximum span queue size before drops occur (passed to BatchSpanProcessor). * @property {number} scheduledDelayMilliseconds Delay between automatic batch flush attempts. * @property {number} exporterTimeoutMilliseconds Maximum time (ms) the BatchSpanProcessor waits for the entire export() call to complete before giving up. Covers partitioning, token resolution, and all HTTP retries. diff --git a/packages/agents-a365-observability/src/tracing/exporter/utils.ts b/packages/agents-a365-observability/src/tracing/exporter/utils.ts index a3429dc4..c924634d 100644 --- a/packages/agents-a365-observability/src/tracing/exporter/utils.ts +++ b/packages/agents-a365-observability/src/tracing/exporter/utils.ts @@ -152,9 +152,7 @@ export function isPerRequestExportEnabled( const provider = configProvider ?? defaultPerRequestSpanProcessorConfigurationProvider; const enabled = provider.getConfiguration().isPerRequestExportEnabled; - if (enabled) { - logger.info('[Agent365Exporter] Per-request export is enabled'); - } + logger.info(`[Agent365Exporter] Per-request export enabled: ${enabled}`); return enabled; } diff --git a/packages/agents-a365-observability/src/tracing/util.ts b/packages/agents-a365-observability/src/tracing/util.ts index 94a6505e..b5351b2a 100644 --- a/packages/agents-a365-observability/src/tracing/util.ts +++ b/packages/agents-a365-observability/src/tracing/util.ts @@ -22,6 +22,28 @@ export const isAgent365ExporterEnabled = ( return provider.getConfiguration().isObservabilityExporterEnabled; }; +/** + * Maximum length for span attribute values. + * Values exceeding this limit will be truncated with a suffix. + */ +export const MAX_ATTRIBUTE_LENGTH = 8_192; + +const TRUNCATION_SUFFIX = '...[truncated]'; + +/** + * Truncate a string value to {@link MAX_ATTRIBUTE_LENGTH} characters. + * If the value exceeds the limit, it is trimmed and a truncation suffix is appended, + * with the total length capped at exactly {@link MAX_ATTRIBUTE_LENGTH}. + * @param value The string to truncate + * @returns The original string if within limits, otherwise the truncated string + */ +export function truncateValue(value: string): string { + if (value.length > MAX_ATTRIBUTE_LENGTH) { + return value.substring(0, MAX_ATTRIBUTE_LENGTH - TRUNCATION_SUFFIX.length) + TRUNCATION_SUFFIX; + } + return value; +} + /** * Ensures the value is always a JSON-parseable string. * - Objects are serialized via JSON.stringify. diff --git a/packages/agents-a365-tooling-extensions-claude/src/McpToolRegistrationService.ts b/packages/agents-a365-tooling-extensions-claude/src/McpToolRegistrationService.ts index 4c63730f..fa138b60 100644 --- a/packages/agents-a365-tooling-extensions-claude/src/McpToolRegistrationService.ts +++ b/packages/agents-a365-tooling-extensions-claude/src/McpToolRegistrationService.ts @@ -64,10 +64,7 @@ export class McpToolRegistrationService { const tools: McpClientTool[] = []; for (const server of servers) { - // server.headers contains the per-audience Authorization token set by listToolServers. - // Merge with non-auth headers (channelId, user-agent); server.headers auth takes precedence. - const baseHeaders = Utility.GetToolRequestHeaders(authToken, turnContext, options); - const headers = { ...baseHeaders, ...server.headers }; + const headers: Record = Utility.GetToolRequestHeaders(authToken, turnContext, options); // Add each server to the config object mcpServers[server.mcpServerName] = { @@ -78,7 +75,10 @@ export class McpToolRegistrationService { let clientTools = await this.configService.getMcpClientTools( server.mcpServerName, - { url: server.url, headers: headers } as MCPServerConfig, + { + url: server.url, + headers: headers + } as MCPServerConfig, ); // Claude will add a prefix to the tool name based on the server name. diff --git a/packages/agents-a365-tooling-extensions-langchain/src/McpToolRegistrationService.ts b/packages/agents-a365-tooling-extensions-langchain/src/McpToolRegistrationService.ts index 805308f3..615d2e8f 100644 --- a/packages/agents-a365-tooling-extensions-langchain/src/McpToolRegistrationService.ts +++ b/packages/agents-a365-tooling-extensions-langchain/src/McpToolRegistrationService.ts @@ -79,9 +79,8 @@ export class McpToolRegistrationService { const mcpServers: Record = {}; for (const server of servers) { - // Merge base headers (channel, user-agent) with per-audience Authorization from server.headers - const baseHeaders: Record = Utility.GetToolRequestHeaders(authToken, turnContext, options); - const headers: Record = { ...baseHeaders, ...server.headers }; + // Compose headers if values are available + const headers: Record = Utility.GetToolRequestHeaders(authToken, turnContext, options); // Create Connection instance for LangChain agents mcpServers[server.mcpServerName] = { diff --git a/packages/agents-a365-tooling-extensions-openai/src/McpToolRegistrationService.ts b/packages/agents-a365-tooling-extensions-openai/src/McpToolRegistrationService.ts index b63ab341..890f44d8 100644 --- a/packages/agents-a365-tooling-extensions-openai/src/McpToolRegistrationService.ts +++ b/packages/agents-a365-tooling-extensions-openai/src/McpToolRegistrationService.ts @@ -67,9 +67,8 @@ export class McpToolRegistrationService { const mcpServers: MCPServerStreamableHttp[] = []; for (const server of servers) { - // Merge base headers (channel, user-agent) with per-audience Authorization from server.headers - const baseHeaders: Record = Utility.GetToolRequestHeaders(authToken, turnContext, options); - const headers: Record = { ...baseHeaders, ...server.headers }; + // Compose headers if values are available + const headers: Record = Utility.GetToolRequestHeaders(authToken, turnContext, options); // Create MCPServerStreamableHttp instance for OpenAI agents const mcpServer = new MCPServerStreamableHttp({ diff --git a/packages/agents-a365-tooling/src/McpToolServerConfigurationService.ts b/packages/agents-a365-tooling/src/McpToolServerConfigurationService.ts index 7fb474f7..40578895 100644 --- a/packages/agents-a365-tooling/src/McpToolServerConfigurationService.ts +++ b/packages/agents-a365-tooling/src/McpToolServerConfigurationService.ts @@ -9,17 +9,11 @@ import { OperationResult, OperationError, IConfigurationProvider, AgenticAuthent import { MCPServerConfig, MCPServerManifestEntry, McpClientTool, ToolOptions } from './contracts'; import { ChatHistoryMessage, ChatMessageRequest } from './models/index'; import { Utility } from './Utility'; -import { ToolingConfiguration, defaultToolingConfigurationProvider, resolveTokenScopeForServer } from './configuration'; +import { ToolingConfiguration, defaultToolingConfigurationProvider } from './configuration'; import { StreamableHTTPClientTransport } from '@modelcontextprotocol/sdk/client/streamableHttp.js'; import { Client } from '@modelcontextprotocol/sdk/client/index.js'; -/** - * Resolves a Bearer token for one MCP server given its computed scope. - * Returns null when no token is available (dev no-op); prod implementations throw instead. - */ -type TokenAcquirer = (server: MCPServerConfig, scope: string) => Promise; - /** * Service responsible for discovering and normalizing MCP (Model Context Protocol) * tool servers and producing configuration objects consumable by the Claude SDK. @@ -89,20 +83,9 @@ export class McpToolServerConfigurationService { const authToken = authTokenOrAuthorization; const toolOptions = optionsOrAuthHandlerName as ToolOptions | undefined; - const servers = await (this.isDevScenario() + return await (this.isDevScenario() ? this.getMCPServerConfigsFromManifest() : this.getMCPServerConfigsFromToolingGateway(agenticAppId, authToken, undefined, toolOptions)); - - // Apply per-audience tokens on the legacy path too, using the same structural path as the - // new overload so V2 servers are never silently missing an Authorization header. - // Dev: reads from BEARER_TOKEN_ / BEARER_TOKEN env vars, supports V1 and V2. - // Prod: uses the shared authToken for V1 servers; throws for V2 servers (OBO requires - // Authorization and authHandlerName — use the TurnContext-based overload instead). - const acquire = this.isDevScenario() - ? this.createDevTokenAcquirer() - : this.createLegacyProdTokenAcquirer(authToken); - - return await this.attachPerAudienceTokens(servers, acquire); } else { // NEW PATH: listToolServers(turnContext, authorization, authHandlerName, authToken?, options?) const turnContext = agenticAppIdOrTurnContext; @@ -135,121 +118,12 @@ export class McpToolServerConfigurationService { // Resolve agenticAppId from TurnContext const agenticAppId = RuntimeUtility.ResolveAgentIdentity(turnContext, authToken); - // Discover servers: manifest in dev, gateway in prod - const servers = await (this.isDevScenario() + return await (this.isDevScenario() ? this.getMCPServerConfigsFromManifest() : this.getMCPServerConfigsFromToolingGateway(agenticAppId, authToken, turnContext, toolOptions)); - - // Acquire and attach per-server tokens via the same structural path in both envs. - // Token source differs: env vars in dev, OBO in prod. - const acquire = this.isDevScenario() - ? this.createDevTokenAcquirer() - : this.createOboTokenAcquirer(authorization, authHandlerName, turnContext); - - return await this.attachPerAudienceTokens(servers, acquire); } } - /** - * Acquire one token per unique audience across the provided server list and attach - * the correct `Authorization: Bearer` header to each server's headers. - * V1 servers (no `audience` field, or ATG AppId) all share the same token (one exchange). - * V2 servers each get a token scoped to their own audience GUID. - * Token acquisition is delegated to `acquire`, enabling different strategies in dev - * (env vars via createDevTokenAcquirer) and prod (OBO via createOboTokenAcquirer) - * while keeping scope resolution, deduplication, and header attachment identical. - */ - private async attachPerAudienceTokens( - servers: MCPServerConfig[], - acquire: TokenAcquirer - ): Promise { - // Fetch once so scope resolution and the legacy-path guard use the same value. - const sharedScope = this.configProvider.getConfiguration().mcpPlatformAuthenticationScope; - const tokenCache = new Map(); // scope → token (null = no token available) - - const result: MCPServerConfig[] = []; - for (const server of servers) { - const scope = resolveTokenScopeForServer(server, sharedScope); - if (!tokenCache.has(scope)) { - tokenCache.set(scope, await acquire(server, scope)); - } - const token = tokenCache.get(scope) as string | null; - result.push(token - ? { ...server, headers: { ...server.headers, Authorization: `Bearer ${token}` } } - : server // no token available — dev no-op; prod acquirer would have thrown already - ); - } - return result; - } - - /** - * Returns a TokenAcquirer that resolves tokens from environment variables (local dev only). - * Resolution order per server: - * 1. BEARER_TOKEN_ — per-server token (effective for V2 unique audiences) - * 2. BEARER_TOKEN — shared fallback (V1 servers share one token) - * Returns null when neither variable is set; no Authorization header is attached. - * Emits a warning when a V2 server (distinct audience) falls back to the shared BEARER_TOKEN, - * because that token is scoped to the shared ATG audience and will cause a 401 at the server. - */ - private createDevTokenAcquirer(): TokenAcquirer { - const sharedScope = this.configProvider.getConfiguration().mcpPlatformAuthenticationScope; - return (server, scope) => { - const serverName = server.mcpServerName ?? ''; - const config = this.configProvider.getConfiguration(); - const token = config.getBearerTokenForServer(serverName); - if (token && !config.hasPerServerBearerToken(serverName) && scope !== sharedScope) { - this.logger.warn( - `Dev: MCP server '${serverName}' requires scope '${scope}' but only BEARER_TOKEN is set. ` + - `The shared token is scoped to a different audience and will likely cause a 401. ` + - `Set BEARER_TOKEN_${serverName.toUpperCase()} to a token acquired for the correct audience.` - ); - } - return Promise.resolve(token ?? null); - }; - } - - /** - * Returns a TokenAcquirer for the deprecated legacy (agenticAppId, authToken) overload in prod. - * V1 servers (ATG shared scope) receive the caller-supplied authToken directly. - * V2 servers (per-audience scope) throw immediately — OBO exchange requires Authorization and - * authHandlerName which the legacy signature does not provide; callers must migrate to the - * TurnContext-based overload. - */ - private createLegacyProdTokenAcquirer(authToken: string): TokenAcquirer { - const sharedScope = this.configProvider.getConfiguration().mcpPlatformAuthenticationScope; - return (server, scope) => { - if (scope !== sharedScope) { - throw new Error( - `MCP server '${server.mcpServerName}' requires a per-audience token (scope: '${scope}'). ` + - `Per-audience token exchange is not supported by the deprecated listToolServers(agenticAppId, authToken) overload. ` + - `Migrate to listToolServers(turnContext, authorization, authHandlerName) instead.` - ); - } - return Promise.resolve(authToken); - }; - } - - /** - * Returns a TokenAcquirer that performs OBO token exchange via AgenticAuthenticationService. - * Throws if the exchange returns null so callers receive an explicit error rather than a - * silently missing Authorization header. - */ - private createOboTokenAcquirer( - authorization: Authorization, - authHandlerName: string, - turnContext: TurnContext - ): TokenAcquirer { - return async (server, scope) => { - const token = await AgenticAuthenticationService.GetAgenticUserToken( - authorization, authHandlerName, turnContext, [scope] - ); - if (!token) { - throw new Error(`Failed to obtain token for MCP server '${server.mcpServerName}' (scope: ${scope})`); - } - return token; - }; - } - /** * Connect to the MCP server and return tools with names prefixed by the server name. * Throws if the server URL is missing or the client fails to list tools. @@ -411,15 +285,7 @@ export class McpToolServerConfigurationService { } ); - const rawServers: MCPServerConfig[] = response.data || []; - return rawServers.map(s => ({ - mcpServerName: s.mcpServerName, - url: s.url, - headers: s.headers, - audience: s.audience, - scope: s.scope, - publisher: s.publisher, - })); + return (response.data) || []; } catch (err: unknown) { const error = err as Error & { code?: string }; throw new Error(`Failed to read MCP servers from endpoint: ${error.code || 'UNKNOWN'} ${error.message || 'Unknown error'}`); @@ -477,10 +343,7 @@ export class McpToolServerConfigurationService { return { mcpServerName: serverName, url: s.url || this.buildMcpServerUrl(serverName), - headers: s.headers, - audience: s.audience, - scope: s.scope, - publisher: s.publisher, + headers: s.headers }; }); } catch (err: unknown) { @@ -510,7 +373,7 @@ export class McpToolServerConfigurationService { * Construct the tooling gateway URL for a given agent identity. */ private getToolingGatewayUrl(agenticAppId: string): string { - return `${this.getMcpPlatformBaseUrl()}/agents/v2/${agenticAppId}/mcpServers`; + return `${this.getMcpPlatformBaseUrl()}/agents/${agenticAppId}/mcpServers`; } /** diff --git a/packages/agents-a365-tooling/src/Utility.ts b/packages/agents-a365-tooling/src/Utility.ts index eefd6b34..32b0084b 100644 --- a/packages/agents-a365-tooling/src/Utility.ts +++ b/packages/agents-a365-tooling/src/Utility.ts @@ -153,7 +153,7 @@ export class Utility { * * Example: * Utility.GetToolingGatewayForDigitalWorker(agenticAppId) - * // => "https://agent365.svc.cloud.microsoft/agents/v2/{agenticAppId}/mcpServers" + * // => "https://agent365.svc.cloud.microsoft/agents/{agenticAppId}/mcpServers" * * @param agenticAppId - The unique identifier for the agent identity. * @param configProvider - Optional configuration provider. Defaults to defaultToolingConfigurationProvider. @@ -164,7 +164,7 @@ export class Utility { agenticAppId: string, configProvider?: IConfigurationProvider ): string { - return `${this.getMcpPlatformBaseUrl(configProvider)}/agents/v2/${agenticAppId}/mcpServers`; + return `${this.getMcpPlatformBaseUrl(configProvider)}/agents/${agenticAppId}/mcpServers`; } /** diff --git a/packages/agents-a365-tooling/src/configuration/ToolingConfiguration.ts b/packages/agents-a365-tooling/src/configuration/ToolingConfiguration.ts index 586f3e3e..349791e0 100644 --- a/packages/agents-a365-tooling/src/configuration/ToolingConfiguration.ts +++ b/packages/agents-a365-tooling/src/configuration/ToolingConfiguration.ts @@ -3,49 +3,11 @@ import { RuntimeConfiguration } from '@microsoft/agents-a365-runtime'; import { ToolingConfigurationOptions } from './ToolingConfigurationOptions'; -import { MCPServerConfig } from '../contracts'; // Constants for tooling-specific settings const MCP_PLATFORM_PROD_BASE_URL = 'https://agent365.svc.cloud.microsoft'; const PROD_MCP_PLATFORM_AUTHENTICATION_SCOPE = 'ea9ffc3e-8a23-4a7d-836d-234d7c7565c1/.default'; -/** - * Resolve the OAuth scope to request for a given MCP server. - * - * V2 servers carry their own audience in the `audience` field and get a per-audience token. - * V1 servers (no `audience`, or audience matching the shared scope's own audience in plain - * or api:// form) fall back to `sharedScope` — the configured mcpPlatformAuthenticationScope. - * - * @param server The MCP server config returned by the gateway or manifest. - * @param sharedScope The configured shared scope (mcpPlatformAuthenticationScope). - * Defaults to the prod ATG scope so that external callers without a custom config - * continue to work without passing the argument. - */ -export function resolveTokenScopeForServer( - server: MCPServerConfig, - sharedScope: string = PROD_MCP_PLATFORM_AUTHENTICATION_SCOPE -): string { - if (server.audience) { - // Extract the audience portion of sharedScope (everything before the last '/'). - // e.g. 'ea9ffc3e-.../.default' → 'ea9ffc3e-...' - // 'api://ea9ffc3e-.../.default' → 'api://ea9ffc3e-...' - const sharedAudience = sharedScope.slice(0, sharedScope.lastIndexOf('/')); - // Build the alternate form so we match both 'guid' and 'api://guid'. - const sharedAudienceAlt = sharedAudience.startsWith('api://') - ? sharedAudience.slice(6) // 'api://guid' → 'guid' - : `api://${sharedAudience}`; // 'guid' → 'api://guid' - - if (server.audience !== sharedAudience && server.audience !== sharedAudienceAlt) { - // V2 server: use its own audience with explicit scope or /.default fallback. - return server.scope - ? `${server.audience}/${server.scope}` - : `${server.audience}/.default`; - } - } - // V1 server: no audience, or audience matches the shared ATG audience. - return sharedScope; -} - /** * Normalize URL by trimming whitespace and removing trailing slashes. * Prevents double-slash issues in URL construction (e.g., "https://example.com//api"). @@ -106,25 +68,4 @@ export class ToolingConfiguration extends RuntimeConfiguration { return PROD_MCP_PLATFORM_AUTHENTICATION_SCOPE; } - - /** - * Returns the dev-mode bearer token for an MCP server by name. - * Checks BEARER_TOKEN_ first, then falls back to BEARER_TOKEN. - * Returns undefined when the variable is not set (no Authorization header will be attached). - */ - getBearerTokenForServer(mcpServerName: string): string | undefined { - const key = mcpServerName.toUpperCase(); - return process.env[`BEARER_TOKEN_${key}`] ?? process.env['BEARER_TOKEN']; - } - - /** - * Returns true when a per-server bearer token env var (BEARER_TOKEN_) - * is explicitly set for the given server, false when only the shared BEARER_TOKEN fallback - * would be used. Used to detect V2 servers that are silently falling back to a - * wrong-audience token in dev mode. - */ - hasPerServerBearerToken(mcpServerName: string): boolean { - const key = mcpServerName.toUpperCase(); - return !!process.env[`BEARER_TOKEN_${key}`]; - } } diff --git a/packages/agents-a365-tooling/src/contracts.ts b/packages/agents-a365-tooling/src/contracts.ts index ef3b341a..8a737527 100644 --- a/packages/agents-a365-tooling/src/contracts.ts +++ b/packages/agents-a365-tooling/src/contracts.ts @@ -5,17 +5,11 @@ export interface MCPServerConfig { mcpServerName: string; url: string; headers?: Record; - audience?: string; // per-server AppId (V2) or ATG AppId (V1) — undefined = treat as V1 - scope?: string; // e.g. "Tools.ListInvoke.All" (V2) or "McpServers.Mail.All" (V1) - publisher?: string; } export type MCPServerManifestEntry = { url?: string; headers?: Record; - audience?: string; - scope?: string; - publisher?: string; } & ( | { mcpServerName: string; mcpServerUniqueName?: string } | { mcpServerUniqueName: string; mcpServerName?: string } @@ -36,4 +30,4 @@ export interface InputSchema { export interface ToolOptions { orchestratorName?: string; -} +} \ No newline at end of file diff --git a/tests/jest.config.cjs b/tests/jest.config.cjs index 3627b5ff..8511a6d0 100644 --- a/tests/jest.config.cjs +++ b/tests/jest.config.cjs @@ -69,6 +69,7 @@ module.exports = { moduleNameMapper: { '^@microsoft/agents-a365-runtime$': '/packages/agents-a365-runtime/src', '^@microsoft/agents-a365-observability$': '/packages/agents-a365-observability/src', + '^@microsoft/agents-a365-observability-hosting$': '/packages/agents-a365-observability-hosting/src', '^@microsoft/agents-a365-observability-extensions-langchain$': '/packages/agents-a365-observability-extensions-langchain/src', '^@microsoft/agents-a365-observability-extensions-openai$': '/packages/agents-a365-observability-extensions-openai/src', '^@microsoft/agents-a365-observability-tokencache$': '/packages/agents-a365-observability-tokencache/src', diff --git a/tests/observability/core/agent365-exporter.test.ts b/tests/observability/core/agent365-exporter.test.ts index dcf958b2..60e66092 100644 --- a/tests/observability/core/agent365-exporter.test.ts +++ b/tests/observability/core/agent365-exporter.test.ts @@ -135,7 +135,7 @@ describe('Agent365Exporter', () => { expect(fetchCalls.length).toBe(1); const urlArg = fetchCalls[0][0]; const headersArg = fetchCalls[0][1].headers; - expect(urlArg).toBe(`${expectedUrl}/observability/tenants/${tenantId}/otlp/agents/${agentId}/traces?api-version=1`); + expect(urlArg).toBe(`${expectedUrl}/observability/tenants/${tenantId}/agents/${agentId}/traces?api-version=1`); expect(headersArg['x-ms-tenant-id']).toBe(tenantId); expect(headersArg['authorization']).toBe(`Bearer ${token}`); }); @@ -190,7 +190,7 @@ describe('Agent365Exporter', () => { const urlArg = fetchCalls[0][0] as string; const headersArg = fetchCalls[0][1].headers as Record; - expect(urlArg).toBe(`${expectedBaseUrl}/observability/tenants/${tenantId}/otlp/agents/${agentId}/traces?api-version=1`); + expect(urlArg).toBe(`${expectedBaseUrl}/observability/tenants/${tenantId}/agents/${agentId}/traces?api-version=1`); expect(headersArg['x-ms-tenant-id']).toBe(tenantId); expect(headersArg['authorization']).toBe(`Bearer ${token}`); }); @@ -215,7 +215,7 @@ describe('Agent365Exporter', () => { expect(fetchCalls.length).toBe(1); const urlArg = fetchCalls[0][0]; const headersArg = fetchCalls[0][1].headers; - expect(urlArg).toBe(`https://agent365.svc.cloud.microsoft/observability/tenants/${tenantId}/otlp/agents/${agentId}/traces?api-version=1`); + expect(urlArg).toBe(`https://agent365.svc.cloud.microsoft/observability/tenants/${tenantId}/agents/${agentId}/traces?api-version=1`); expect(headersArg['x-ms-tenant-id']).toBe(tenantId); expect(headersArg['authorization']).toBe(`Bearer ${token}`); }); @@ -250,7 +250,7 @@ describe('Agent365Exporter', () => { expect(fetchCalls.length).toBe(1); const urlArg = fetchCalls[0][0] as string; - expect(urlArg).toBe(`https://agent365.svc.cloud.microsoft/observabilityService/tenants/${tenantId}/otlp/agents/${agentId}/traces?api-version=1`); + expect(urlArg).toMatch(`/observabilityService/tenants/${tenantId}/agents/${agentId}/traces?api-version=1`); const headersArg = fetchCalls[0][1].headers as Record; expect(headersArg['authorization']).toBe(`Bearer ${token}`); expect(headersArg['x-ms-tenant-id']).toBe(tenantId); @@ -279,7 +279,8 @@ describe('Agent365Exporter', () => { const fetchCalls = getFetchCalls(); expect(fetchCalls.length).toBe(1); const urlArg = fetchCalls[0][0] as string; - expect(urlArg).toBe(`https://custom.domain/observabilityService/tenants/${tenantId}/otlp/agents/${agentId}/traces?api-version=1`); + expect(urlArg).toMatch(`/observabilityService/tenants/${tenantId}/agents/${agentId}/traces?api-version=1`); + expect(urlArg).toContain('https://custom.domain'); const headersArg = fetchCalls[0][1].headers as Record; expect(headersArg['authorization']).toBe(`Bearer ${token}`); expect(headersArg['x-ms-tenant-id']).toBe(tenantId); diff --git a/tests/observability/extension/helpers/message-schema-validator.ts b/tests/observability/extension/helpers/message-schema-validator.ts deleted file mode 100644 index 4e3f5a2e..00000000 --- a/tests/observability/extension/helpers/message-schema-validator.ts +++ /dev/null @@ -1,68 +0,0 @@ -// Copyright (c) Microsoft Corporation. -// Licensed under the MIT License. - -import { expect } from '@jest/globals'; - -function validateMessageEnvelope(value: unknown): Record { - // Attributes are stored as JSON strings; parse and validate the common envelope. - expect(typeof value).toBe('string'); - const parsed = JSON.parse(value as string); - expect(parsed).toHaveProperty('version', '0.1.0'); - expect(parsed.messages).toEqual(expect.arrayContaining([expect.anything()])); - return parsed; -} - -function validateMessagePart(part: Record): void { - // Validate required fields and type-specific shape for each message part. - expect(typeof part.type).toBe('string'); - expect((part.type as string).length).toBeGreaterThan(0); - - const type = part.type as string; - if (type === 'text' || type === 'reasoning') { - expect(typeof part.content).toBe('string'); - } else if (type === 'tool_call') { - expect(typeof part.name).toBe('string'); - if (part.id !== undefined) expect(typeof part.id).toBe('string'); - } else if (type === 'tool_call_response') { - if (part.id !== undefined) expect(typeof part.id).toBe('string'); - } else if (type === 'blob' || type === 'file' || type === 'uri') { - expect(part).toHaveProperty('modality'); - } -} - -export function expectValidInputMessages(value: unknown): void { - // Input messages must have a role and at least one valid part. - const parsed = validateMessageEnvelope(value); - for (const msg of parsed.messages as Array>) { - expect(typeof msg.role).toBe('string'); - const parts = msg.parts as Array>; - expect(parts.length).toBeGreaterThan(0); - parts.forEach(validateMessagePart); - } -} - -export function expectValidOutputMessages(value: unknown): void { - // Output messages follow the same structure, with optional finish_reason. - const parsed = validateMessageEnvelope(value); - for (const msg of parsed.messages as Array>) { - expect(typeof msg.role).toBe('string'); - const parts = msg.parts as Array>; - expect(parts.length).toBeGreaterThan(0); - parts.forEach(validateMessagePart); - if (msg.finish_reason !== undefined) { - expect(typeof msg.finish_reason).toBe('string'); - } - } -} - -export function getSpanAttribute(mockSpan: { setAttribute: jest.Mock }, key: string): unknown { - // Helper to read a specific attribute from a mocked span. - const match = mockSpan.setAttribute.mock.calls.find(([k]: [string, unknown]) => k === key); - return match ? match[1] : undefined; -} - -export function getAttrFromArray(attrs: Array<[string, unknown]>, key: string): unknown { - // Helper to read a key from an array of [key, value] tuples. - const entry = attrs.find(([k]) => k === key); - return entry ? entry[1] : undefined; -} diff --git a/tests/observability/extension/hosting/BaggageBuilderUtils.test.ts b/tests/observability/extension/hosting/BaggageBuilderUtils.test.ts index 023d2d03..23983744 100644 --- a/tests/observability/extension/hosting/BaggageBuilderUtils.test.ts +++ b/tests/observability/extension/hosting/BaggageBuilderUtils.test.ts @@ -45,7 +45,7 @@ describe('BaggageBuilderUtils', () => { expect(result).toBe(builder); // Validate every expected OpenTelemetry baggage key and value const asObj = Object.fromEntries(capturedPairs); - expect(asObj[OpenTelemetryConstants.USER_ID_KEY]).toBeUndefined(); + expect(asObj[OpenTelemetryConstants.USER_ID_KEY]).toBe('agentic-user-1'); expect(asObj[OpenTelemetryConstants.USER_NAME_KEY]).toBe('User One'); expect(asObj[OpenTelemetryConstants.USER_EMAIL_KEY]).toBe('agentic-user-1'); expect(asObj[OpenTelemetryConstants.GEN_AI_AGENT_ID_KEY]).toBe('agent-app-1'); diff --git a/tests/observability/extension/hosting/TurnContextUtils.test.ts b/tests/observability/extension/hosting/TurnContextUtils.test.ts index 675c01ce..61507f46 100644 --- a/tests/observability/extension/hosting/TurnContextUtils.test.ts +++ b/tests/observability/extension/hosting/TurnContextUtils.test.ts @@ -32,6 +32,39 @@ describe('TurnContextUtils', () => { expect(pairs.length).toBeGreaterThan(0); }); + it('should use aadObjectId for userId when present (precedence test)', () => { + const ctx = { + activity: { + from: { id: 'from-id-1', name: 'User', aadObjectId: 'aad-oid-1', agenticUserId: 'agentic-uid-1' }, + }, + } as any; + const pairs = getCallerBaggagePairs(ctx); + const obj = Object.fromEntries(pairs); + expect(obj[OpenTelemetryConstants.USER_ID_KEY]).toBe('aad-oid-1'); + }); + + it('should fall back to agenticUserId when aadObjectId is absent (A2A scenario)', () => { + const ctx = { + activity: { + from: { id: 'from-id-1', name: 'Agent Caller', agenticUserId: 'agentic-uid-1' }, + }, + } as any; + const pairs = getCallerBaggagePairs(ctx); + const obj = Object.fromEntries(pairs); + expect(obj[OpenTelemetryConstants.USER_ID_KEY]).toBe('agentic-uid-1'); + }); + + it('should fall back to from.id when aadObjectId and agenticUserId are absent (non-Teams channel)', () => { + const ctx = { + activity: { + from: { id: 'webchat-user-42', name: 'Web User' }, + }, + } as any; + const pairs = getCallerBaggagePairs(ctx); + const obj = Object.fromEntries(pairs); + expect(obj[OpenTelemetryConstants.USER_ID_KEY]).toBe('webchat-user-42'); + }); + it('should get target agent baggage pairs', () => { const pairs = getTargetAgentBaggagePairs(mockTurnContext); expect(Array.isArray(pairs)).toBe(true); diff --git a/tests/observability/extension/langchain/LangChainMessageContract.test.ts b/tests/observability/extension/langchain/LangChainMessageContract.test.ts deleted file mode 100644 index 01847439..00000000 --- a/tests/observability/extension/langchain/LangChainMessageContract.test.ts +++ /dev/null @@ -1,119 +0,0 @@ -// Copyright (c) Microsoft Corporation. -// Licensed under the MIT License. - -import { HumanMessage, AIMessage, SystemMessage, ToolMessage } from "@langchain/core/messages"; -import { Run } from "@langchain/core/tracers/base"; -import { Span } from "@opentelemetry/api"; -import { OpenTelemetryConstants } from "@microsoft/agents-a365-observability"; -import * as Utils from "../../../../packages/agents-a365-observability-extensions-langchain/src/Utils"; -import { expectValidInputMessages, expectValidOutputMessages, getSpanAttribute } from "../helpers/message-schema-validator"; - -describe("LangChain Message Contract Tests", () => { - let mockSpan: { setAttribute: jest.Mock; setStatus: jest.Mock; end: jest.Mock; recordException: jest.Mock }; - - beforeEach(() => { - mockSpan = { - setAttribute: jest.fn(), - setStatus: jest.fn(), - end: jest.fn(), - recordException: jest.fn(), - }; - }); - - function getInputAttr(): string { - const value = getSpanAttribute(mockSpan, OpenTelemetryConstants.GEN_AI_INPUT_MESSAGES_KEY); - expect(value).toBeDefined(); - return value as string; - } - - function getOutputAttr(): string { - const value = getSpanAttribute(mockSpan, OpenTelemetryConstants.GEN_AI_OUTPUT_MESSAGES_KEY); - expect(value).toBeDefined(); - return value as string; - } - - function setInput(messages: unknown[]): void { - const run: Partial = { run_type: "llm", inputs: { messages: [messages] } }; - Utils.setInputMessagesAttribute(run as Run, mockSpan as unknown as Span); - } - - describe("Input messages from real LangChain types", () => { - it("should map a full conversation with all message types", () => { - setInput([ - new SystemMessage("You are a weather assistant."), - new HumanMessage("What's the weather in Seattle?"), - new AIMessage({ - content: "Let me check.", - tool_calls: [{ name: "get_weather", args: { city: "Seattle" }, id: "call_1" }], - }), - new ToolMessage({ content: "Rainy, 45°F", tool_call_id: "call_1" }), - ]); - - const value = getInputAttr(); - expectValidInputMessages(value); - - const parsed = JSON.parse(value); - const roles = parsed.messages.map((m: Record) => m.role); - expect(roles).toEqual(["system", "user", "assistant", "tool"]); - - expect(parsed.messages[0].parts[0].content).toBe("You are a weather assistant."); - expect(parsed.messages[1].parts[0].content).toBe("What's the weather in Seattle?"); - - const aiParts = parsed.messages[2].parts; - expect(aiParts.find((p: Record) => p.type === "text").content).toBe("Let me check."); - const toolCallPart = aiParts.find((p: Record) => p.type === "tool_call"); - expect(toolCallPart.name).toBe("get_weather"); - expect(toolCallPart.id).toBe("call_1"); - - expect(parsed.messages[3].parts[0].content).toBe("Rainy, 45°F"); - }); - }); - - describe("Output messages from real LangChain types", () => { - it("should map AIMessage text output via generations", () => { - const run: Partial = { - run_type: "llm", - outputs: { generations: [[{ text: "Hello!", message: new AIMessage("Hello!") }]] }, - }; - Utils.setOutputMessagesAttribute(run as Run, mockSpan as unknown as Span); - - const value = getOutputAttr(); - expectValidOutputMessages(value); - - const parsed = JSON.parse(value); - expect(parsed.messages[0].role).toBe("assistant"); - expect(parsed.messages[0].parts[0].content).toBe("Hello!"); - }); - - it("should map AIMessage with tool_calls in output via generations", () => { - const aiMsg = new AIMessage({ - content: "", - tool_calls: [{ name: "search", args: { query: "weather" }, id: "call_456" }], - }); - const run: Partial = { - run_type: "llm", - outputs: { generations: [[{ text: "", message: aiMsg }]] }, - }; - Utils.setOutputMessagesAttribute(run as Run, mockSpan as unknown as Span); - - const value = getOutputAttr(); - expectValidOutputMessages(value); - - const toolPart = JSON.parse(value).messages[0].parts.find((p: Record) => p.type === "tool_call"); - expect(toolPart.name).toBe("search"); - expect(toolPart.id).toBe("call_456"); - }); - - it("should map direct output messages array (LangGraph path)", () => { - const run: Partial = { - run_type: "chain", - serialized: { id: ["langgraph", "graph", "CompiledStateGraph"] }, - outputs: { messages: [new AIMessage("Task complete.")] }, - }; - Utils.setOutputMessagesAttribute(run as Run, mockSpan as unknown as Span); - - const value = getOutputAttr(); - expectValidOutputMessages(value); - }); - }); -}); diff --git a/tests/observability/extension/langchain/LangChainObservabilityAttributes.test.ts b/tests/observability/extension/langchain/LangChainObservabilityAttributes.test.ts index 5b0e044c..db1b872b 100644 --- a/tests/observability/extension/langchain/LangChainObservabilityAttributes.test.ts +++ b/tests/observability/extension/langchain/LangChainObservabilityAttributes.test.ts @@ -5,7 +5,6 @@ import { Run } from "@langchain/core/tracers/base"; import { Span } from "@opentelemetry/api"; import { OpenTelemetryConstants } from "@microsoft/agents-a365-observability"; import * as Utils from "../../../../packages/agents-a365-observability-extensions-langchain/src/Utils"; -import { expectValidInputMessages, expectValidOutputMessages, getSpanAttribute } from "../helpers/message-schema-validator"; describe("LangChain Observability - InvokeAgentScope Attributes", () => { let mockSpan: Partial; @@ -43,12 +42,9 @@ describe("LangChain Observability - InvokeAgentScope Attributes", () => { Utils.setInputMessagesAttribute(run as Run, mockSpan as Span); - const inputValue = getSpanAttribute(mockSpan as any, OpenTelemetryConstants.GEN_AI_INPUT_MESSAGES_KEY); - expectValidInputMessages(inputValue); - expect(mockSpan.setAttribute).toHaveBeenCalledWith( OpenTelemetryConstants.GEN_AI_INPUT_MESSAGES_KEY, - JSON.stringify({"version":"0.1.0","messages":[{"role":"user","parts":[{"type":"text","content":"hi"}]}]}) + JSON.stringify(["hi"]) ); }); @@ -68,12 +64,9 @@ describe("LangChain Observability - InvokeAgentScope Attributes", () => { Utils.setInputMessagesAttribute(run as Run, mockSpan as Span); - const inputValue = getSpanAttribute(mockSpan as any, OpenTelemetryConstants.GEN_AI_INPUT_MESSAGES_KEY); - expectValidInputMessages(inputValue); - expect(mockSpan.setAttribute).toHaveBeenCalledWith( OpenTelemetryConstants.GEN_AI_INPUT_MESSAGES_KEY, - JSON.stringify({"version":"0.1.0","messages":[{"role":"user","parts":[{"type":"text","content":"hello agent"}]}]}) + JSON.stringify(["hello agent"]) ); }); @@ -93,12 +86,9 @@ describe("LangChain Observability - InvokeAgentScope Attributes", () => { Utils.setOutputMessagesAttribute(run as Run, mockSpan as Span); - const outputValue = getSpanAttribute(mockSpan as any, OpenTelemetryConstants.GEN_AI_OUTPUT_MESSAGES_KEY); - expectValidOutputMessages(outputValue); - expect(mockSpan.setAttribute).toHaveBeenCalledWith( OpenTelemetryConstants.GEN_AI_OUTPUT_MESSAGES_KEY, - JSON.stringify({"version":"0.1.0","messages":[{"role":"assistant","parts":[{"type":"text","content":"Hello! How can I assist you today?"}]}]}) + JSON.stringify(["Hello! How can I assist you today?"]) ); }); @@ -187,45 +177,9 @@ describe("LangChain Observability - ExecuteToolScope Attributes", () => { ); expect(mockSpan.setAttribute).toHaveBeenCalledWith( OpenTelemetryConstants.GEN_AI_TOOL_CALL_RESULT_KEY, - '{"result":"The weather in Seattle is currently rainy with a temperature of 39°C."}' - ); - // Tool args: string input is already valid JSON, passed through by safeSerializeToJson - expect(mockSpan.setAttribute).toHaveBeenCalledWith( - OpenTelemetryConstants.GEN_AI_TOOL_ARGS_KEY, - '{"city": "Seattle"}' - ); - }); - - it("should extract tool result from v1 plain string output", () => { - const run: Partial = { - run_type: "tool", - name: "get_weather", - serialized: { name: "get_weather" }, - inputs: { - input: '{"city": "Seattle"}', - tool_call_id: "call_v1_abc123", - }, - outputs: { - output: "Sunny, 25°C in Seattle.", - }, - }; - - Utils.setToolAttributes(run as Run, mockSpan as Span); - - expect(mockSpan.setAttribute).toHaveBeenCalledWith( - OpenTelemetryConstants.GEN_AI_TOOL_CALL_RESULT_KEY, - '{"result":"Sunny, 25°C in Seattle."}' - ); - expect(mockSpan.setAttribute).toHaveBeenCalledWith( - OpenTelemetryConstants.GEN_AI_TOOL_ARGS_KEY, - '{"city": "Seattle"}' - ); - expect(mockSpan.setAttribute).toHaveBeenCalledWith( - OpenTelemetryConstants.GEN_AI_TOOL_CALL_ID_KEY, - "call_v1_abc123" + JSON.stringify("The weather in Seattle is currently rainy with a temperature of 39°C.") ); }); - }); }); @@ -420,150 +374,5 @@ describe("LangChain Observability - InferenceScope Attributes", () => { "You are a code generator" ); }); - - it("should extract system instructions from v1 constructor format", () => { - const run: Partial = { - inputs: { - messages: [[ - { - lc: 1, - type: "constructor", - id: ["langchain_core", "messages", "SystemMessage"], - kwargs: { content: "v1 system prompt" }, - }, - { - lc: 1, - type: "constructor", - id: ["langchain_core", "messages", "HumanMessage"], - kwargs: { content: "user input" }, - }, - ]], - }, - }; - - Utils.setSystemInstructionsAttribute(run as Run, mockSpan as Span); - - expect(mockSpan.setAttribute).toHaveBeenCalledWith( - OpenTelemetryConstants.GEN_AI_SYSTEM_INSTRUCTIONS_KEY, - "v1 system prompt" - ); - }); - - it("should extract tokens from tokenUsage shape (promptTokens/completionTokens)", () => { - const run: Partial = { - outputs: { - generations: [[{ - message: { - kwargs: { - response_metadata: { - tokenUsage: { - promptTokens: 100, - completionTokens: 50, - totalTokens: 150, - }, - }, - }, - }, - }]], - }, - }; - - Utils.setTokenAttributes(run as Run, mockSpan as Span); - - expect(mockSpan.setAttribute).toHaveBeenCalledWith( - OpenTelemetryConstants.GEN_AI_USAGE_INPUT_TOKENS_KEY, - 100 - ); - expect(mockSpan.setAttribute).toHaveBeenCalledWith( - OpenTelemetryConstants.GEN_AI_USAGE_OUTPUT_TOKENS_KEY, - 50 - ); - }); - }); -}); - -describe("LangChain Observability - v1 Format Coverage", () => { - let mockSpan: Partial; - - beforeEach(() => { - mockSpan = { setAttribute: jest.fn() }; - }); - - it("should extract input content from v1 constructor format", () => { - const run: Partial = { - run_type: "llm", - inputs: { - messages: [[ - { - lc: 1, - type: "constructor", - id: ["langchain_core", "messages", "HumanMessage"], - kwargs: { content: "v1 format message" }, - }, - ]], - }, - }; - - Utils.setInputMessagesAttribute(run as Run, mockSpan as Span); - - const value = getSpanAttribute(mockSpan as any, OpenTelemetryConstants.GEN_AI_INPUT_MESSAGES_KEY); - expectValidInputMessages(value); - const parsed = JSON.parse(value as string); - expect(parsed.messages[0].parts[0].content).toBe("v1 format message"); - }); - - it("should extract output content from v1 AIMessage constructor format", () => { - const run: Partial = { - run_type: "llm", - outputs: { - generations: [[{ - message: { - lc: 1, - type: "constructor", - id: ["langchain_core", "messages", "AIMessage"], - kwargs: { content: "v1 AI response", tool_calls: [] }, - }, - }]], - }, - }; - - Utils.setOutputMessagesAttribute(run as Run, mockSpan as Span); - - const value = getSpanAttribute(mockSpan as any, OpenTelemetryConstants.GEN_AI_OUTPUT_MESSAGES_KEY); - expectValidOutputMessages(value); - const parsed = JSON.parse(value as string); - expect(parsed.messages[0].role).toBe("assistant"); - expect(parsed.messages[0].parts[0].content).toBe("v1 AI response"); - }); - - it("should not set input attribute when inputs.messages is missing", () => { - const run: Partial = { inputs: { other_key: "value" } }; - Utils.setInputMessagesAttribute(run as Run, mockSpan as Span); - expect(mockSpan.setAttribute).not.toHaveBeenCalledWith( - OpenTelemetryConstants.GEN_AI_INPUT_MESSAGES_KEY, - expect.anything() - ); - }); - - it("should filter out non-AI messages from agent scope outputs", () => { - const run: Partial = { - run_type: "chain", - serialized: { id: ["langgraph", "graph", "CompiledStateGraph"] }, - outputs: { - messages: [ - { role: "user", content: "user message in output" }, - { role: "system", content: "system in output" }, - { role: "assistant", content: "ai response" }, - ], - }, - }; - - Utils.setOutputMessagesAttribute(run as Run, mockSpan as Span); - - const value = getSpanAttribute(mockSpan as any, OpenTelemetryConstants.GEN_AI_OUTPUT_MESSAGES_KEY); - const parsed = JSON.parse(value as string); - expect(parsed.messages).toHaveLength(1); - expect(parsed.messages[0].role).toBe("assistant"); - expect(parsed.messages[0].parts[0].content).toBe("ai response"); }); }); diff --git a/tests/observability/extension/openai/OpenAIAgentsTraceProcessor.test.ts b/tests/observability/extension/openai/OpenAIAgentsTraceProcessor.test.ts index 7719c8de..4890d1cd 100644 --- a/tests/observability/extension/openai/OpenAIAgentsTraceProcessor.test.ts +++ b/tests/observability/extension/openai/OpenAIAgentsTraceProcessor.test.ts @@ -13,7 +13,6 @@ import { OpenTelemetryConstants } from '@microsoft/agents-a365-observability'; import { OpenAIAgentsTraceProcessor } from '@microsoft/agents-a365-observability-extensions-openai'; import { ObservabilityManager } from '@microsoft/agents-a365-observability'; import { trace } from '@opentelemetry/api'; -import { expectValidInputMessages, expectValidOutputMessages, getAttrFromArray } from '../helpers/message-schema-validator'; describe('OpenAIAgentsTraceProcessor', () => { let tracer: Tracer; @@ -35,7 +34,7 @@ describe('OpenAIAgentsTraceProcessor', () => { let processor: OpenAIAgentsTraceProcessor; beforeEach(() => { - processor = new OpenAIAgentsTraceProcessor(tracer, {}); + processor = new OpenAIAgentsTraceProcessor(tracer, { isContentRecordingEnabled: true }); }); afterEach(async () => { @@ -480,36 +479,8 @@ describe('OpenAIAgentsTraceProcessor', () => { tracerSpy.mockRestore(); }); - it('should record tool arguments, result, and type on function span', async () => { - const processor = new OpenAIAgentsTraceProcessor(tracer); - const traceData = { traceId: 'trace-func-args', name: 'Agent' } as any; - await processor.onTraceStart(traceData); - - const funcSpan = { - spanId: 'func-args-1', - traceId: 'trace-func-args', - startedAt: new Date().toISOString(), - spanData: { - type: 'function' as const, - name: 'get_weather', - input: { city: 'Seattle' }, - output: 'Sunny, 25°C', - }, - } as any; - - await processor.onSpanStart(funcSpan); - await processor.onSpanEnd(funcSpan); - - const mock = spansByName['get_weather']; - const attrs = mock._attrs as Array<[string, unknown]>; - - expect(attrs.find(([k]) => k === OpenTelemetryConstants.GEN_AI_TOOL_ARGS_KEY)?.[1]).toBe('{"city":"Seattle"}'); - expect(attrs.find(([k]) => k === OpenTelemetryConstants.GEN_AI_TOOL_CALL_RESULT_KEY)?.[1]).toBe('{"result":"Sunny, 25°C"}'); - expect(attrs.find(([k]) => k === OpenTelemetryConstants.GEN_AI_TOOL_TYPE_KEY)?.[1]).toBe('function'); - }); - it('does not record GEN_AI_INPUT_MESSAGES when disabled', async () => { - const processor = new OpenAIAgentsTraceProcessor(tracer, { suppressInvokeAgentInput: true }); + const processor = new OpenAIAgentsTraceProcessor(tracer, { suppressInvokeAgentInput: true, isContentRecordingEnabled: true }); const traceData = { traceId: 'trace-suppress', name: 'Agent' } as any; await processor.onTraceStart(traceData); @@ -533,7 +504,7 @@ describe('OpenAIAgentsTraceProcessor', () => { }); it('records GEN_AI_INPUT_MESSAGES when content recording is enabled', async () => { - const processor = new OpenAIAgentsTraceProcessor(tracer, {}); + const processor = new OpenAIAgentsTraceProcessor(tracer, { isContentRecordingEnabled: true }); const traceData = { traceId: 'trace-allow', name: 'Agent' } as any; await processor.onTraceStart(traceData); @@ -557,7 +528,7 @@ describe('OpenAIAgentsTraceProcessor', () => { }); it('suppresses input on response spans when disabled', async () => { - const processor = new OpenAIAgentsTraceProcessor(tracer, { suppressInvokeAgentInput: true }); + const processor = new OpenAIAgentsTraceProcessor(tracer, { suppressInvokeAgentInput: true, isContentRecordingEnabled: true }); const traceData = { traceId: 'trace-resp', name: 'Agent' } as any; await processor.onTraceStart(traceData); @@ -580,8 +551,8 @@ describe('OpenAIAgentsTraceProcessor', () => { expect(keys).not.toContain(OpenTelemetryConstants.GEN_AI_INPUT_MESSAGES_KEY); }); - it('records structured InputMessages when only assistant messages are present', async () => { - const processor = new OpenAIAgentsTraceProcessor(tracer, {}); + it('records full array JSON when only assistant messages are present', async () => { + const processor = new OpenAIAgentsTraceProcessor(tracer, { isContentRecordingEnabled: true }); const traceData = { traceId: 'trace-assistant-only', name: 'Agent' } as any; await processor.onTraceStart(traceData); @@ -612,17 +583,12 @@ describe('OpenAIAgentsTraceProcessor', () => { const entry = attrs.find(([k]) => k === OpenTelemetryConstants.GEN_AI_INPUT_MESSAGES_KEY); expect(entry).toBeDefined(); - expectValidInputMessages(entry![1]); - const value = entry![1] as string; const parsed = JSON.parse(value); - expect(parsed.version).toBe('0.1.0'); - expect(parsed.messages).toHaveLength(1); - expect(parsed.messages[0].role).toBe('assistant'); - expect(parsed.messages[0].parts[0]).toEqual({ type: 'text', content: 'Assistant reply' }); + expect(parsed).toEqual(inputArray); }); - it('records structured InputMessages for array _input on response spans', async () => { - const processor = new OpenAIAgentsTraceProcessor(tracer, {}); + it('records user text content for array _input on response spans', async () => { + const processor = new OpenAIAgentsTraceProcessor(tracer, { isContentRecordingEnabled: true }); const traceData = { traceId: 'trace-array-input', name: 'Agent' } as any; await processor.onTraceStart(traceData); @@ -651,15 +617,11 @@ describe('OpenAIAgentsTraceProcessor', () => { const value = entry![1] as string; const parsed = JSON.parse(value); - expect(parsed.version).toBe('0.1.0'); - expect(parsed.messages).toHaveLength(2); - expectValidInputMessages(entry![1]); - expect(parsed.messages[0]).toEqual({ role: 'user', parts: [{ type: 'text', content: 'Hello user 1' }] }); - expect(parsed.messages[1]).toEqual({ role: 'user', parts: [{ type: 'text', content: 'Hello user 2' }] }); + expect(parsed).toEqual(['Hello user 1', 'Hello user 2']); }); - it('parses stringified array _input and records all messages in structured format', async () => { - const processor = new OpenAIAgentsTraceProcessor(tracer, {}); + it('parses stringified array _input and records only user text content', async () => { + const processor = new OpenAIAgentsTraceProcessor(tracer, { isContentRecordingEnabled: true }); const traceData = { traceId: 'trace-array-input-string', name: 'Agent' } as any; await processor.onTraceStart(traceData); @@ -691,16 +653,11 @@ describe('OpenAIAgentsTraceProcessor', () => { const value = entry![1] as string; const parsed = JSON.parse(value); - expect(parsed.version).toBe('0.1.0'); - expect(parsed.messages).toHaveLength(3); - expectValidInputMessages(entry![1]); - expect(parsed.messages[0].role).toBe('user'); - expect(parsed.messages[1].role).toBe('user'); - expect(parsed.messages[2].role).toBe('assistant'); + expect(parsed).toEqual(['Hello user 1', 'Hello user 2']); }); - it('records structured InputMessages for array input with non standard schema on response spans', async () => { - const processor = new OpenAIAgentsTraceProcessor(tracer, {}); + it('records [gen_ai.input.messages] attribute for array input with non standard schema on response spans', async () => { + const processor = new OpenAIAgentsTraceProcessor(tracer, { isContentRecordingEnabled: true }); const traceData = { traceId: 'trace-array-input', name: 'Agent' } as any; await processor.onTraceStart(traceData); const inputArray = [ @@ -714,7 +671,7 @@ describe('OpenAIAgentsTraceProcessor', () => { spanData: { type: 'response' as const, name: 'ResponseArray', - _input: inputArray, + _input: inputArray, _response: { model: 'gpt-4', output: 'ok' }, }, } as any; @@ -729,13 +686,11 @@ describe('OpenAIAgentsTraceProcessor', () => { const value = entry![1] as string; const parsed = JSON.parse(value); - expect(parsed.version).toBe('0.1.0'); - expect(parsed.messages).toBeDefined(); - expect(parsed.messages.length).toBeGreaterThan(0); + expect(parsed).toEqual(inputArray); }); - it('records GEN_AI_OUTPUT_MESSAGES in versioned envelope when output is a string', async () => { - const processor = new OpenAIAgentsTraceProcessor(tracer, {}); + it('records GEN_AI_OUTPUT_MESSAGES as plain string when output is a string', async () => { + const processor = new OpenAIAgentsTraceProcessor(tracer, { isContentRecordingEnabled: true }); const traceData = { traceId: 'trace-output-string', name: 'Agent' } as any; await processor.onTraceStart(traceData); @@ -758,13 +713,11 @@ describe('OpenAIAgentsTraceProcessor', () => { const attrs = respMock._attrs as Array<[string, unknown]>; const entry = attrs.find(([k]) => k === OpenTelemetryConstants.GEN_AI_OUTPUT_MESSAGES_KEY); expect(entry).toBeDefined(); - expectValidOutputMessages(entry![1]); - const parsed = JSON.parse(entry![1] as string); - expect(parsed.messages[0].parts[0].content).toBe('final answer'); + expect(entry![1]).toBe('final answer'); }); - it('records structured OutputMessages when output is structured', async () => { - const processor = new OpenAIAgentsTraceProcessor(tracer, {}); + it('records GEN_AI_OUTPUT_MESSAGES as aggregated texts when output is structured', async () => { + const processor = new OpenAIAgentsTraceProcessor(tracer, { isContentRecordingEnabled: true }); const traceData = { traceId: 'trace-output-structured', name: 'Agent' } as any; await processor.onTraceStart(traceData); @@ -800,14 +753,41 @@ describe('OpenAIAgentsTraceProcessor', () => { const value = entry![1] as string; const parsed = JSON.parse(value); - expect(parsed.version).toBe('0.1.0'); - expect(parsed.messages).toHaveLength(1); - expect(parsed.messages[0].role).toBe('assistant'); - expectValidOutputMessages(entry![1]); - expect(parsed.messages[0].parts).toEqual([ - { type: 'text', content: 'Hello user 1' }, - { type: 'text', content: 'Hello user 2' }, - ]); + expect(parsed).toEqual(['Hello user 1', 'Hello user 2']); + }); + + it('suppresses all content attributes when isContentRecordingEnabled is false', async () => { + const processor = new OpenAIAgentsTraceProcessor(tracer); + const traceData = { traceId: 'trace-no-content', name: 'Agent' } as any; + await processor.onTraceStart(traceData); + + // Generation span with input/output + const genSpan = { + spanId: 'gen-no-content', + traceId: 'trace-no-content', + startedAt: new Date().toISOString(), + spanData: { + type: 'generation' as const, + model: 'gpt-4', + input: [{ role: 'user', content: 'secret prompt' }], + output: { id: 'resp-1', choices: [{ text: 'secret response' }] }, + }, + } as any; + + await processor.onSpanStart(genSpan); + await processor.onSpanEnd(genSpan); + + const mock = spansByName['generation']; + const attrs = mock._attrs as Array<[string, unknown]>; + const contentKeys = attrs.filter(([k]) => + k === OpenTelemetryConstants.GEN_AI_INPUT_MESSAGES_KEY || + k === OpenTelemetryConstants.GEN_AI_OUTPUT_MESSAGES_KEY + ); + expect(contentKeys).toHaveLength(0); + + // Model attribute should still be present (non-content) + const modelAttr = attrs.find(([k]) => k === OpenTelemetryConstants.GEN_AI_REQUEST_MODEL_KEY); + expect(modelAttr).toBeDefined(); }); }); diff --git a/tests/observability/extension/openai/OpenAIMessageContract.test.ts b/tests/observability/extension/openai/OpenAIMessageContract.test.ts deleted file mode 100644 index 47f98906..00000000 --- a/tests/observability/extension/openai/OpenAIMessageContract.test.ts +++ /dev/null @@ -1,239 +0,0 @@ -// Copyright (c) Microsoft Corporation. -// Licensed under the MIT License. - -import { describe, it, expect, beforeEach, afterEach } from '@jest/globals'; -import { Tracer, trace } from '@opentelemetry/api'; -import { OpenTelemetryConstants, ObservabilityManager, serializeMessages } from '@microsoft/agents-a365-observability'; -import { OpenAIAgentsTraceProcessor } from '@microsoft/agents-a365-observability-extensions-openai'; -import { - buildStructuredInputMessages, - buildStructuredOutputMessages, - wrapRawContentAsInputMessages, - wrapRawContentAsOutputMessages, -} from '../../../../packages/agents-a365-observability-extensions-openai/src/Utils'; -import { expectValidInputMessages, expectValidOutputMessages, getAttrFromArray } from '../helpers/message-schema-validator'; - -describe('OpenAI Message Contract Tests', () => { - - describe('buildStructuredInputMessages', () => { - it('should produce valid InputMessages from a multi-role conversation', () => { - const result = buildStructuredInputMessages([ - { role: 'system', content: 'You are a helpful assistant.' }, - { role: 'user', content: 'Hi!' }, - { role: 'assistant', content: 'Hello! How can I help?' }, - { role: 'user', content: 'What is 2+2?' }, - ]); - expectValidInputMessages(serializeMessages(result)); - expect(result.messages.map(m => m.role)).toEqual(['system', 'user', 'assistant', 'user']); - expect(result.messages[0].parts[0]).toEqual({ type: 'text', content: 'You are a helpful assistant.' }); - }); - - it('should handle array content blocks (input_text, input_image)', () => { - const result = buildStructuredInputMessages([{ - role: 'user', - content: [ - { type: 'input_text', text: 'Describe this image' }, - { type: 'input_image', image: 'https://example.com/img.png' }, - ], - }] as any); - expectValidInputMessages(serializeMessages(result)); - expect(result.messages[0].parts).toHaveLength(2); - expect(result.messages[0].parts[0]).toEqual({ type: 'text', content: 'Describe this image' }); - }); - }); - - describe('buildStructuredOutputMessages', () => { - it('should handle text, tool_call, and reasoning content', () => { - const result = buildStructuredOutputMessages([{ - role: 'assistant', - content: [ - { type: 'reasoning', text: 'The user asked about weather.' }, - { type: 'output_text', text: 'Let me check.' }, - { type: 'tool_call', name: 'get_weather', call_id: 'call_1', arguments: '{"city":"Seattle"}' }, - ], - }]); - expectValidOutputMessages(serializeMessages(result)); - - expect(result.messages[0].parts).toHaveLength(3); - const toolPart = result.messages[0].parts.find(p => p.type === 'tool_call') as any; - expect(toolPart.name).toBe('get_weather'); - expect(toolPart.id).toBe('call_1'); - expect(result.messages[0].parts.find(p => p.type === 'reasoning')).toBeDefined(); - }); - - it('should handle mixed output types including refusal', () => { - const result = buildStructuredOutputMessages([{ - role: 'assistant', - content: [ - { type: 'output_text', text: 'Here is the answer.' }, - { type: 'refusal', refusal: 'I cannot help with that.' }, - ], - }]); - expectValidOutputMessages(serializeMessages(result)); - expect(result.messages[0].parts).toHaveLength(2); - }); - }); - - describe('wrapRawContent', () => { - it.each([ - ['string', 'Hello prompt'], - ['object', { complex: 'data', nested: [1, 2] }], - ])('should produce valid InputMessages from raw %s', (_label, raw) => { - expectValidInputMessages(serializeMessages(wrapRawContentAsInputMessages(raw))); - }); - - it.each([ - ['string', 'Model response'], - ['object', { result: 'data' }], - ])('should produce valid OutputMessages from raw %s', (_label, raw) => { - expectValidOutputMessages(serializeMessages(wrapRawContentAsOutputMessages(raw))); - }); - }); - - describe('End-to-end: response span with real-shaped data', () => { - let tracer: Tracer; - let spansByName: Record; - let tracerSpy: jest.SpyInstance; - - const createMockSpan = (name: string) => { - const attrs: Array<[string, unknown]> = []; - return { - setAttribute: jest.fn((k: string, v: unknown) => { attrs.push([k, v]); }), - updateName: jest.fn(), - setStatus: jest.fn(), - end: jest.fn(), - spanContext: jest.fn(() => ({ traceId: 'tid-' + name, spanId: 'sid-' + name })), - _attrs: attrs, - }; - }; - - beforeEach(() => { - ObservabilityManager.start({ serviceName: 'contract-test', serviceVersion: '1.0.0' }); - tracer = trace.getTracer('contract-test', '1.0.0'); - spansByName = {}; - tracerSpy = jest.spyOn(tracer as any, 'startSpan').mockImplementation((...args: unknown[]) => { - const s = createMockSpan(args[0] as string); - spansByName[args[0] as string] = s; - return s; - }); - }); - - afterEach(async () => { - tracerSpy.mockRestore(); - await ObservabilityManager.shutdown(); - }); - - async function runResponseSpan(traceId: string, spanName: string, input: any[], responseOutput: any[]): Promise> { - const processor = new OpenAIAgentsTraceProcessor(tracer, {}); - await processor.onTraceStart({ traceId, name: 'Agent' } as any); - const span = { - spanId: `sid-${spanName}`, - traceId, - startedAt: new Date().toISOString(), - spanData: { - type: 'response' as const, - name: spanName, - _input: input, - _response: { model: 'gpt-4o', output: responseOutput }, - }, - } as any; - await processor.onSpanStart(span); - await processor.onSpanEnd(span); - await processor.shutdown(); - return spansByName[spanName]._attrs; - } - - it('should produce valid InputMessages and OutputMessages for text response', async () => { - const attrs = await runResponseSpan('t1', 'TextResponse', - [{ role: 'system', content: 'You are helpful.' }, { role: 'user', content: 'What is 2+2?' }], - [{ role: 'assistant', content: [{ type: 'output_text', text: 'The answer is 4.' }] }], - ); - - expectValidInputMessages(getAttrFromArray(attrs, OpenTelemetryConstants.GEN_AI_INPUT_MESSAGES_KEY)); - expectValidOutputMessages(getAttrFromArray(attrs, OpenTelemetryConstants.GEN_AI_OUTPUT_MESSAGES_KEY)); - }); - - it('should produce valid OutputMessages with tool_call in response', async () => { - const attrs = await runResponseSpan('t2', 'ToolResponse', - [{ role: 'user', content: 'Get weather in Seattle' }], - [{ role: 'assistant', content: [{ type: 'tool_call', name: 'get_weather', call_id: 'call_abc', arguments: '{"city":"Seattle"}' }] }], - ); - - const outputValue = getAttrFromArray(attrs, OpenTelemetryConstants.GEN_AI_OUTPUT_MESSAGES_KEY); - expectValidOutputMessages(outputValue); - - const toolPart = JSON.parse(outputValue as string).messages[0].parts.find((p: any) => p.type === 'tool_call'); - expect(toolPart.name).toBe('get_weather'); - expect(toolPart.id).toBe('call_abc'); - expect(toolPart.arguments).toEqual({ city: 'Seattle' }); - }); - }); - - describe('Edge cases', () => { - it('should return empty messages array for empty input', () => { - const result = buildStructuredInputMessages([]); - expect(result.version).toBe('0.1.0'); - expect(result.messages).toHaveLength(0); - }); - - it('should skip null/non-object entries in input array', () => { - const result = buildStructuredInputMessages([null as any, undefined as any, { role: 'user', content: 'valid' }]); - expect(result.messages).toHaveLength(1); - expect(result.messages[0].parts[0]).toEqual({ type: 'text', content: 'valid' }); - }); - - it('should return empty messages array for empty output', () => { - const result = buildStructuredOutputMessages([]); - expect(result.messages).toHaveLength(0); - }); - - it('should map function_call content block to tool_call part', () => { - const result = buildStructuredOutputMessages([{ - role: 'assistant', - content: [ - { type: 'function_call', name: 'my_func', id: 'fc_1', arguments: '{"x":1}' }, - ], - }]); - expect(result.messages[0].parts[0].type).toBe('tool_call'); - const part = result.messages[0].parts[0] as any; - expect(part.name).toBe('my_func'); - expect(part.id).toBe('fc_1'); - expect(part.arguments).toEqual({ x: 1 }); - }); - - it('should map input_file block with modality from mime_type', () => { - const result = buildStructuredInputMessages([{ - role: 'user', - content: [ - { type: 'input_file', mime_type: 'application/pdf', file_id: 'file_123' }, - ], - }] as any); - expect(result.messages[0].parts[0].type).toBe('file'); - expect((result.messages[0].parts[0] as any).modality).toBe('application'); - }); - - it('should fall back to raw wrapper when tool_call arguments are malformed JSON', () => { - const result = buildStructuredOutputMessages([{ - role: 'assistant', - content: [ - { type: 'tool_call', name: 'get_weather', call_id: 'c1', arguments: 'not-json{{{' }, - ], - }]); - const part = result.messages[0].parts[0] as any; - expect(part.type).toBe('tool_call'); - expect(part.arguments).toEqual({ raw: 'not-json{{{' }); - }); - - it('should produce a generic part for unknown input content block types', () => { - const result = buildStructuredInputMessages([{ - role: 'user', - content: [ - { type: 'future_block_type', some_field: 'value' }, - ], - }] as any); - const part = result.messages[0].parts[0] as any; - expect(part.type).toBe('future_block_type'); - expect(typeof part.content).toBe('string'); - }); - }); -}); diff --git a/tests/observability/integration/openai-agent-instrument.test.ts b/tests/observability/integration/openai-agent-instrument.test.ts index eccff909..16d38e6b 100644 --- a/tests/observability/integration/openai-agent-instrument.test.ts +++ b/tests/observability/integration/openai-agent-instrument.test.ts @@ -43,6 +43,7 @@ describe("OpenAI Trace Processor Integration Tests", () => { enabled: true, tracerName: TEST_INSTRUMENTATION_NAME, tracerVersion: TEST_INSTRUMENTATION_VERSION, + isContentRecordingEnabled: true, }); // Start observability @@ -348,7 +349,7 @@ describe("OpenAI Trace Processor Integration Tests", () => { ).toBe('{"a":15,"b":27}'); expect( toolSpan.attributes[OpenTelemetryConstants.GEN_AI_TOOL_CALL_RESULT_KEY], - ).toBe('{"result":"The sum of 15 and 27 is 42"}'); + ).toBe("The sum of 15 and 27 is 42"); validateParentChildRelationship(toolSpan, agentSpan!); diff --git a/tests/observability/tracing/truncation.test.ts b/tests/observability/tracing/truncation.test.ts new file mode 100644 index 00000000..9001abe4 --- /dev/null +++ b/tests/observability/tracing/truncation.test.ts @@ -0,0 +1,51 @@ +// Copyright (c) Microsoft Corporation. +// Licensed under the MIT License. + +import { describe, it, expect } from '@jest/globals'; +import { truncateValue, MAX_ATTRIBUTE_LENGTH } from '../../../packages/agents-a365-observability/src/tracing/util'; + +describe('truncateValue', () => { + const SUFFIX = '...[truncated]'; + + it('should return the original string when within limit', () => { + const value = 'hello world'; + expect(truncateValue(value)).toBe(value); + }); + + it('should return the original string when exactly at limit', () => { + const value = 'x'.repeat(MAX_ATTRIBUTE_LENGTH); + expect(truncateValue(value)).toBe(value); + expect(truncateValue(value).length).toBe(MAX_ATTRIBUTE_LENGTH); + }); + + it('should truncate when 1 character over limit', () => { + const value = 'x'.repeat(MAX_ATTRIBUTE_LENGTH + 1); + const result = truncateValue(value); + expect(result.length).toBe(MAX_ATTRIBUTE_LENGTH); + expect(result.endsWith(SUFFIX)).toBe(true); + }); + + it('should truncate long strings to exactly MAX_ATTRIBUTE_LENGTH', () => { + const value = 'a'.repeat(MAX_ATTRIBUTE_LENGTH * 2); + const result = truncateValue(value); + expect(result.length).toBe(MAX_ATTRIBUTE_LENGTH); + expect(result.endsWith(SUFFIX)).toBe(true); + }); + + it('should preserve the beginning of the string when truncating', () => { + const prefix = 'PREFIX_'; + const value = prefix + 'x'.repeat(MAX_ATTRIBUTE_LENGTH); + const result = truncateValue(value); + expect(result.startsWith(prefix)).toBe(true); + }); + + it('should return empty string unchanged', () => { + expect(truncateValue('')).toBe(''); + }); +}); + +describe('MAX_ATTRIBUTE_LENGTH', () => { + it('should be 8192', () => { + expect(MAX_ATTRIBUTE_LENGTH).toBe(8_192); + }); +}); diff --git a/tests/tooling/McpToolServerConfigurationService.test.ts b/tests/tooling/McpToolServerConfigurationService.test.ts index 1b85d73f..5ab9ce10 100644 --- a/tests/tooling/McpToolServerConfigurationService.test.ts +++ b/tests/tooling/McpToolServerConfigurationService.test.ts @@ -454,7 +454,6 @@ describe('McpToolServerConfigurationService', () => { // Assert expect(servers).toHaveLength(1); - // Dev mode: attachDevTokens is used instead of per-audience OBO — no token exchange expect(getAgenticUserTokenSpy).not.toHaveBeenCalled(); expect(resolveAgentIdentitySpy).toHaveBeenCalledWith(mockContext, mockToken); }); @@ -469,7 +468,6 @@ describe('McpToolServerConfigurationService', () => { jest.spyOn(fs, 'existsSync').mockReturnValue(true); jest.spyOn(fs, 'readFileSync').mockReturnValue(JSON.stringify(manifestContent)); resolveAgentIdentitySpy.mockReturnValue('resolved-agent-id'); - getAgenticUserTokenSpy.mockResolvedValue(mockToken); // Act - new signature with all parameters const servers = await service.listToolServers( @@ -611,7 +609,6 @@ describe('McpToolServerConfigurationService', () => { jest.spyOn(fs, 'existsSync').mockReturnValue(true); const readFileSpy = jest.spyOn(fs, 'readFileSync').mockReturnValue(JSON.stringify(manifestContent)); resolveAgentIdentitySpy.mockReturnValue('resolved-agent-id'); - getAgenticUserTokenSpy.mockResolvedValue(mockToken); // Act - use new signature in development mode const servers = await service.listToolServers(mockContext, mockAuthorization, 'graph', mockToken); @@ -647,7 +644,6 @@ describe('McpToolServerConfigurationService', () => { describe('listToolServers new signature (production mode)', () => { let mockContext: TurnContext; let mockAuthorization: Authorization; - let getAgenticUserTokenSpy: jest.SpiedFunction; let resolveAgentIdentitySpy: jest.SpiedFunction; let validateAuthTokenSpy: jest.SpiedFunction; // eslint-disable-next-line @typescript-eslint/no-explicit-any @@ -678,9 +674,6 @@ describe('McpToolServerConfigurationService', () => { mockAuthorization = {} as Authorization; - // Mock per-server token acquisition (attachPerAudienceTokens is called after gateway discovery) - getAgenticUserTokenSpy = jest.spyOn(AgenticAuthenticationService, 'GetAgenticUserToken') - .mockResolvedValue(createMockJwt()); resolveAgentIdentitySpy = jest.spyOn(RuntimeUtility, 'ResolveAgentIdentity'); validateAuthTokenSpy = jest.spyOn(Utility, 'ValidateAuthToken').mockImplementation(() => {}); @@ -692,7 +685,6 @@ describe('McpToolServerConfigurationService', () => { afterEach(() => { process.env.NODE_ENV = originalEnv; - getAgenticUserTokenSpy.mockRestore(); resolveAgentIdentitySpy.mockRestore(); validateAuthTokenSpy.mockRestore(); axiosGetSpy.mockRestore(); @@ -730,7 +722,7 @@ describe('McpToolServerConfigurationService', () => { // Assert expect(axiosGetSpy).toHaveBeenCalledWith( - expect.stringContaining('/agents/v2/my-agent-id/mcpServers'), + expect.stringContaining('/agents/my-agent-id/mcpServers'), expect.any(Object) ); }); @@ -775,7 +767,7 @@ describe('McpToolServerConfigurationService', () => { // Assert - verify the custom endpoint is used in the gateway URL expect(axiosGetSpy).toHaveBeenCalledWith( - `${customEndpoint}/agents/v2/my-agent-id/mcpServers`, + `${customEndpoint}/agents/my-agent-id/mcpServers`, expect.any(Object) ); }); @@ -815,12 +807,12 @@ describe('McpToolServerConfigurationService', () => { // Assert - each service uses its own endpoint expect(axiosGetSpy).toHaveBeenNthCalledWith( 1, - `${tenant1Endpoint}/agents/v2/agent-1/mcpServers`, + `${tenant1Endpoint}/agents/agent-1/mcpServers`, expect.any(Object) ); expect(axiosGetSpy).toHaveBeenNthCalledWith( 2, - `${tenant2Endpoint}/agents/v2/agent-2/mcpServers`, + `${tenant2Endpoint}/agents/agent-2/mcpServers`, expect.any(Object) ); }); @@ -849,7 +841,7 @@ describe('McpToolServerConfigurationService', () => { // Assert - URL should not have double slashes expect(axiosGetSpy).toHaveBeenCalledWith( - 'https://custom.endpoint.com/agents/v2/my-agent-id/mcpServers', + 'https://custom.endpoint.com/agents/my-agent-id/mcpServers', expect.any(Object) ); }); @@ -1052,9 +1044,7 @@ describe('McpToolServerConfigurationService', () => { await service1.listToolServers(mockContext, mockAuthorization, 'graph'); await service2.listToolServers(mockContext, mockAuthorization, 'graph'); - // Assert - each service uses its own scope for gateway discovery. - // Dev mode (useToolingManifest=true) uses attachDevTokens — no per-server OBO call. - // So each service makes exactly one discovery call. + // Assert - each service uses its own scope expect(getAgenticUserTokenSpy).toHaveBeenNthCalledWith( 1, mockAuthorization, @@ -1116,7 +1106,7 @@ describe('McpToolServerConfigurationService', () => { // Assert - should use the environment-based endpoint expect(axiosGetSpy).toHaveBeenCalledWith( - `${customEndpoint}/agents/v2/my-agent-id/mcpServers`, + `${customEndpoint}/agents/my-agent-id/mcpServers`, expect.any(Object) ); }); @@ -1130,418 +1120,4 @@ describe('McpToolServerConfigurationService', () => { return `${header}.${payload}.${signature}`; } }); - - describe('dev mode token attachment (TokenAcquirer with env vars)', () => { - let mockContext: TurnContext; - let mockAuthorization: Authorization; - let resolveAgentIdentitySpy: jest.SpiedFunction; - let getAgenticUserTokenSpy: jest.SpiedFunction; - - const createMockJwt = () => { - const header = Buffer.from(JSON.stringify({ alg: 'HS256', typ: 'JWT' })).toString('base64url'); - const payload = Buffer.from(JSON.stringify({ exp: Math.floor(Date.now() / 1000) + 3600 })).toString('base64url'); - return `${header}.${payload}.mock-sig`; - }; - - beforeEach(() => { - process.env.NODE_ENV = 'Development'; - mockContext = { - activity: { - from: { agenticAppBlueprintId: 'blueprint-dev' }, - channelId: 'msteams', - recipient: { id: 'recipient-id' }, - conversation: { id: 'conv-id' }, - isAgenticRequest: jest.fn().mockReturnValue(false), - getAgenticInstanceId: jest.fn().mockReturnValue(undefined) - }, - sendActivity: jest.fn() - } as unknown as TurnContext; - mockAuthorization = {} as Authorization; - resolveAgentIdentitySpy = jest.spyOn(RuntimeUtility, 'ResolveAgentIdentity').mockReturnValue('agent-id'); - getAgenticUserTokenSpy = jest.spyOn(AgenticAuthenticationService, 'GetAgenticUserToken'); - }); - - afterEach(() => { - resolveAgentIdentitySpy.mockRestore(); - getAgenticUserTokenSpy.mockRestore(); - delete process.env.BEARER_TOKEN_MAILSERVER; - delete process.env.BEARER_TOKEN_CALENDARSERVER; - delete process.env.BEARER_TOKEN_SERVER1; - delete process.env.BEARER_TOKEN_SERVER2; - }); - - it('should not call GetAgenticUserToken for per-server tokens in dev mode (env var acquirer, not OBO)', async () => { - const mockToken = createMockJwt(); - const manifestContent = { - mcpServers: [{ mcpServerName: 'testServer', url: 'http://localhost:3000' }] - }; - jest.spyOn(fs, 'existsSync').mockReturnValue(true); - jest.spyOn(fs, 'readFileSync').mockReturnValue(JSON.stringify(manifestContent)); - - // Pre-provide token so discovery is also skipped - await service.listToolServers(mockContext, mockAuthorization, 'graph', mockToken); - - // attachDevTokens reads env vars — no OBO exchange at all in dev mode - expect(getAgenticUserTokenSpy).not.toHaveBeenCalled(); - }); - - it('should attach independent BEARER_TOKEN_ headers for two V2 servers with distinct audiences', async () => { - // V2 servers have unique audience GUIDs → unique scopes → separate cache entries → independent env var lookups. - const token1 = 'dev-token-server1'; - const token2 = 'dev-token-server2'; - process.env.BEARER_TOKEN_SERVER1 = token1; - process.env.BEARER_TOKEN_SERVER2 = token2; - const mockToken = createMockJwt(); - const manifestContent = { - mcpServers: [ - { mcpServerName: 'server1', url: 'http://localhost:3000', audience: 'aaaabbbb-0001-0001-0001-000000000001' }, - { mcpServerName: 'server2', url: 'http://localhost:3001', audience: 'aaaabbbb-0002-0002-0002-000000000002' } - ] - }; - jest.spyOn(fs, 'existsSync').mockReturnValue(true); - jest.spyOn(fs, 'readFileSync').mockReturnValue(JSON.stringify(manifestContent)); - - const servers = await service.listToolServers(mockContext, mockAuthorization, 'graph', mockToken); - - expect(servers[0].headers?.Authorization).toBe(`Bearer ${token1}`); - expect(servers[1].headers?.Authorization).toBe(`Bearer ${token2}`); - }); - - it('should attach BEARER_TOKEN_ independently for V2 server; V1 server with no env var gets no header', async () => { - // V2 servers have a unique audience GUID → unique scope → own cache entry → own env var lookup. - // V1 servers (no audience) share the ATG scope → own cache entry → own env var lookup (no fallback). - const perServerToken = 'per-server-mail-token'; - const v2Audience = 'aaaabbbb-1234-5678-abcd-111122223333'; - process.env.BEARER_TOKEN_MAILSERVER = perServerToken; - const mockToken = createMockJwt(); - const manifestContent = { - mcpServers: [ - { mcpServerName: 'mailServer', url: 'http://localhost:3000', audience: v2Audience }, // V2 — unique scope - { mcpServerName: 'calendarServer', url: 'http://localhost:3001' } // V1 — ATG scope, no env var - ] - }; - jest.spyOn(fs, 'existsSync').mockReturnValue(true); - jest.spyOn(fs, 'readFileSync').mockReturnValue(JSON.stringify(manifestContent)); - - const servers = await service.listToolServers(mockContext, mockAuthorization, 'graph', mockToken); - - // mailServer (V2): unique scope → BEARER_TOKEN_MAILSERVER set → header attached - expect(servers[0].headers?.Authorization).toBe(`Bearer ${perServerToken}`); - // calendarServer (V1): ATG scope → BEARER_TOKEN_CALENDARSERVER not set → no header - expect(servers[1].headers?.Authorization).toBeUndefined(); - }); - - it('should not attach Authorization header when no env var tokens are set', async () => { - const mockToken = createMockJwt(); - const manifestContent = { - mcpServers: [{ mcpServerName: 'testServer', url: 'http://localhost:3000' }] - }; - jest.spyOn(fs, 'existsSync').mockReturnValue(true); - jest.spyOn(fs, 'readFileSync').mockReturnValue(JSON.stringify(manifestContent)); - - const servers = await service.listToolServers(mockContext, mockAuthorization, 'graph', mockToken); - - expect(servers[0].headers?.Authorization).toBeUndefined(); - }); - }); - - describe('V1/V2 per-audience token acquisition (TurnContext path)', () => { - let mockContext: TurnContext; - let mockAuthorization: Authorization; - let getAgenticUserTokenSpy: jest.SpiedFunction; - let resolveAgentIdentitySpy: jest.SpiedFunction; - let validateAuthTokenSpy: jest.SpiedFunction; - - const createMockJwt = (seed = 'default') => { - const header = Buffer.from(JSON.stringify({ alg: 'HS256', typ: 'JWT' })).toString('base64url'); - const payload = Buffer.from(JSON.stringify({ exp: Math.floor(Date.now() / 1000) + 3600, sub: seed })).toString('base64url'); - return `${header}.${payload}.mock-sig`; - }; - - beforeEach(() => { - // Per-audience OBO is production-only — dev mode uses attachDevTokens instead - process.env.NODE_ENV = 'production'; - mockContext = { - activity: { - from: { agenticAppBlueprintId: 'blueprint-v2' }, - channelId: 'msteams', - recipient: { id: 'recipient-id' }, - conversation: { id: 'conv-id' }, - isAgenticRequest: jest.fn().mockReturnValue(false), - getAgenticInstanceId: jest.fn().mockReturnValue(undefined) - }, - sendActivity: jest.fn() - } as unknown as TurnContext; - mockAuthorization = {} as Authorization; - getAgenticUserTokenSpy = jest.spyOn(AgenticAuthenticationService, 'GetAgenticUserToken'); - resolveAgentIdentitySpy = jest.spyOn(RuntimeUtility, 'ResolveAgentIdentity').mockReturnValue('agent-id'); - validateAuthTokenSpy = jest.spyOn(Utility, 'ValidateAuthToken').mockImplementation(() => {}); - }); - - afterEach(() => { - getAgenticUserTokenSpy.mockRestore(); - resolveAgentIdentitySpy.mockRestore(); - validateAuthTokenSpy.mockRestore(); - }); - - it('should attach Authorization header using ATG scope for a V1 server (no audience field)', async () => { - const mockToken = createMockJwt('atg'); - // eslint-disable-next-line @typescript-eslint/no-require-imports - const axios = require('axios'); - jest.spyOn(axios, 'get').mockResolvedValue({ - data: [{ mcpServerName: 'mailServer', url: 'http://localhost:3001' }] - }); - getAgenticUserTokenSpy.mockResolvedValue(mockToken); - - const servers = await service.listToolServers(mockContext, mockAuthorization, 'graph', mockToken); - - expect(servers[0].headers?.Authorization).toBe(`Bearer ${mockToken}`); - expect(getAgenticUserTokenSpy).toHaveBeenCalledWith( - mockAuthorization, 'graph', mockContext, ['ea9ffc3e-8a23-4a7d-836d-234d7c7565c1/.default'] - ); - }); - - it('should acquire a per-server token using V2 server audience GUID as scope', async () => { - const v2Audience = 'aaaabbbb-1234-5678-abcd-111122223333'; - const v2Token = createMockJwt('v2'); - // eslint-disable-next-line @typescript-eslint/no-require-imports - const axios = require('axios'); - jest.spyOn(axios, 'get').mockResolvedValue({ - data: [{ - mcpServerName: 'v2ToolsServer', - url: 'https://v2.example.com/mcp', - audience: v2Audience, - scope: 'Tools.ListInvoke.All' - }] - }); - getAgenticUserTokenSpy.mockResolvedValue(v2Token); - - const servers = await service.listToolServers(mockContext, mockAuthorization, 'graph', v2Token); - - expect(getAgenticUserTokenSpy).toHaveBeenCalledWith( - mockAuthorization, 'graph', mockContext, [`${v2Audience}/Tools.ListInvoke.All`] - ); - expect(servers[0].headers?.Authorization).toBe(`Bearer ${v2Token}`); - }); - - it('should perform one token exchange for multiple servers sharing the same V2 audience', async () => { - const sharedAudience = 'aaaabbbb-1234-5678-abcd-111122223333'; - const sharedToken = createMockJwt('shared'); - // eslint-disable-next-line @typescript-eslint/no-require-imports - const axios = require('axios'); - jest.spyOn(axios, 'get').mockResolvedValue({ - data: [ - { mcpServerName: 'v2Server1', url: 'http://v2-1.example.com', audience: sharedAudience }, - { mcpServerName: 'v2Server2', url: 'http://v2-2.example.com', audience: sharedAudience }, - ] - }); - getAgenticUserTokenSpy.mockResolvedValue(sharedToken); - - const servers = await service.listToolServers(mockContext, mockAuthorization, 'graph', sharedToken); - - expect(getAgenticUserTokenSpy).toHaveBeenCalledTimes(1); - expect(servers[0].headers?.Authorization).toBe(`Bearer ${sharedToken}`); - expect(servers[1].headers?.Authorization).toBe(`Bearer ${sharedToken}`); - }); - - it('should use different tokens for V1 and V2 servers in the same list', async () => { - const v2Audience = 'ccccdddd-5678-9012-efab-444455556666'; - const atgToken = createMockJwt('atg'); - const v2Token = createMockJwt('v2'); - // eslint-disable-next-line @typescript-eslint/no-require-imports - const axios = require('axios'); - jest.spyOn(axios, 'get').mockResolvedValue({ - data: [ - { mcpServerName: 'v1MailServer', url: 'http://v1.example.com' }, - { mcpServerName: 'v2ToolsServer', url: 'http://v2.example.com', audience: v2Audience } - ] - }); - getAgenticUserTokenSpy - .mockResolvedValueOnce(atgToken) // V1 ATG scope - .mockResolvedValueOnce(v2Token); // V2 per-audience scope - - const servers = await service.listToolServers(mockContext, mockAuthorization, 'graph', atgToken); - - expect(getAgenticUserTokenSpy).toHaveBeenCalledTimes(2); - expect(getAgenticUserTokenSpy).toHaveBeenNthCalledWith(1, mockAuthorization, 'graph', mockContext, ['ea9ffc3e-8a23-4a7d-836d-234d7c7565c1/.default']); - expect(getAgenticUserTokenSpy).toHaveBeenNthCalledWith(2, mockAuthorization, 'graph', mockContext, [`${v2Audience}/.default`]); - expect(servers[0].headers?.Authorization).toBe(`Bearer ${atgToken}`); - expect(servers[1].headers?.Authorization).toBe(`Bearer ${v2Token}`); - }); - - it('should throw when per-server token exchange fails', async () => { - const mockToken = createMockJwt(); - // eslint-disable-next-line @typescript-eslint/no-require-imports - const axios = require('axios'); - jest.spyOn(axios, 'get').mockResolvedValue({ - data: [{ mcpServerName: 'mailServer', url: 'http://localhost:3001' }] - }); - // Discovery token OK; per-server exchange returns null - getAgenticUserTokenSpy - .mockResolvedValueOnce(mockToken) - .mockResolvedValueOnce(null as unknown as string); - - await expect( - service.listToolServers(mockContext, mockAuthorization, 'graph') - ).rejects.toThrow("Failed to obtain token for MCP server 'mailServer'"); - }); - - it('should use OBO acquirer (not env var acquirer) in production mode', async () => { - // Verifies the prod branch: gateway discovery → OBO per-server token - const mockToken = createMockJwt('prod'); - // eslint-disable-next-line @typescript-eslint/no-require-imports - const axios = require('axios'); - jest.spyOn(axios, 'get').mockResolvedValue({ - data: [{ mcpServerName: 'prodServer', url: 'http://prod.example.com' }] - }); - getAgenticUserTokenSpy.mockResolvedValue(mockToken); - - const servers = await service.listToolServers(mockContext, mockAuthorization, 'graph', mockToken); - - // OBO must have been called for per-server token (ATG scope for V1 server) - expect(getAgenticUserTokenSpy).toHaveBeenCalledWith( - mockAuthorization, 'graph', mockContext, ['ea9ffc3e-8a23-4a7d-836d-234d7c7565c1/.default'] - ); - expect(servers[0].headers?.Authorization).toBe(`Bearer ${mockToken}`); - }); - - it('should pass audience and scope through from gateway into MCPServerConfig (TurnContext path)', async () => { - // Verifies gateway response fields are preserved end-to-end using the preferred TurnContext path. - const v2Audience = 'eeeeffff-0000-1111-2222-333344445555'; - const mockToken = createMockJwt('v2-fields'); - // eslint-disable-next-line @typescript-eslint/no-require-imports - const axios = require('axios'); - jest.spyOn(axios, 'get').mockResolvedValue({ - data: [{ - mcpServerName: 'v2Server', - url: 'https://v2.example.com/mcp', - audience: v2Audience, - scope: 'Tools.ListInvoke.All' - }] - }); - getAgenticUserTokenSpy.mockResolvedValue(mockToken); - - const servers = await service.listToolServers(mockContext, mockAuthorization, 'graph', mockToken); - - expect(servers[0].audience).toBe(v2Audience); - expect(servers[0].scope).toBe('Tools.ListInvoke.All'); - }); - }); - - describe('listToolServers legacy path — per-audience token attachment', () => { - const createMockJwt = () => { - const header = Buffer.from(JSON.stringify({ alg: 'HS256', typ: 'JWT' })).toString('base64url'); - const payload = Buffer.from(JSON.stringify({ exp: Math.floor(Date.now() / 1000) + 3600 })).toString('base64url'); - return `${header}.${payload}.mock-sig`; - }; - - afterEach(() => { - delete process.env.BEARER_TOKEN; - delete process.env.BEARER_TOKEN_MAILSERVER; - delete process.env.BEARER_TOKEN_V2SERVER; - }); - - describe('dev mode (manifest)', () => { - beforeEach(() => { process.env.NODE_ENV = 'Development'; }); - - it('should attach BEARER_TOKEN for a V1 server when BEARER_TOKEN is set', async () => { - process.env.BEARER_TOKEN = 'shared-dev-token'; - const manifestContent = { - mcpServers: [{ mcpServerName: 'mailServer', url: 'http://localhost:3000' }] - }; - jest.spyOn(fs, 'existsSync').mockReturnValue(true); - jest.spyOn(fs, 'readFileSync').mockReturnValue(JSON.stringify(manifestContent)); - - const servers = await service.listToolServers('agent-id', 'mock-auth-token'); - - expect(servers[0].headers?.Authorization).toBe('Bearer shared-dev-token'); - }); - - it('should attach BEARER_TOKEN_ for a V2 server when per-server env var is set', async () => { - process.env.BEARER_TOKEN_V2SERVER = 'v2-dev-token'; - const v2Audience = 'aaaabbbb-1234-5678-abcd-111122223333'; - const manifestContent = { - mcpServers: [{ mcpServerName: 'v2Server', url: 'http://localhost:3001', audience: v2Audience }] - }; - jest.spyOn(fs, 'existsSync').mockReturnValue(true); - jest.spyOn(fs, 'readFileSync').mockReturnValue(JSON.stringify(manifestContent)); - - const servers = await service.listToolServers('agent-id', 'mock-auth-token'); - - expect(servers[0].headers?.Authorization).toBe('Bearer v2-dev-token'); - }); - - it('should leave headers undefined when no env var tokens are set', async () => { - const manifestContent = { - mcpServers: [{ mcpServerName: 'mailServer', url: 'http://localhost:3000' }] - }; - jest.spyOn(fs, 'existsSync').mockReturnValue(true); - jest.spyOn(fs, 'readFileSync').mockReturnValue(JSON.stringify(manifestContent)); - - const servers = await service.listToolServers('agent-id', 'mock-auth-token'); - - expect(servers[0].headers?.Authorization).toBeUndefined(); - }); - }); - - describe('prod mode (gateway)', () => { - beforeEach(() => { - process.env.NODE_ENV = 'production'; - jest.spyOn(Utility, 'ValidateAuthToken').mockImplementation(() => {}); - }); - - it('should attach the shared authToken for a V1 server (no audience)', async () => { - const mockToken = createMockJwt(); - // eslint-disable-next-line @typescript-eslint/no-require-imports - const axios = require('axios'); - jest.spyOn(axios, 'get').mockResolvedValue({ - data: [{ mcpServerName: 'mailServer', url: 'http://prod.example.com' }] - }); - - const servers = await service.listToolServers('agent-id', mockToken); - - expect(servers[0].headers?.Authorization).toBe(`Bearer ${mockToken}`); - }); - - it('should throw for a V2 server (non-ATG audience) with a message directing to the TurnContext overload', async () => { - const v2Audience = 'aaaabbbb-1234-5678-abcd-111122223333'; - // eslint-disable-next-line @typescript-eslint/no-require-imports - const axios = require('axios'); - jest.spyOn(axios, 'get').mockResolvedValue({ - data: [{ mcpServerName: 'v2Server', url: 'http://v2.example.com', audience: v2Audience }] - }); - - await expect( - service.listToolServers('agent-id', 'mock-auth-token') - ).rejects.toThrow("MCP server 'v2Server' requires a per-audience token"); - }); - - it('should throw with a message that names the migration overload', async () => { - const v2Audience = 'ccccdddd-5678-9012-efab-444455556666'; - // eslint-disable-next-line @typescript-eslint/no-require-imports - const axios = require('axios'); - jest.spyOn(axios, 'get').mockResolvedValue({ - data: [{ mcpServerName: 'v2Tools', url: 'http://v2tools.example.com', audience: v2Audience }] - }); - - await expect( - service.listToolServers('agent-id', 'mock-auth-token') - ).rejects.toThrow('listToolServers(turnContext, authorization, authHandlerName)'); - }); - - it('should NOT throw for a V1 server whose audience explicitly equals the shared ATG AppId', async () => { - const atgAppId = 'ea9ffc3e-8a23-4a7d-836d-234d7c7565c1'; - const mockToken = createMockJwt(); - // eslint-disable-next-line @typescript-eslint/no-require-imports - const axios = require('axios'); - jest.spyOn(axios, 'get').mockResolvedValue({ - data: [{ mcpServerName: 'legacyServer', url: 'http://legacy.example.com', audience: atgAppId }] - }); - - const servers = await service.listToolServers('agent-id', mockToken); - - expect(servers[0].headers?.Authorization).toBe(`Bearer ${mockToken}`); - }); - }); - }); }); diff --git a/tests/tooling/configuration/ToolingConfiguration.test.ts b/tests/tooling/configuration/ToolingConfiguration.test.ts index 098ff7da..787acbf6 100644 --- a/tests/tooling/configuration/ToolingConfiguration.test.ts +++ b/tests/tooling/configuration/ToolingConfiguration.test.ts @@ -4,8 +4,7 @@ import { describe, it, expect, beforeEach, afterEach } from '@jest/globals'; import { ToolingConfiguration, - defaultToolingConfigurationProvider, - resolveTokenScopeForServer + defaultToolingConfigurationProvider } from '../../../packages/agents-a365-tooling/src'; import { RuntimeConfiguration, DefaultConfigurationProvider, ClusterCategory } from '../../../packages/agents-a365-runtime/src'; @@ -225,41 +224,6 @@ describe('ToolingConfiguration', () => { }); }); - describe('getBearerTokenForServer', () => { - it('should return per-server token when BEARER_TOKEN_ is set', () => { - process.env.BEARER_TOKEN_MYSERVER = 'per-server-token'; - const config = new ToolingConfiguration({}); - expect(config.getBearerTokenForServer('myserver')).toBe('per-server-token'); - }); - - it('should fall back to BEARER_TOKEN when per-server var is not set', () => { - delete process.env.BEARER_TOKEN_MYSERVER; - process.env.BEARER_TOKEN = 'shared-token'; - const config = new ToolingConfiguration({}); - expect(config.getBearerTokenForServer('myserver')).toBe('shared-token'); - }); - - it('should return undefined when neither per-server nor shared token is set', () => { - delete process.env.BEARER_TOKEN_MYSERVER; - delete process.env.BEARER_TOKEN; - const config = new ToolingConfiguration({}); - expect(config.getBearerTokenForServer('myserver')).toBeUndefined(); - }); - - it('should prefer per-server token over shared BEARER_TOKEN when both are set', () => { - process.env.BEARER_TOKEN_MYSERVER = 'per-server-token'; - process.env.BEARER_TOKEN = 'shared-token'; - const config = new ToolingConfiguration({}); - expect(config.getBearerTokenForServer('myserver')).toBe('per-server-token'); - }); - - it('should uppercase the server name when looking up the env var', () => { - process.env.BEARER_TOKEN_MY_SERVER = 'upper-token'; - const config = new ToolingConfiguration({}); - expect(config.getBearerTokenForServer('my_server')).toBe('upper-token'); - }); - }); - describe('combined overrides', () => { it('should allow overriding both runtime and tooling settings', () => { const config = new ToolingConfiguration({ @@ -289,89 +253,6 @@ describe('ToolingConfiguration', () => { }); }); -describe('resolveTokenScopeForServer', () => { - const ATG_SCOPE = 'ea9ffc3e-8a23-4a7d-836d-234d7c7565c1/.default'; - const ATG_APP_ID = 'ea9ffc3e-8a23-4a7d-836d-234d7c7565c1'; - - it('should return ATG scope when audience is undefined (V1 server)', () => { - expect(resolveTokenScopeForServer({ mcpServerName: 'mail', url: 'https://mail.example.com' })).toBe(ATG_SCOPE); - }); - - it('should return ATG scope when audience equals the shared ATG AppId', () => { - expect(resolveTokenScopeForServer({ mcpServerName: 'mail', url: 'https://mail.example.com', audience: ATG_APP_ID })).toBe(ATG_SCOPE); - }); - - it('should return ATG scope when audience is the ATG api:// URI form', () => { - const atgAppIdUri = `api://${ATG_APP_ID}`; - expect(resolveTokenScopeForServer({ mcpServerName: 'mail', url: 'https://mail.example.com', audience: atgAppIdUri })).toBe(ATG_SCOPE); - }); - - it('should return per-server scope when audience is a non-ATG api:// URI (V2 server)', () => { - const v2AppIdUri = 'api://custom-app-id'; - expect(resolveTokenScopeForServer({ mcpServerName: 'mail', url: 'https://mail.example.com', audience: v2AppIdUri })).toBe(`${v2AppIdUri}/.default`); - }); - - it('should return per-server scope for a V2 GUID audience', () => { - const v2AppId = 'aaaabbbb-1234-5678-abcd-111122223333'; - expect(resolveTokenScopeForServer({ mcpServerName: 'tools', url: 'https://tools.example.com', audience: v2AppId })).toBe(`${v2AppId}/.default`); - }); - - it('should return per-server scope using explicit scope field when provided (V2)', () => { - const v2AppId = 'aaaabbbb-1234-5678-abcd-111122223333'; - expect(resolveTokenScopeForServer({ - mcpServerName: 'tools', - url: 'https://tools.example.com', - audience: v2AppId, - scope: 'Tools.ListInvoke.All' - })).toBe(`${v2AppId}/Tools.ListInvoke.All`); - }); - - describe('custom sharedScope (configurable mcpPlatformAuthenticationScope)', () => { - const customScope = 'api://custom-atg/.default'; - const customAudience = 'api://custom-atg'; - - it('should return customScope for a V1 server with no audience when sharedScope is overridden', () => { - expect(resolveTokenScopeForServer( - { mcpServerName: 'mail', url: 'https://mail.example.com' }, - customScope - )).toBe(customScope); - }); - - it('should return customScope when server audience matches the custom shared audience (api:// form)', () => { - expect(resolveTokenScopeForServer( - { mcpServerName: 'mail', url: 'https://mail.example.com', audience: customAudience }, - customScope - )).toBe(customScope); - }); - - it('should return customScope when server audience matches the custom shared audience (plain form)', () => { - // audience field may arrive as plain GUID/id even when sharedScope uses api:// prefix - expect(resolveTokenScopeForServer( - { mcpServerName: 'mail', url: 'https://mail.example.com', audience: 'custom-atg' }, - customScope - )).toBe(customScope); - }); - - it('should still treat a V2 server as V2 even when sharedScope is custom', () => { - const v2Audience = 'aaaabbbb-1234-5678-abcd-111122223333'; - expect(resolveTokenScopeForServer( - { mcpServerName: 'tools', url: 'https://tools.example.com', audience: v2Audience }, - customScope - )).toBe(`${v2Audience}/.default`); - }); - - it('should not raise false migration error in legacy prod acquirer when sharedScope is overridden', () => { - // Regression guard: with the old hardcoded constant, resolveTokenScopeForServer returned - // 'ea9ffc3e-.../.default' while createLegacyProdTokenAcquirer compared against the custom - // scope — mismatch → false throw. Now both sides use the same configured value. - expect(resolveTokenScopeForServer( - { mcpServerName: 'mail', url: 'https://mail.example.com' }, - customScope - )).toBe(customScope); // returned scope === sharedScope → no throw - }); - }); -}); - describe('defaultToolingConfigurationProvider', () => { const originalEnv = process.env; diff --git a/version.json b/version.json index 80dc7452..5f68d5b1 100644 --- a/version.json +++ b/version.json @@ -1,6 +1,6 @@ { "$schema": "https://raw.githubusercontent.com/dotnet/Nerdbank.GitVersioning/main/src/NerdBank.GitVersioning/version.schema.json", - "version": "0.2.0-preview.{height}", + "version": "0.1.0-preview.{height}", "publicReleaseRefSpec": [ "^refs/heads/main$", "^refs/heads/master$",