Trustworthy sub-agent delegation: tamper-evident transparency, harness-guaranteed Direct Line, and forced delegation

[iret77]

Problem

omadia routes every user turn through a single orchestrator agent. The orchestrator decides — entirely at LLM discretion — whether, when, and with what input to delegate to a specialist sub-agent, and it owns the rephrasing of the sub-agent's answer before the user sees it.

In day-to-day use (channel: MS Teams) this produces a concrete, reproducible failure mode:

1. A user creates a specialist agent (e.g. "Strategist", a sparring partner for strategy work) and asks the orchestrator to run a process together with the Strategist.
2. The orchestrator confirms — but its questions do not follow the Strategist's defined process. On inspection, the orchestrator had silently set up its own process first, intending to hand the whole package to the Strategist only at the end.
3. After the user complains, the orchestrator switches to acting as a "postman", relaying input and output — but the user now has no way to verify that relaying is faithful and complete, and no structural guarantee that the orchestrator will not quietly resume moderating the process itself.

This is gatekeeper drift: an orchestrator LLM gradually concentrates competence on itself and mediates everything. It is a known tendency of orchestrator-style agents on current frontier models and a blocking concern for enterprise adoption — when a company delegates real work to a named specialist, "trust me, I consulted them" is not acceptable; the delegation must be observable and non-suppressible.

Importantly, the orchestrator is not purely harmful here. It provides genuinely valuable functions — conversation scope, identity/auth, cross-session memory and recall, privacy/PII masking, multi-agent synthesis, follow-ups, scheduling. A naive "direct link" that cuts the orchestrator out would throw those away. The fix must keep the orchestrator in the loop and aware, while removing its ability to suppress or distort a requested delegation.

Goal & success criteria

Tier	Criterion	Definition of done
Minimum	Transparency	In every channel (incl. Teams), the user can see whether — and which — sub-agent was actually invoked this turn, sourced from the deterministic runtime trace, not from the orchestrator's narrative (so a fabricated "I asked the Strategist" is detectable).
Maximum	Reliability	The harness makes it structurally impossible for the orchestrator to (a) silently skip a requested specialist, or (b) suppress / falsify that specialist's input or output. We constrain the orchestrator's destructive powers only; its constructive powers (cross-cutting judgement, additive caveats) are preserved through a gated, attributed, non-redacting channel.

Verified current-state findings

All claims below were checked against the code at main (5e6cd7f).

There is already a deterministic choke point for sub-agent calls. Every delegation flows through Orchestrator.dispatchTool() → dispatchToolInner() → for a sub-agent (DomainTool) it always calls domainTool.handle(input, observer) (middleware/packages/harness-orchestrator/src/orchestrator.ts, ~:3164 / :3424 / :3466), which always calls agent.ask(question, observer) (tools/domainQueryTool.ts:185). The RunTraceCollector records each invocation deterministically. The LLM cannot fake or hide this — if a sub-agent was actually invoked, the trace contains it; if the orchestrator only claims it consulted the agent, the trace is empty. This is the linchpin for tamper-evidence.

The trace exists but is deliberately dropped at the channel boundary. toSemanticAnswer() (packages/harness-channel-sdk/src/toSemanticAnswer.ts:9–20) states verbatim: "Observability-only fields (runTrace, toolCalls, iterations) are dropped — connectors must not see them." So:

• web-ui renders full transparency (agent pills, tool trace, per-agent modal) by consuming the done event's runTrace directly.
• Teams / Telegram receive only the channel-agnostic SemanticAnswer (text + structured cards). They get no agent-invocation data at all. → This is exactly the user's blind spot.

There is a clean precedent for a runtime-asserted badge in the channel contract. SemanticAnswer already carries verifier?: VerifierBadge (packages/harness-channel-sdk/src/outgoing.ts:30/35/134), populated by the harness (not the LLM) in toSemanticAnswer (:86). Adding a curated agentsConsulted projection follows this exact pattern. The source data RunAgentInvocation { index, agentName, durationMs, subIterations, status, toolCalls } already exists (chatAgent.ts:154).

Building blocks for deterministic / forced routing already exist:

• LLM-free dispatch by id: deterministicActionRegistry.ts + invokeAgentTool(toolId, input) (kernel dispatches a tool directly, no LLM) — the precedent for harness-driven invocation with harness-controlled input.
• Stable agent identity + reverse lookup: stable agentId (e.g. de.byte5.agent.strategist), labelFromAgentId() (agents/resolveAgentForTool.ts), findAgentIdByToolName() and domainToolFor(agentId) (plugins/dynamicAgentRuntime.ts:214/626). A user-facing #strategist token can be resolved deterministically — scoped to the sub-agents whitelisted for this orchestrator (reuses OB-29-1 access gating).
• Forcing a tool call against LLM discretion: LocalSubAgent.ask()'s AskOptions.expectedTurnToolUse (localSubAgent.ts:41–60) already forces a tool via Anthropic tool_choice:{type:'tool',name} + a synthetic reminder + a bounded escalation budget (:282–314). This is OB-31; it is currently available for sub-agents but not at the orchestrator level.
• Deterministic harness wrapper around the LLM loop: chatStream() (orchestrator.ts:2382) wraps chatStreamInner() (:2523) and already intercepts and decorates the done event deterministically (privacy receipt, :2440/:2477). This is the natural home for the "switch".
• Deterministic privacy/PII masking runs at exactly that wrapper: privacyGuard().finalizeTurn(...) (:1755/:2399, package harness-plugin-privacy-guard).
• Durable per-conversation state: ChatSessionStore (scoped per orchestrator, buildOrchestrator.ts:167).

No existing issue covers this. The Conductor (Spec 005, PR #321) is a multi-step deterministic workflow engine — related but heavier and not merged. This issue delivers the per-turn trust primitives and the user-facing trust UX that the Conductor does not provide, and which are valuable on their own (see Relationship to the Conductor).

Proposed solution — three layers

The layers are independently shippable and increase in strength.

Layer 1 — Tamper-evident agent transparency in every channel (the minimum)

Project the deterministic runTrace.agentInvocations into a small, curated field on the channel contract and render it in every connector.

• Contract (packages/harness-channel-sdk/src/outgoing.ts): add

  interface AgentConsultation {
    agentId?: string;      // stable id when resolvable
    label: string;         // human label (labelFromAgentId)
    status: 'success' | 'error';
    durationMs?: number;
    toolCalls?: number;    // count only; never the orchestrator's prose
  }
  // SemanticAnswer.agentsConsulted?: readonly AgentConsultation[]

• Projection (toSemanticAnswer.ts): derive agentsConsulted from r.runTrace?.agentInvocations — a curated subset, not the raw runTrace (the "connectors must not see runTrace" boundary stays intact). Mirrors the existing verifier mapping at :86.
• Render: web-ui already has richer UI; Teams/Telegram render a compact footer — e.g. 🔎 Consulted: Strategist ✓ · 2 steps. The SDK provides a plain-text fallback so even a minimal connector appends a readable line.

Why this is tamper-evident: the footer is built by the harness from the choke-point trace, outside the LLM's output stream. If the orchestrator says "I asked the Strategist" but never invoked the tool, agentsConsulted is empty and the contradiction is visible. This fully satisfies the minimum criterion. It reveals suppression; Layers 2–3 prevent it.

Layer 2 — Direct Line: the harness-guaranteed delegation "switch" (core of the maximum)

A switch in the omadia core, not a bypass. The turn still enters and is owned by the orchestrator (scope, identity, memory, transport all intact). But when a user directs input at a named specialist, the harness — inside chatStream, deterministically, not the LLM — guarantees:

1. Faithful input. The harness binds the sub-agent's input to the user's verbatim payload (deterministic dispatch via the existing choke point / invokeAgentTool pattern), so the orchestrator cannot reshape the question.
2. Faithful, un-suppressible output. The sub-agent's verbatim answer is captured at the choke point and delivered to the user as a harness-owned, attributed segment on SemanticAnswer (new delegatedAnswer?: { agentId, label, text, status }), independent of the orchestrator's own text. The orchestrator can neither remove nor rewrite it.
3. Orchestrator stays aware. The verbatim exchange is recorded into the turn history / ChatSessionStore, so the orchestrator sees it (memory, recall, continuity) and may add an attributed note (see below) — but never a replacement.

Constructive vs. destructive — the switch removes only the destructive powers. The orchestrator is not the enemy here; it has a legitimate cross-cutting vantage point the specialist lacks. We constrain only its destructive capabilities — suppressing the delegation, hiding/removing the specialist's answer, rewriting or redacting its words, silently substituting its own process. Its constructive capability is preserved through a gated, additive channel:

• When the orchestrator judges that a specialist strayed outside its competence, or missed something only the orchestrator can see (cross-domain risk, policy, prior context), it may append an attributed note alongside the verbatim answer.
• Hurdles (so additive never becomes destructive): the note is additive-only — the specialist's segment stays byte-for-byte intact and is delivered independently by the harness; the note is clearly attributed and visually separated (e.g. ▸ omadia note: …); and it is recorded in the trace as an intervention (so the user sees that the orchestrator chose to add something, and why).
• Policy/user controls the level: strict passthrough (no orchestrator note at all — pure relay) vs. guarded additive (note allowed under the hurdles above). Either way the rule is invariant: the orchestrator may take note and add, never redact.

Trigger syntax (channel-native mention passes through to a core-parsed directive). Chat clients resolve @-mentions themselves, so a raw @Strategist would never reach the sub-agent. Instead the user addresses the omadia bot natively and names the specialist inside the payload. The # token below is illustrative only — the actual directive syntax is configurable; its sole hard requirement is that it survives the channel's own mention/markup parsing and arrives in the text omadia receives:

@omadia #strategist  What are three risks in plan A?
└─ channel mention   └─ in-payload directive parsed deterministically by omadia core
   (routes to bot)      (token is a placeholder; resolved to a whitelisted sub-agent; channel-agnostic)

The directive parser lives in core (the chatStream entry / dispatcher), not in each channel adapter — one implementation works for Teams, Telegram, web-ui alike. A Teams Adaptive-Card button ("Talk to Strategist directly"), surfaced on the Layer-1 footer, injects the same directive — no typing required. (Optional extension: a sticky "direct-line mode" persisted in ChatSessionStore for sustained multi-turn sparring, with a persistent on-screen indicator and a deterministic exit — see Open questions; default v1 is per-message directives.)

Layer 3 — Forced delegation obligation (the strongest in-process guarantee)

For orchestrator-driven turns where a consult must be guaranteed (not bypassed), port OB-31 to the orchestrator loop: a turn can carry an obligation "sub-agent X must be invoked". If the orchestrator tries to end the turn without it, the harness forces tool_choice:{type:'tool',name:<X's tool>} + a synthetic reminder, exactly as LocalSubAgent already does for sub-agents (localSubAgent.ts:282–314). Combined with the verifier/postcondition machinery, the runtime can assert "X was consulted" and block a turn that skips it.

Guarantee boundary (stated honestly): Layer 3 enforces that the consult happens and surfaces it (Layer 1), with the input still crafted by the orchestrator (its legitimate function). Full input-and-output faithfulness is the Direct Line's (Layer 2) job. The two are complementary, not redundant.

Direct Line — simulated turn flow

Canonical turn for @omadia #strategist <payload> (Teams):

1. Inbound → channel adapter forms IncomingTurn with the raw text (unchanged) → coreApi.handleTurnStream → orchestratorDispatcher → orchestrator.chatStream (turn is owned by the orchestrator, as today).
2. Directive parse (deterministic, in chatStream pre-step). Detect #<token>; resolve token → agentId against this orchestrator's whitelisted sub-agents. Strip the directive; the remainder is the verbatim payload.
3. Harness invocation. Call the resolved sub-agent through the existing choke point with the verbatim payload as input; capture verbatim output V; RunTraceCollector records it.
4. Faithful delivery. V is placed on SemanticAnswer.delegatedAnswer (attributed to the agent) and routed through privacyGuard().finalizeTurn(...) so PII masking still applies. agentsConsulted (Layer 1) is set.
5. Orchestrator awareness + guarded additive note. The (toolcall, V) exchange is written to turn history / ChatSessionStore. In strict passthrough the orchestrator's own visible generation is suppressed (no double-answer); it stays aware for the next real turn. In guarded additive it may append an attributed, separately-rendered note (▸ omadia note: …) — never editing or removing V. Which mode applies is a policy/user setting, not the orchestrator's choice.
6. Render. Teams shows the attributed Strategist answer + the consulted-footer; the orchestrator could not have removed or altered V.

Pitfalls & mitigations (from simulation)

#	Pitfall	Mitigation	Anchor
1	Forcing the call doesn't guarantee faithful input	Direct Line binds input to the verbatim user payload (harness-dispatched), not LLM-crafted	choke point :3164
2	Orchestrator omits/paraphrases the answer	Verbatim output delivered as harness-owned delegatedAnswer, independent of orchestrator text	toSemanticAnswer, outgoing.ts
3	Privacy/PII bypass — verbatim delivery skips masking	Route delegatedAnswer through the same privacyGuard().finalizeTurn	:1755/:2399; watch sub-agent bypass flag localSubAgent.ts:495/538
4	Double-answer noise (verbatim + orchestrator text)	Strict passthrough by default; any orchestrator addition is an additive, attributed note (▸ omadia note:) that never edits/removes the verbatim block	Layer 2 step 5
5	Keeping orchestrator "aware" without paying for a hidden LLM gen	Record exchange into history/ChatSessionStore; skip orchestrator generation in passthrough	chatSessionStore.ts
6	Mention pass-through — @strategist consumed by the client	In-payload directive (@omadia #strategist …) parsed in core, channel-agnostic	Layer 2 trigger
7	Agent resolution miss / ambiguity → wrong agent	Resolve only against whitelisted sub-agents; on miss/ambiguity surface ask_user_choice, never silently fall back	findAgentIdByToolName, OB-29-1
8	Mixed intent in one message	v1: directive delegates the entire remaining message; mixed orchestrator+direct unsupported (documented)	—
9	Sub-agent error/empty/timeout	Deliver faithful failure ("Strategist could not respond: …"), never a cover-up or hallucinated answer	postcondition / repeat-failure breaker
10	Sparring agent asks a clarifying question back	Verbatim question reaches the user; sticky mode routes the reply back automatically; per-message mode = re-issue directive	OB-31 escalation, sticky (optional)
11	Recursion / call-budget double-invoke	Count the harness invocation against the OB-29-1 budget; tell orchestrator "already consulted"; constrain tools in passthrough	OB-29-1 recursion guard
12	Sticky-mode state durability & exit (if enabled)	Persist in ChatSessionStore keyed by conversation; reserved exit directive; persistent mode indicator each turn	buildOrchestrator.ts:167
13	Channel parity (Teams pkg is external to this repo)	Extend in-repo SemanticAnswer + provide SDK plain-text fallback so every connector degrades gracefully	harness-channel-sdk
14	Verifier semantics on a verbatim block	Verifier badge attaches to the orchestrator's own claims only; the delegated block is labelled "attributed to , not orchestrator-verified"	VerifierBadge
15	Proactive/scheduled messages during a sticky session	Routines run in their own scope, unaffected by direct-line state	routines/proactiveSender

Implementation plan (in-repo seams)

Step	Files	Change
L1 contract	packages/harness-channel-sdk/src/outgoing.ts	add AgentConsultation + SemanticAnswer.agentsConsulted (and delegatedAnswer for L2)
L1 projection	packages/harness-channel-sdk/src/toSemanticAnswer.ts	map runTrace.agentInvocations → agentsConsulted (curated)
L1 render	web-ui chat; SDK text fallback; (Teams pkg parallel change — external repo)	footer + plain-text fallback
L2 directive parser	orchestrator.ts chatStream pre-step (or orchestratorDispatcher)	parse #<agent>, resolve to whitelisted sub-agent, strip
L2 harness invocation + delivery	orchestrator.ts choke point + chatStream done-decoration	verbatim dispatch + delegatedAnswer + finalizeTurn routing
L2 button	Teams pkg (external)	Adaptive-Card action injecting the directive token
L3 obligation	orchestrator.ts loop	accept expectedDomainTool, force tool_choice (port OB-31)

Acceptance criteria

• In Teams, every turn that invoked ≥1 sub-agent shows a consulted-footer sourced from runTrace (not the LLM). A fabricated "I consulted X" with no real invocation shows no footer entry.
• A direct-line directive (placeholder @omadia #strategist <payload>) delivers the Strategist's verbatim answer, attributed, in Teams; the orchestrator cannot suppress or reword it.
• In guarded additive mode the orchestrator can append an attributed, visually separated note, but the verbatim block stays byte-for-byte intact and independently delivered (test the no-redaction invariant).
• The verbatim delegated answer is still PII-masked by the privacy guard (Pitfall 3 covered by a test).
• An unknown/ambiguous #token yields a disambiguation/"no such agent" response, never a silent wrong-agent route.
• A sub-agent error is delivered faithfully, not papered over.
• (L3) A turn carrying an obligation cannot end without invoking the required sub-agent; verified by test.
• No regression to orchestrator memory, recall, privacy receipts, follow-ups, or scheduling.

Relationship to the Conductor (Spec 005 / #321)

Complementary, not overlapping:

• This issue = per-turn trust primitives (tamper-evident transparency, harness-guaranteed Direct Line, forced-delegation obligation) + the user-facing trust UX in channels. Ships independently of and faster than the Conductor; solves the acute ad-hoc-delegation pain now.
• The Conductor = composing Layer-3 obligations into multi-step, durable, defined processes (the eventual home for a formalized "strategy process" with human steps and roles). The Conductor reuses Layer 3 as its step-enforcement atom.

Open questions

1. Sticky direct-line mode: ship in v1 (better for multi-turn sparring; needs durable state + indicator + exit) or defer to a follow-up and start with per-message directives only?
2. Additive-intervention policy: should guarded additive be the default, or strict passthrough? What are the right hurdles for the orchestrator to add an attributed note (always-allowed-but-marked, confidence-gated, or only when it can cite a concrete cross-domain/policy reason)? The invariant — additive-only, attributed, traced, never redacting — holds regardless; the question is the gate.
3. Directive syntax: the #<agent> token is a placeholder — pick a final syntax that reliably survives each channel's mention/markup parsing; define collision rules with literal occurrences in user text.
4. Verifier treatment of the delegated block (Pitfall 14) — label only, or a lightweight independent check?

[Weegy]

Status: implemented + shipped

Core (this repo) — merged in #335. All three layers landed:

• L1 Transparency — SemanticAnswer.agentsConsulted, a curated projection of the deterministic runTrace.agentInvocations (harness-built, not LLM prose), with a plain-text footer fallback so every channel can surface it.
• L2 Direct Line — a core-parsed #<specialist> directive binds the sub-agent's input to the user's verbatim payload via the deterministic choke point and delivers its verbatim answer as a harness-owned delegatedAnswer the orchestrator can neither suppress nor reword (strict passthrough by default; guarded-additive note degrades to strict under an active privacy guard). Wired into both the non-streaming (chat()/Teams) and streaming (chatStream/web-ui) paths.
• L3 Forced delegation — ChatTurnInput.expectedDomainTool ports OB-31 to the orchestrator loop (forced tool_choice + synthetic reminder).

Collision rule: an unknown #token falls through to a normal LLM turn (no hijack of ordinary #… messages); ambiguous tokens disambiguate. Direct-line turns are persisted via the session logger (KG continuity). Independently reviewed with Codex/GPT-5.4.

MS Teams connector — @omadia/channel-teams@0.11.0, published to hub.omadia.ai. Renders the L1 footer chip, the L2 attributed-answer chip, and a dynamic per-agent "Direkt mit <Agent>" button (token derived per agent, never hardcoded) whose click routes straight to the named specialist. Source + review (4 Codex rounds, GO 96/100) in the private connector repo: byte5ai/omadia-byte5-plugins#19.

Open / follow-up: web-ui curated render of agentsConsulted/delegatedAnswer; enrich the run trace with stable agentId (Teams token currently derives from the label); a privacy-aware guarded-additive note; the verifier/postcondition layer for an absolute (vs. one-escalation) L3 guarantee.