Skip to content

Expose model cost rates in the models contract (AgentModelRow) #5

@alexanderkreidich

Description

@alexanderkreidich

Split out of appx-org/agent-client#2 (per-session usage/cost metrics), implemented in appx-org/agent-client#4.

AssistantMessage.usage.cost is often zero for custom LiteLLM-routed models, so consumers recalculate cost from per-million-token rates. AgentModelRow (GET /v1/sessions/models) and the model row embedded in GET .../sessions/{id}/settings expose contextWindow but no cost rates, which forces every consumer to maintain its own rates map (create-appx-app currently uses LITELLM_COST_*_PER_MILLION env vars plus a hard-coded gpt-5.5 default).

Proposal: add an optional cost object to AgentModelRow:

cost?: {
  input: number;      // USD per 1M input tokens
  output: number;     // USD per 1M output tokens
  cacheRead: number;  // USD per 1M cache-read tokens
  cacheWrite: number; // USD per 1M cache-write tokens
}

pi's models.json already carries per-model cost for built-in providers, so for those this is just a passthrough; custom providers could accept the same shape in CustomProviderModel. Consumers would then drop their env-based rates maps and pass model.cost straight into agent-client's aggregateSessionUsage as costRates.

🤖 Generated with Claude Code

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type
    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions