Skip to content

feat: Migrate from llamastack to OGX 0.5.2#4

Open
gurvinder wants to merge 1 commit into
rh-ai-quickstart:mainfrom
gurvinder:main
Open

feat: Migrate from llamastack to OGX 0.5.2#4
gurvinder wants to merge 1 commit into
rh-ai-quickstart:mainfrom
gurvinder:main

Conversation

@gurvinder

Copy link
Copy Markdown

Migrate from llamastack to OGX 0.5.2

The llamastack/distribution-remote-vllm Docker image was archived as part of the Llama Stack → OGX rename. This PR migrates the entire deployment to use ogxai/distribution-starter:0.5.2.

Changes

Helm Charts

  • Updated llama-stack image from llamastack/distribution-remote-vllm:0.2.12 to ogxai/distribution-starter:0.5.2
  • Rewrote ConfigMap to use the 0.5.2 config schema (centralized storage section, registered_resources, new provider config format, removed deprecated telemetry API)
  • Replaced --yaml-config CLI args with RUN_CONFIG_PATH and CLI_NAME=llama env vars (new entrypoint format)
  • Added VLLM_API_TOKEN env var passthrough for LiteLLM-proxied endpoints
  • Fixed port mismatch: react-agent now connects to http://llama-stack:80 (matching the K8s service port)

React Agent (Streamlit UI)

  • Upgraded base image from ubi9/python-311 to ubi9/python-312 (required by llama-stack-client>=0.5.0)
  • Pinned client SDK to llama-stack-client>=0.5.0,<0.6.0 (0.6.0+ removes the agents API)
  • Updated model listing to use OpenAI-compatible model.id (replaces model.identifier / model.model_type)
  • Updated MCP tool format from string ("mcp::hr-api-tools") to dict ({"type": "mcp", "server_label": ..., "server_url": ...})
  • Removed unsupported response_format and sampling_params kwargs from ReActAgent
  • Fixed streaming event parsing to use 0.5.x structure (chunk.event.delta.text / chunk.event.final_text)
  • Display only the thought field from the agent's JSON response

Other

  • Updated react-agent image repository to quay.io/rh-ai-quickstart/llama-stack-react
  • Filters out safety/guard models when selecting the inference model

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant