Enable Xeon support

[jgespino]

README improvements

• Added Xeon-based local deployment option (Option C) with clear example config
• Replaced curl example with port-forwarding

Helm configuration updates

• Updated dependency versions in Chart.yaml (llm-service)
• Updated rag-values.yaml.example with Xeon model config comments
• Updated llama-stack image tag (0.2.23 → 0.6.1)

[ganeshmurthy]

Do we need to add deploy/helm/rag/templates/route-llamastack.yaml LlamaStack route?

The architecture is:
User → Streamlit (exposed via route) → F5 XC Security → LlamaStack (internal only)

By exposing LlamaStack directly with an external route, we're creating a path that bypasses all F5 security controls (WAF, rate limiting, API spec enforcement) that this quickstart is designed to showcase.

The README currently has curl examples that assume this route exists (lines 187-193), but I believe those examples should be updated to use port-forwarding instead:

  # For local testing
  oc port-forward svc/llamastack 8321:8321
  curl -sS http://localhost:8321/v1/models

This allows developers to test LlamaStack without exposing it externally.

Is there a specific reason you are adding that route. Is it necessary to enable Xeon support ?

[jgespino]

@ganeshmurthy Thanks for the feedback! I initially added the route-llamastack to test the llama-stack using the README steps. Based on your suggestion, I’ve removed it and updated the README to use port-forwarding for testing instead.

[ganeshmurthy]

Thank you for removing the route-llamastack

To Enable Xeon support, is it necessary to change these versions -

dependencies:
  - name: pgvector
    version: 0.5.6
    repository: https://rh-ai-quickstart.github.io/ai-architecture-charts
  - name: llm-service
    version: 0.5.10
    repository: https://rh-ai-quickstart.github.io/ai-architecture-charts
  - name: llama-stack
    version: 0.8.7
    repository: https://rh-ai-quickstart.github.io/ai-architecture-charts

[jgespino]

Only the update to llm-service version 0.5.10 is required, as it includes the 3.4.0 image with Xeon support. I can revert the other components if needed.

[ganeshmurthy]

Can you please only update the llm-service version to 0.5.10 in this PR. As for the version changes to pgvector and llama-stack, please do raise a separate PR. This PR should only contain the changes that are necessary to Enable Xeon support

[jgespino]

Sure, I've made the updates.

[ganeshmurthy]

Sorry, this is my last comment

This line was added to the README:
| Generation| meta-llama/Llama-3.1-8B-Instruct | L4/HPU/XEON | g6.2xlarge |

But no corresponding Xeon configuration example was added to deploy/helm/rag-values.yaml.example.

The file only includes:

   # Example Xeon configurations:
   # llama-3-2-3b-instruct:
   #   id: meta-llama/Llama-3.2-3B-Instruct
   #   enabled: true
   #   device: "xeon"
   #   args:
   #   - --max-model-len
   #   - "14336"
   #   - --max-num-seqs
   #   - "32"

Should we either:

1. Add a llama-3-1-8b-instruct Xeon example to the values file, or
2. Remove XEON from the 8B model's hardware column in the README if it's not validated/recommended?

Users following the README will expect to find configuration examples for all advertised hardware options.

[jgespino]

Thanks for the feedback! I added llama-3-1-8b-instruct Xeon example to the values file.