A4 & A4X TRTLLM GKE single-host inference benchmarking recipes#146

Merged

hmhv1222 merged 8 commits intomainfrom

mhvictorhau-20260303-a4-a4x-trtllm-benchmarking

Apr 21, 2026

Commits on Apr 21, 2026

A4 & A4X TRTLLM GKE Single-Host Inference Recipes, ReadMe and Config Files
hmhv1222
committed
Add benchmarking configs warning paragraphs in TRTLLM inference ReadMe
hmhv1222
committed
Remove memory resources constraint in A4X serving-launcher.yaml and change default number of GPUs in A4 TRTLLM inference recipe
hmhv1222
committed
Qwen3 235B A22B, Qwen 2.5 VL 7B & Llama 3.1 405B config files and launcher modification
hmhv1222
committed
Support running VL models with trtllm-launcher.sh
hmhv1222
committed
Add/modify A4 & A4X Qwen2.5-VL-7B, Qwen3-235B-A22B, Llama3.1-405B, TRTLLM launcher, TRTLLM values YAML files for A4 & A4X
hmhv1222
committed
Update model lists for TRTLLM inference on A4 & A4X in main ReadMe and individual A4/A4X ReadMes
hmhv1222
committed
Use all 8 GPUs for A4 TRTLLM DeepSeek-R1 recipe; Adjust values.yaml
hmhv1222
committed