TensorRT Edge-LLM 0.5.0 Release

[nvluxiaoz]

TensorRT Edge-LLM 0.5.0 Release 2026-02-19

We are very excited to announce the second release of TensorRT Edge-LLM! We welcome community contribution and have merged the first community PR #13 @taoz27) in this release. Our developer roadmap for H1 2026 is listed in #32.

Breaking Changes

• Due to the standalone embedding processing module features, ONNX models exported by previous versions are not compatible with 0.5.0.

Key Features

• Implemented and used standalone embedding processing module to reduce multi-modal modeling complexity and reduce Eagle inference memory footprint
• Added FP8 KV Cache support
• Unified TensorRT execution context for prefill and decode to reduce memory footprint
• Supported vanilla decoding for speculative decoding runtime
• Used collision resistant hashing for CUDA graphs
• Updated int4GroupwiseGemmPlugin to TensorRT Plugin-v3 interface
• Refactored documentations
• Added ViT attention mask and RoPE parameter caching to reduce recomputation for Qwen (Merged feat: Cache ViT attention mask and RoPE to reduce recomputation for Qwen ViT #13 by @taoz27)
• Added Jetpack 6.2 compatibility

NVIDIA Contributors

@fans-nv @nvluxiaoz @nvamberl @fkaster-nvidia @chemudupatiks @poweiw @charllll @mahu888 @xinrzhang-star @byte-deve @JCalafato @nvmbreughe

---

This discussion was created from the release v0.5.0.