Skip to content

[Bugfix] Allow fp8_e5m2 KV cache on W4A16 compressed-tensors models (V100/SM70)#49

Open
rivetphilbot wants to merge 44 commits into
1CatAI:mainfrom
rivetphilbot:p7-fp8-kv-ct-classifier-fix
Open

[Bugfix] Allow fp8_e5m2 KV cache on W4A16 compressed-tensors models (V100/SM70)#49
rivetphilbot wants to merge 44 commits into
1CatAI:mainfrom
rivetphilbot:p7-fp8-kv-ct-classifier-fix

P7: skip CT KVCacheMethod when kv_cache_scheme is None

9a7a7cd
Select commit
Loading
Failed to load commit list.

Workflow runs completed with no jobs