feat: KV cache compaction via attention matching#4

Open

fabiantax wants to merge 107 commits into

feat/kv-compaction

Commits on Mar 4, 2026

Commits on Mar 5, 2026

opencl: add `SET`, support i32 for `CPY`, minor refactor for cpy (#20101)
lhez
authored
hexagon: Flash Attention optimizations (dma, mpyacc, multi-row) and MatMul updates (#20118)

max-krasnyansky
and
chraac
authored
chore : correct typos [no ci] (#20041)

authored
webui: Improvements for Models Selector UI (#20066)
allozaur
authored
convert : register Qwen 3.5 ForCausalLM for text only (#20119)
CISC
authored
cli : add command and file auto-completion (#19985)
CISC
authored
model : update Qwen3.5 model type detection (#20126)

EZForever
and
CISC
authored
CUDA: Improve performance via less synchronizations between token (#17795)

aendk
and
ggerganov
authored
models : kda chunk size = 16 (#19827)

ymcki
and
ggerganov
authored

Commits on Mar 6, 2026

Commits on Mar 7, 2026

Commits on Mar 8, 2026

Commits on Mar 9, 2026