Submission/qat bigram12k stride32#348

Open

EthanYangTW wants to merge 15 commits intoopenai:mainfrom

EthanYangTW:submission/qat-bigram12k-stride32

Commits on Mar 21, 2026

Commits on Mar 22, 2026

Add #315/#388 full stack: 11L, XSA4, Partial RoPE, LN Scale, EMA, Late QAT, TTT
EthanYangTW
committed
Fix speed and artifact size: disable FA3, reduce BigramHash, EMA every 10 steps
EthanYangTW
committed
Reduce BigramHash to 2048, increase pruning to 10% to fit under 16MB
EthanYangTW
committed
Remove BigramHash, increase pruning to 15% — must fit under 16MB
EthanYangTW
committed
New record: 11L XSA4 + Tight SWA + TTT (based on PR #374)
EthanYangTW
committed
Fix TTT: compile + DDP + 3 epochs + batch 64 for speed
EthanYangTW
committed
Fix FA3 import: add fallback to flash_attn and SDPA
EthanYangTW
committed