- 资料整理和学习笔记
- Interstellar: 数据、算法、模型、硬件、架构
搜索了一下，目前市面上没有比较符合思路的整理，计划9月构建完本仓库roadmap, 好的资料/笔记欢迎提交 pr，可以私信我加aiwiki交流群(｀・ω・´)
- CS224n
- CS336: Language Modeling from Scratch (Stanford / Spring 2026)
- CSCI 1390, Spring 2025: Systems for Machine Learning
-
对齐微调: InstructGPT (RLHF)，Constitutional AI，Self-Instruct，Direct Preference Optimization (DPO)，ORPO，GRPO
- FlashAttention
- FlashAttention-2
- RoPE (Rotary Position Embeddings)
- ALiBi
- Multi-Query Attention (MQA)
- Grouped-Query Attention (GQA)
- Speculative Decoding
- Medusa: Simple LLM Inference Acceleration Framework with Multiple Decoding Heads
- Fast Inference from Transformers via Speculative Decoding
- Break the Sequential Dependency of LLM Inference Using Lookahead Decoding
- Accelerating Large Language Model Decoding with Speculative Sampling
- PagedAttention (vLLM)
- Efficient Memory Management for Large Language Model Serving with PagedAttention
- KV Cache Compression & Optimization
- Mooncake: A KVCache-centric Disaggregated Architecture for LLM Serving
- Splitwise: Efficient Generative LLM Inference Using Phase Splitting
- DualPath: Breaking the Storage Bandwidth Bottleneck in Agentic LLM Inference
- DistServe: Disaggregating Prefill and Decoding for Goodput-optimized Large Language Model Serving
- MemServe: Context Caching for Disaggregated LLM Serving with Elastic Memory Pool
- TetriInfer: Inference without Interference: Disaggregate LLM Inference for Mixed Downstream Workloads
- Retrieval-Augmented Generation (RAG)
- METIS: Fast Quality-Aware RAG Systems with Configuration Adaptation
- CacheBlend: Fast Large Language Model Serving for RAG with Cached Knowledge Fusion
- Parrot: Efficient Serving of LLM-based Applications with Semantic Variable
- Towards End-to-End Optimization of LLM-based Applications with Ayo
- Chain-of-Thought Prompting
- Tree of Thoughts
- ReAct
- DeepSpeed-FastGen: High-throughput Text Generation for LLMs
- SARATHI: Efficient LLM Inference by Piggybacking Decodes with Chunked Prefills
- Taming Throughput-Latency Tradeoff in LLM Inference with Sarathi-Serve
- ELI5: FlashAttention
- FlashAttention from First Principles
- Flash Attention 2.0 with Tri Dao (author)!
- FlashAttention学习过程【详】解
- FlashAttention — Visually and Exhaustively Explained
- Designing Hardware-Aware Algorithms: FlashAttention
- FlashAttention: Fast and Memory-Efficient Exact Attention With IO-Awareness
设计环境、规则、测试反馈系统，让 AI Agent 自动生成并改进代码
- Minions: Stripe’s one-shot, end-to-end coding agents—Part 2
- Effective harnesses for long-running agents
- Minions: Stripe’s one-shot, end-to-end coding agents
- Harness engineering: leveraging Codex in an agent-first world
- Vibe Coding AReaL：零手打代码开发分布式 RL 训练框架
- Deep Dive into Triton Internals (Part 3)
- Deep Dive into Triton Internals (Part 1)
- Deep Dive into Triton Internals (Part 2)
- Build a Large Language Model (From Scratch)
- AI Systems Performance Engineering：GPU CUDA Kernel 调优、PyTorch 算法优化、多节点训练推理系统调优...