Skip to content

Main w8a8 fp8#2

Open
HandH1998 wants to merge 259 commits intoyychfrom
main_w8a8_fp8
Open

Main w8a8 fp8#2
HandH1998 wants to merge 259 commits intoyychfrom
main_w8a8_fp8

Conversation

@HandH1998
Copy link
Copy Markdown
Owner

@HandH1998 HandH1998 commented Jan 21, 2025

Motivation

Modifications

Checklist

  • Format your code according to the Contributor Guide.
  • Add unit tests as outlined in the Contributor Guide.
  • Update documentation as needed, including docstrings or example tutorials.

XiaotongJiang and others added 30 commits January 9, 2025 13:43
@XiaotongJiang
[Doc]: Deepseek reference docs (sgl-project#2787)
11fffbc
@zhaochenyang20
Doc: add block-wise FP8 in dpsk model reference (sgl-project#2830)
5cc1170
@merrymercy @heiner
Update README.md (sgl-project#2833)
2db03a0 
Co-authored-by: Heiner <heiner@x.ai>
@mutinifni
Add more metrics to serving benchmark. (sgl-project#2819)
8f15789
[Bugfix] Fix embedding model hangs with --enable-metrics (sgl-proje…
f290bd4 
…ct#2822)
@Muqi1029
[Bugfix] Fix bug in fork logic caused by null text_ (sgl-project#2835)
5413ec2
@gty111
Fix port number overflow (sgl-project#2826)
b170646
@coolhok
[Eagle2] Fix multiple concurrent request crashes (sgl-project#2730)
a47bf39
@xiezhq-hermann
Cache controller for hierarchical caching (sgl-project#2804)
5d6e946
@merrymercy
Update threshold in test_nightly_gsm8k_eval.py (sgl-project#2836)
f176958
@BBuf
[HotFix] fix fp8 scale load failed in tp>1 (sgl-project#2837)
f0e15dc
@zhyncs
chore: bump v0.4.1.post5 (sgl-project#2840)
f624901
@zhyncs
docs: update README (sgl-project#2841)
197cbf9
@shuaills
Improve: Token-In Token-Out Usage for RLHF (sgl-project#2843)
c4f9707
@BBuf
add sampling_scaling_penalties kernel (sgl-project#2846)
e2b16c4
@zhyncs
fix sgl-kernel build (sgl-project#2850)
a879c2f
@ispobock
Add int8 quant kernel (sgl-project#2848)
85b2e05
@bjmsong
Support FP8 E4M3 KV Cache (sgl-project#2786)
0bb0f76 
Co-authored-by: root <bjmsong@126.com>
@sogalin @HaiShaw
Update base image for ROCm (sgl-project#2852)
a18ab81 
Co-authored-by: HAI <hixiao@gmail.com>
@kkHuang-amd @sogalin
Integrate ROCm ater package for ck moe function feasibility (sgl-proj…
e808c1d 
…ect#2854)

Co-authored-by: wunhuang <wunhuang@amd.com>
Co-authored-by: Lin, Soga <soga.lin@amd.com>
@coolhok
[Fix]eagle2 health_generate is first request,apiserver will core (sgl…
4093aa4 
…-project#2853)
@merrymercy @rkooo567
Fix linear.py and improve weight loading (sgl-project#2851)
72c7776 
Co-authored-by: SangBin Cho <rkooo567@gmail.com>
@kkHuang-amd @sogalin
Unify sglang coding style (sgl-project#2856)
42f3909 
Co-authored-by: Lin, Soga <soga.lin@amd.com>
@zhyncs
fix: not delete CNAME (sgl-project#2860)
20a9f5d
@zhyncs
docs: update link (sgl-project#2857)
41d7e5b
@zhyncs
minor: use ubuntu-latest instead of self-hosted runner for amd build (s…
4536d72 
…gl-project#2861)
@merrymercy
Use only one GPU for MLA CI tests (sgl-project#2858)
67008f4
@merrymercy
Collect more metrics: num_requests_total (sgl-project#2859)
51ab3cc
@bjmsong
Integration of TurboMind AWQ (sgl-project#2828)
17de02f 
Co-authored-by: root <bjmsong@126.com>
@ispobock
Fix quant kernel accuracy issue (sgl-project#2865)
f3516c2
HandH1998 and others added 20 commits January 24, 2025 09:47
@HandH1998
Merge remote-tracking branch 'origin/main' into main_w8a8_fp8
4a98c75
@HandH1998
fix bug
8c3dc13
@zhyncs
minor: sync flashinfer and add turbomind as 3rdparty (sgl-project#3105)
153b414
@trevor-m
Allow local cutlass directory to be used in sgl-kernel build (sgl-pro…
685a573 
…ject#3037)
@adarshxs
[Docs] minor update for phi-3 and phi-4 (sgl-project#3096)
4505a43
@zhyncs
minor: update sgl-kernel setup (sgl-project#3107)
04f0b4c
@ispobock
Add workflow for sgl-kernel cu118 release (sgl-project#3109)
a22f60a
@ispobock
Add step to update sgl-kernel whl index (sgl-project#3110)
665e5e8
@BBuf
support fp32 in sampling_scaling_penalties kernel (sgl-project#3121)
5d9d15e
@yizhang2077
mirror fix for custom allreduce (sgl-project#3124)
9852214
@zhyncs
chore: bump v0.0.2.post17 for sgl-kernel (sgl-project#3125)
14e754a
@zhyncs
speedup pr test for sgl-kernel (sgl-project#3126)
3cab5f7
@ispobock
Update tag name for whl release (sgl-project#3127)
67ad433
@ispobock
Update whl index path (sgl-project#3128)
c23d570
@zhyncs
update installation doc for sgl-kernel (sgl-project#3129)
896c074
@FlamingoPg @zhyncs
feat: refactor sgl-kernel and use TORCH_LIBRARY instead of PYBIND11_M…
9286740 
…ODULE for custom ops (sgl-project#3130)

Co-authored-by: yinfan.1024 <yinfan.1024@bytedance.com>
Co-authored-by: yinfan98 <1106110035@qq.com>
Co-authored-by: Yineng Zhang <me@zhyncs.com>
@merrymercy
Fix CI tests (sgl-project#3132)
da6f808
@merrymercy
Use torch.compile for scaling penalty (sgl-project#3133)
27acf63
@hliuca
enable kv_scale for Gemma2 (sgl-project#3113)
8e48ca8
@HandH1998
Merge remote-tracking branch 'origin/main' into main_w8a8_fp8
a1b582e
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

No reviews

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.