Deepseek V4 RL Roadmap

[yueming-yuan]

Miles PR: #1045

Megatron for Miles PR: #28

Usage

Docker:

H200/B200 (cu129 x86):

docker pull radixark/miles:deepseek-v4

GB300 (cu130 arm64):
Coming soon

Single-node, layer-pruned model smoke test:

This run will not generate readable output - just for the minimal check of the infrastructure.

docker run -it --name flash-smoke --gpus all --privileged --network=host --ipc=host --shm-size=16g --ulimit memlock=-1 radixark/miles:deepseek-v4 bash
# in docker run
rm -rf /root/miles && git clone --depth 1 --branch deepseek-v4 https://github.com/yueming-yuan/miles.git /root/miles && cd /root/miles && git pull --ff-only origin deepseek-v4 && pip install -e . --no-deps
rm -rf /root/sglang && git clone --depth 1 --branch deepseek_v4 https://github.com/sgl-project/sglang.git /root/sglang && cd /root/sglang && git pull --ff-only origin deepseek_v4 && pip install -e python --no-deps
cd /root/miles && python scripts/run_deepseek_v4.py full-train --model-name DeepSeek-V4-Flash-FP8-4layer --num-nodes 1 --num-gpus-per-node 8

DeepSeek-V4-Flash (284B)

This command may differ, depending on your cluster settings. Check and configure in run_deepseek_v4.py for full training configs, parallel settings, and RL features.
By default, it requires 8 H200 nodes with 8 GPUs per node, shared storage for /root/models and /root/datasets, working NCCL/Gloo networking, and either the script-managed local Ray head or an existing external Ray cluster enabled via MILES_SCRIPT_EXTERNAL_RAY=1 and RAY_ADDRESS.

cd /root/miles
python scripts/run_deepseek_v4.py full-train \
    --model-name DeepSeek-V4-Flash-FP8 \
    --num-nodes 8 \
    --num-gpus-per-node 8

DeepSeek-V4-Pro (1.6T)

TODO, ETA Apr 29/30

Verified Model Sizes

• DeepSeek-V4-Flash 284B
• DeepSeek-V4-Pro 1.6T

Low Precisions

• FP8 Training
• FP4 QAT training recipe
  • MXFP4/NVFP4 support
  • FP4 QAT

Kernels

@Zhichenzzz

• Integration of TileKernels
  • mHC forward & backward
• Compressor kernels

Features / Optimizations

• Anticipatory Routing
• Ring-style CSA Context Parallel

[jqwang2373]

Is the training receipt for DeepSeek-v4 available now?

[yuleiqin]

Request for the v4 training recipe....

[yueming-yuan]

@jqwang2373 @yuleiqin Thanks for your interest! Added the recipe for the smoke test and the 285B Flash model. 1.6T model's recipe is coming soon!

[boxiaowave]

Great job.

Can we still convert the saved checkpoint to fp8 with convert_hf_to_fp8 script and deploy it with sglang?

[boxiaowave]

Try to switch to sft mode, meets the following error:
AssertionError: TorchMemorySaver observes invalid LD_PRELOAD. You can use configure_subprocess() utility, or directly specify `LD_PRELOAD=/path/t
o/torch_memory_saver_cpp.some-postfix.so python your_script.py. (LD_PRELOAD="" process_id=786168)

[yushengsu-thu]

@Shi-Dong @yueming-yuan
Docker img GB300 (dsk-v4-flash)

docker pull radixark/miles:gb300-dev-dskv4

miles_artifact.py - have a PR update here: yueming-yuan#2

#!/usr/bin/env python3
# /// script
# dependencies = ["typer"]
# ///
"""Miles artifact CLI — build Miles Docker images at pinned commits.

Layout philosophy:
    Part 1 = NightFall sglang preset (verbatim from NightFall-nda/preset.py
             `flavor_sglang_sunrise`, H200/B200 branch).
    Part 2 = Miles-specific layers stacked on top (from .rcli/preset.py
             `flavor_sunrise_miles`).
    Part 3 = COPY pinned source trees + editable installs.
    Part 4 = DeepGEMM (the NightFall preset's setup-time step, placed last
             because install.sh pulls tvm-ffi 0.1.10 which we then repin).

Two environments: H200 (x86 cu129, the focus) and GB300 (arm64 cu130, stub).

Usage:
    uv run miles_artifact.py build-h200
    uv run miles_artifact.py build-h200 --tag myrepo/miles:test --push
    uv run miles_artifact.py build-gb300      # stub
"""

import os
import random
import subprocess
import time
from pathlib import Path

import typer

# ================================ Pinned commits ================================
SGLANG_COMMIT = "7f58083587cda2c8bfa490abda9357ee056d1ba1"    # sgl-project/sglang@deepseek_v4
MILES_COMMIT = "e561465d0b9bbf06188b7a5e2020dc7fd691f732"     # yueming-yuan/miles@deepseek-v4
MEGATRON_COMMIT = "8455dbfefd4740c944fb1876d4a6599dc9622cbc"  # yueming-yuan/Megatron-LM@deepseek-v4

REPO_SGLANG = "sgl-project/sglang"
REPO_MILES = "yueming-yuan/miles"
REPO_MEGATRON = "yueming-yuan/Megatron-LM"

# ====================== Third-party pins (miles-only layer) =====================
MILES_WHEELS_URL = "https://github.com/yueming-yuan/miles-wheels/releases/download/cu129-x86_64"
TORCH_MEMORY_SAVER_COMMIT = "d64a6394d1e09c613fab90260054cecc2684586d"
MBRIDGE_COMMIT = "89eb10887887bc74853f89a4de258c0702932a1c"
TRANSFORMERS_COMMIT = "8cb5963cc22174954e7dca2c0a3320b7dc2f4edc"
NCCL_TESTS_COMMIT = "ae98985f5599617be94042f4aa3637d10014ce89"

# ====================== Third-party pins (NightFall preset) =====================
# Source: DarkSharpness/NightFall origin/rcli-config preset.py flavor_sglang_sunrise
DEEPGEMM_COMMIT = "7f2a70"
FLASHINFER_JIT_CACHE_VER = "0.6.8"
# tilelang 0.1.8: sglang mhc.py calls T.gemm(wg_wait=0), which was removed from the
# public T.gemm signature in 0.1.9 (moved to T.wgmma_gemm). Must pin 0.1.8.
TILELANG_VER = "0.1.8"

GITHUB_TOKEN = os.environ.get("GITHUB_TOKEN", "")

app = typer.Typer(help="Build Miles Docker images at pinned commits")


# ================================== Dockerfiles =================================

DOCKERFILE_H200 = f"""\
FROM lmsysorg/sglang:v0.5.7

WORKDIR /root/

# ================================================================================
# PART 1 — NightFall sglang preset (flavor_sglang_sunrise, H200/B200)
# Source: NightFall-nda/preset.py
# ================================================================================

# --- preset build_commands ---
RUN pip install tilelang=={TILELANG_VER}
RUN pip install flashinfer-jit-cache=={FLASHINFER_JIT_CACHE_VER} --index-url https://flashinfer.ai/whl/cu129
RUN cd /tmp && git clone https://github.com/deepseek-ai/FlashMLA.git flash-mla && \\
    cd flash-mla && git submodule update --init --recursive && \\
    pip install --no-build-isolation -v . && \\
    cd /tmp && rm -rf flash-mla

# preset setup_command `pip install -e sglang/python/` is deferred to Part 3
# (after COPY). DeepGEMM step is deferred to Part 4 (last — tvm-ffi repin).

# ================================================================================
# PART 2 — Miles-specific additions (on top of the sglang preset)
# Source: sunrise/.rcli/preset.py flavor_sunrise_miles
# ================================================================================

# --- apt: miles needs ethtool + nccl-tests binaries that NightFall preset skips ---
RUN apt update && apt install -y nvtop rsync dnsutils ethtool
RUN apt remove -y libgtest-dev || true

RUN git clone https://github.com/NVIDIA/nccl-tests.git /tmp/nccl-tests && \\
    cd /tmp/nccl-tests && git checkout {NCCL_TESTS_COMMIT} && \\
    make -j$(nproc) CUDA_HOME=/usr/local/cuda && \\
    cp /tmp/nccl-tests/build/*_perf /usr/local/bin/ && \\
    rm -rf /tmp/nccl-tests

# --- prebuilt wheels (flash_attn, FA3, apex) — rebuilding takes 30+ min, reuse ---
RUN mkdir -p /tmp/wheels && cd /tmp/wheels && \\
    curl -fSL -O {MILES_WHEELS_URL}/flash_attn-2.7.4.post1-cp312-cp312-linux_x86_64.whl && \\
    curl -fSL -O {MILES_WHEELS_URL}/flash_attn_3-3.0.0b1-cp39-abi3-linux_x86_64.whl && \\
    curl -fSL -O {MILES_WHEELS_URL}/apex-0.1-cp312-cp312-linux_x86_64.whl && \\
    pip install --no-deps /tmp/wheels/flash_attn-*.whl /tmp/wheels/flash_attn_3-*.whl /tmp/wheels/apex-*.whl && \\
    rm -rf /tmp/wheels
# FA3 wheel installs flash_attn_interface.py at site-packages root;
# TE 2.10 imports it as flash_attn_3.flash_attn_interface — mirror it.
RUN python -c "import site, shutil; sp = site.getsitepackages()[0]; \\
shutil.copy(f'{{sp}}/flash_attn_interface.py', f'{{sp}}/flash_attn_3/flash_attn_interface.py')"

# --- fast-hadamard-transform: used by Miles DSv4 rotate_activation plugin ---
# Install non-editable so the package lives in site-packages, not pinned to
# /tmp source. /tmp must remain ephemeral — any pod that mounts it (emptyDir,
# tmpfs, ephemeral-storage volume) would otherwise shadow the source and
# break the egg-link at runtime.
RUN cd /tmp && git clone https://github.com/Dao-AILab/fast-hadamard-transform.git && \\
    cd fast-hadamard-transform && pip install . -v --no-build-isolation && \\
    cd / && rm -rf /tmp/fast-hadamard-transform

# --- TE + Miles numerics stack ---
RUN pip install nvidia-mathdx==25.6.0
RUN pip install --no-build-isolation transformer_engine[core_cu12,pytorch]==2.10.0
RUN pip install --no-deps git+https://github.com/ISEEKYAN/mbridge.git@{MBRIDGE_COMMIT}
RUN pip install flash-linear-attention==0.4.0
# torch_memory_saver @d64a639 (NOT @dc68769 — the newer commit regresses on B200
# with tms_get_interesting_region AssertionError in MegatronTrainRayActor init).
RUN pip install --no-cache-dir --force-reinstall \\
    git+https://github.com/fzyzcjy/torch_memory_saver.git@{TORCH_MEMORY_SAVER_COMMIT}
RUN pip install --no-build-isolation git+https://github.com/fzyzcjy/Megatron-Bridge.git@dev_rl
RUN pip install --no-build-isolation "nvidia-modelopt[torch]>=0.37.0"
RUN pip install "numpy<2"

# tvm-ffi 0.1.9 — tilelang compat; DeepGEMM's install.sh will upgrade to 0.1.10,
# we re-pin in Part 4.
RUN pip install apache-tvm-ffi==0.1.9

# cudnn pin (pytorch/pytorch#168167)
RUN pip install nvidia-cudnn-cu12==9.16.0.29

# ================================================================================
# PART 3 — Pinned source trees + editable installs
# ================================================================================

RUN pip uninstall -y miles megatron sglang transformers deep-gemm deep_gemm || true
RUN rm -rf /sgl-workspace/sglang /root/miles /root/Megatron-LM

COPY sglang /root/sglang
COPY megatron /root/Megatron-LM
COPY miles /root/miles

# transformers + miles deepseekv32 patch (must come before miles install).
# Non-editable install: the patched bits get baked into site-packages, source
# dir under /tmp is then disposable. Keeping it editable would tie the install
# to /tmp's lifetime — see fast-hadamard-transform note above for why that's
# brittle.
RUN cd /tmp && git clone https://github.com/huggingface/transformers.git && \\
    cd transformers && git checkout {TRANSFORMERS_COMMIT} && \\
    git apply /root/miles/docker/deepseekv32/transformers.patch && \\
    pip install . && \\
    cd / && rm -rf /tmp/transformers

# sglang without --no-deps so pyproject.toml pulls sgl-kernel==0.3.21,
# quack-kernels, cuda-python, nvidia-cutlass-dsl — the versions NightFall was
# built against.  This is the NightFall preset's `setup_command`.
RUN pip install -e /root/sglang/python

RUN pip install -r /root/miles/requirements.txt
RUN pip install -e /root/miles
RUN pip install --no-deps -e /root/Megatron-LM

# ================================================================================
# PART 4 — DeepGEMM (NightFall preset setup_command, deferred)
# install.sh pulls tvm-ffi 0.1.10; pin back to 0.1.9 for tilelang.
# ================================================================================

RUN cd /tmp && git clone https://github.com/sgl-project/DeepGEMM.git -b release && \\
    cd DeepGEMM && git checkout {DEEPGEMM_COMMIT} && \\
    git submodule update --init --recursive && bash install.sh
RUN pip install --no-deps apache-tvm-ffi==0.1.9

# ================================================================================
# Cleanup
# ================================================================================
# /tmp source dirs are removed inline above (each install does its own cleanup);
# nothing here installs editable into /tmp. Verify, then drop pip cache.
RUN set -eu; \\
    leftover=$(pip freeze 2>/dev/null | awk '/^-e \\/tmp\\// {{print $2}}'); \\
    if [ -n "$leftover" ]; then \\
      echo "FAIL: /tmp-rooted editable installs detected:"; echo "$leftover"; exit 1; \\
    fi; \\
    for pkg in $(pip list --format=freeze | grep -v '^-e ' | sed 's/[ =@].*//'); do \\
      [ -z "$pkg" ] && continue; \\
      loc=$(pip show "$pkg" 2>/dev/null | awk -F': ' '/^Editable project location:/{{print $2; exit}}'); \\
      case "$loc" in /tmp/*) echo "FAIL: $pkg still editable @ $loc"; exit 1 ;; esac; \\
    done; \\
    echo "OK — no /tmp-rooted editable installs."
RUN rm -rf /root/.cache/pip /tmp/DeepGEMM
"""

DOCKERFILE_GB300 = f"""\
# GB300 is a thin relayer on top of the prebuilt miles-sunrise image.
# Heavy deps (flash-attn, FA3, apex, TE[core_cu13], mbridge, flash-linear-attention,
# Megatron-Bridge, modelopt, fast-hadamard-transform, numpy) are already baked into
# yuemingy/miles-sunrise:gb300 (built from NightFall-nda artifact_cli.py
# DOCKERFILE_MILES_GB300). We upgrade what the base image ships stale:
#
#   Upgrade          | Base image state          | Target (NightFall GB300 preset)
#   -----------------|---------------------------|----------------------------------
#   cuda-python      | baseline                  | --upgrade
#   flashinfer-python| 0.6.2                     | {FLASHINFER_JIT_CACHE_VER}
#   flashinfer-cubin | 0.6.2                     | {FLASHINFER_JIT_CACHE_VER}
#   flashinfer-      | 0.6.2 --index-url cu129   | {FLASHINFER_JIT_CACHE_VER} --index-url cu130
#     jit-cache      |   (cu129 is a bug on cu130)|
#     (jit-cache, python, and cubin must match version; flashinfer's runtime
#      version check errors out if they differ — RuntimeError at JIT import time.)
#   DeepGEMM         | not installed             | @{DEEPGEMM_COMMIT} + cutlass symlinks + install.sh
#   torch_           | @dc6876905 (regresses on   | @{TORCH_MEMORY_SAVER_COMMIT[:7]}
#     memory_        |   B200: tms_get_            |
#     saver          |   interesting_region)      |
#
# Everything else (apt packages, TE, mbridge, etc.) stays as baked.
FROM yuemingy/miles-sunrise:gb300

WORKDIR /root/

# ---------- PART 1: NightFall sglang preset (GB300 branch) upgrades ----------
RUN pip install cuda-python --upgrade
# flashinfer {{python,cubin,jit-cache}} must all match 0.6.8 — else JIT import raises
# `RuntimeError: flashinfer-jit-cache version does not match flashinfer version`.
RUN pip install flashinfer-python=={FLASHINFER_JIT_CACHE_VER} flashinfer-cubin=={FLASHINFER_JIT_CACHE_VER}
RUN pip install flashinfer-jit-cache=={FLASHINFER_JIT_CACHE_VER} --index-url https://flashinfer.ai/whl/cu130
# DeepGEMM (base image lacks it; NightFall preset installs with cutlass symlinks).
RUN cd /tmp && git clone https://github.com/sgl-project/DeepGEMM.git -b release && \\
    cd DeepGEMM && git checkout {DEEPGEMM_COMMIT} && \\
    git submodule update --init --recursive && \\
    ln -sf $(pwd)/third-party/cutlass/include/cutlass $(pwd)/deep_gemm/include/cutlass && \\
    ln -sf $(pwd)/third-party/cutlass/include/cute $(pwd)/deep_gemm/include/cute && \\
    bash install.sh && \\
    cd / && rm -rf /tmp/DeepGEMM

# ---------- PART 1.5: Stabilize /tmp-rooted editable installs from the base ----------
# Base image (yuemingy/miles-sunrise:gb300, built upstream by NightFall-nda
# artifact_cli.py DOCKERFILE_MILES_GB300) ships some packages installed as
# `pip install -e /tmp/<pkg>` (e.g. fast-hadamard-transform). Any pod that
# mounts /tmp (emptyDir, tmpfs, ephemeral-storage volume) shadows the source
# and breaks the egg-link at import time.
# Fix: scan for /tmp editables, reinstall as non-editable so they live in
# site-packages. /tmp then carries no live state.
# Once the upstream NightFall builder is patched analogously, this layer
# becomes a no-op (just emits "No /tmp-rooted editable installs in base.").
RUN set -eu; \\
    : > /tmp/_tmp_editables.txt; \\
    for pkg in $(pip list --format=freeze | grep -v '^-e ' | sed 's/[ =@].*//' | sort -u); do \\
      [ -z "$pkg" ] && continue; \\
      loc=$(pip show "$pkg" 2>/dev/null | awk -F': ' '/^Editable project location:/{{print $2; exit}}'); \\
      case "$loc" in /tmp/*) echo "$loc" >> /tmp/_tmp_editables.txt ;; esac; \\
    done; \\
    pip freeze 2>/dev/null | awk '/^-e \\/tmp\\// {{print $2}}' >> /tmp/_tmp_editables.txt || true; \\
    sort -u /tmp/_tmp_editables.txt -o /tmp/_tmp_editables.txt; \\
    if [ -s /tmp/_tmp_editables.txt ]; then \\
      echo "=== /tmp-rooted editables in base image ==="; cat /tmp/_tmp_editables.txt; \\
      while read -r path; do \\
        [ -z "$path" ] && continue; \\
        echo ">>> Reinstalling $path as non-editable"; \\
        pip install --no-deps --force-reinstall --no-build-isolation "$path"; \\
      done < /tmp/_tmp_editables.txt; \\
      for pkg in $(pip list --format=freeze | grep -v '^-e ' | sed 's/[ =@].*//' | sort -u); do \\
        [ -z "$pkg" ] && continue; \\
        loc=$(pip show "$pkg" 2>/dev/null | awk -F': ' '/^Editable project location:/{{print $2; exit}}'); \\
        case "$loc" in /tmp/*) echo "FAIL: $pkg still editable @ $loc"; exit 1 ;; esac; \\
      done; \\
      echo "OK — no /tmp editables remain."; \\
    else \\
      echo "No /tmp-rooted editable installs in base."; \\
    fi; \\
    rm -f /tmp/_tmp_editables.txt

# ---------- PART 2: Miles-side upgrades ----------
# torch_memory_saver swap (see table above).
RUN pip install --no-cache-dir --force-reinstall \\
    git+https://github.com/fzyzcjy/torch_memory_saver.git@{TORCH_MEMORY_SAVER_COMMIT}

# ---------- PART 3: Pinned source trees + editable installs ----------
RUN pip uninstall -y miles megatron sglang || true
RUN rm -rf /sgl-workspace/sglang /root/miles /root/Megatron-LM

COPY sglang /root/sglang
COPY megatron /root/Megatron-LM
COPY miles /root/miles

# --no-deps: base image already resolves all cu130/arm64 deps (sgl-kernel,
# cuda-python, etc.). If pyproject pins drift, drop --no-deps on the affected line.
RUN pip install --no-deps -e /root/sglang/python
RUN pip install --no-deps -e /root/miles
RUN pip install --no-deps -e /root/Megatron-LM

RUN rm -rf /root/.cache/pip
"""


# =================================== Commands ===================================


@app.command("build-h200")
def build_h200(
    sglang_commit: str = typer.Option(SGLANG_COMMIT),
    miles_commit: str = typer.Option(MILES_COMMIT),
    megatron_commit: str = typer.Option(MEGATRON_COMMIT),
    tag: str | None = typer.Option(None, help="Override image tag."),
    push: bool = typer.Option(False, help="docker push after build."),
):
    """Build the H200 (x86, CUDA 12.9) Miles artifact."""
    if tag is None:
        tag = (
            f"radixark/miles-sunrise:h200-"
            f"sgl-{sglang_commit[:7]}-"
            f"miles-{miles_commit[:7]}-"
            f"mega-{megatron_commit[:7]}"
        )
    repos = [
        (REPO_SGLANG, "sglang", sglang_commit),
        (REPO_MILES, "miles", miles_commit),
        (REPO_MEGATRON, "megatron", megatron_commit),
    ]
    _build_image(DOCKERFILE_H200, repos, tag, push=push)


@app.command("build-gb300")
def build_gb300(
    sglang_commit: str = typer.Option(SGLANG_COMMIT),
    miles_commit: str = typer.Option(MILES_COMMIT),
    megatron_commit: str = typer.Option(MEGATRON_COMMIT),
    tag: str | None = typer.Option(None),
    push: bool = typer.Option(False),
):
    """Build the GB300 (arm64, CUDA 13) Miles artifact — thin relayer over yuemingy/miles-sunrise:gb300."""
    if tag is None:
        tag = (
            f"radixark/miles-sunrise:gb300-"
            f"sgl-{sglang_commit[:7]}-"
            f"miles-{miles_commit[:7]}-"
            f"mega-{megatron_commit[:7]}"
        )
    repos = [
        (REPO_SGLANG, "sglang", sglang_commit),
        (REPO_MILES, "miles", miles_commit),
        (REPO_MEGATRON, "megatron", megatron_commit),
    ]
    _build_image(DOCKERFILE_GB300, repos, tag, push=push)


# ==================================== Helpers ===================================


def _build_image(
    dockerfile_content: str,
    repos: list[tuple[str, str, str]],
    tag: str,
    push: bool,
) -> None:
    dir_tmp = Path(f"/tmp/miles-artifact-{int(time.time())}-{random.randint(1000, 9999)}")
    dir_tmp.mkdir(parents=True, exist_ok=True)
    typer.echo(f"Working directory: {dir_tmp}")

    (dir_tmp / "Dockerfile").write_text(dockerfile_content)
    typer.echo(f"Dockerfile written to {dir_tmp / 'Dockerfile'}")

    for repo_name, local_name, commit in repos:
        typer.echo(f"\n=== Cloning {repo_name} -> {local_name} @ {commit} ===")
        _clone_repo(repo_name, dir_tmp / local_name, commit)

    typer.echo(f"\n=== Building {tag} ===")
    _exec(f"cd {dir_tmp} && docker build . -t {tag}")

    if push:
        typer.echo(f"\n=== Pushing {tag} ===")
        _exec(f"docker push {tag}")

    typer.echo(f"\nDone! Image: {tag}")


def _clone_repo(repo_name: str, repo_dir: Path, commit: str) -> None:
    url = f"https://{GITHUB_TOKEN}@github.com/{repo_name}" if GITHUB_TOKEN else f"https://github.com/{repo_name}"
    log_url = f"https://github.com/{repo_name}"
    cmd = (
        f"rm -rf {repo_dir} && "
        f"git clone --recursive {url} {repo_dir} && "
        f"cd {repo_dir} && "
        f"git fetch --all && "
        f"git checkout {commit} && "
        f"git reset --hard {commit} && "
        f"find . -name .git -exec rm -rf {{}} + 2>/dev/null || true"
    )
    _exec(cmd, log=cmd.replace(url, log_url))


def _exec(cmd: str, log: str | None = None) -> None:
    typer.echo(f"$ {log or cmd}")
    subprocess.run(["bash", "-c", cmd], check=True)


if __name__ == "__main__":
    app()

[yueming-yuan]

Try to switch to sft mode, meets the following error: AssertionError: TorchMemorySaver observes invalid LD_PRELOAD. You can use configure_subprocess() utility, or directly specify `LD_PRELOAD=/path/t o/torch_memory_saver_cpp.some-postfix.so python your_script.py. (LD_PRELOAD="" process_id=786168)

Hi @boxiaowave , are you using the provided Docker image? Can you provide more information on the environment

[yiakwy-xpu-ml-framework-team]

@yueming-yuan model type patch should be reverted

yueming-yuan#3

[yiakwy-xpu-ml-framework-team]

@yushengsu-thu support of H800 (torch 2.8cu128/2.9.1cu129 ) with the latest SGLang can be referred to this sgl-project/sglang#24524, could we add it ?

For Hopper we should use this command:

cd /root/sglang && git fetch origin deepseek_v4_hopper && git checkout deepseek_v4_hopper && pip install -e python --no-deps

We can steadily reproduced FP8 inference on Hoppper platform, recommend main branch (do not use deepseek-v4) now:

[yiakwy-xpu-ml-framework-team]

Compressed Topk Implementation for Training:

huggingface/transformers#45837

cc @yushengsu-thu @yueming-yuan

[xs1997zju]

No module named 'tile_kernels', is this image "radixark/miles:deepseek-v4" correct? @yueming-yuan