Add mooncake dockerfile build by ZhangLirong-amd · Pull Request #771 · ROCm/ATOM

ZhangLirong-amd · 2026-05-13T08:13:46Z

Summary

Add PD (prefill-decode) disaggregation benchmark to the nightly CI, running after single-node benchmarks complete
Support multi-node coordination: dynamic node discovery via SSH, with 1 prefill + N decode node configurations
Two model configs: TP4 prefill / TP8 decode (1 decode node) and TP8 prefill / TP8 decode (2 decode nodes)
Results display alongside single-node results in the benchmark dashboard under ATOM-PD backend

Technical Details

Test Plan

Test Result

Submission Checklist

Look over the contributing guidelines at https://github.com/ROCm/ROCm/blob/develop/CONTRIBUTING.md#pull-requests.

Copilot

Pull request overview

Adds Mooncake to the ATOM Docker multi-stage “parallel build” image so it can be built in its own stage and then merged into the final atom_image runtime.

Changes:

Extend the BuildKit parallel stage layout to include a new build_mooncake stage.
Add a build_mooncake stage that installs dependencies, builds Mooncake with HIP/etcd enabled, and installs its Python wheel.
Copy Mooncake runtime artifacts (libs, etcd binaries, Python package) into the final atom_image stage and validate import.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

+        openssh-server openmpi-bin openmpi-common libopenmpi-dev && \
+    rm -rf /var/lib/apt/lists/*
+
+RUN apt-get update && apt-get remove -y golang golang-go 2>/dev/null || true && \


+    wget -q https://go.dev/dl/go1.22.2.linux-amd64.tar.gz && \
+    tar -C /usr/local -xzf go1.22.2.linux-amd64.tar.gz && \
+    rm go1.22.2.linux-amd64.tar.gz


+COPY --from=build_mooncake /opt/venv/lib/python3.12/site-packages/mooncake /opt/venv/lib/python3.12/site-packages/mooncake
+COPY --from=build_mooncake /opt/venv/lib/python3.12/site-packages/mooncake-*.dist-info /opt/venv/lib/python3.12/site-packages/
+RUN apt-get update && \
+    apt-get install -y --no-install-recommends librdmacm-dev libibverbs-dev rdma-core && \


Copilot

Pull request overview

Copilot reviewed 5 out of 5 changed files in this pull request and generated 4 comments.

Comments suppressed due to low confidence (4)

.github/workflows/atom-benchmark.yaml:393

Same issue here: inputs.pd-benchmark is not a valid property access for an input id containing -. Use inputs['pd-benchmark'] or rename the input key to avoid hyphens.

    if: >-
      !cancelled()
      && (github.event_name == 'schedule'
          || inputs.pd-benchmark != false)
    needs: [benchmark]

.github/scripts/atom_pd_test.sh:406

local idx=0 is used at top-level inside the launch-all branch. local is only valid within functions; this will cause the script to exit. Use idx=0 instead (or move this logic into a function).

    launch_proxy "$local_ip"
    sleep 2

    # Launch consumer on each decode node (different port per node)
    local idx=0
    while IFS= read -r decode_host; do
        [ -z "$decode_host" ] && continue
        consumer_port=$((CONSUMER_BASE_PORT + idx))

.github/scripts/atom_pd_test.sh:523

local idx=0 is used at top-level in the dump-logs branch, which will error because local is only valid inside functions. Change it to idx=0 (or wrap this section in a function) to prevent the script from aborting.

if [ "$TYPE" == "dump-logs" ]; then
    echo "========== Proxy Log =========="
    cat "$PROXY_LOG" 2>/dev/null || echo "(no proxy log)"
    echo ""
    echo "========== Producer Log =========="
    cat "$PRODUCER_LOG" 2>/dev/null || echo "(no producer log)"
    local idx=0
    while IFS= read -r decode_host; do

.github/scripts/atom_pd_test.sh:194

Same issue for consumers: "kv_connector": "mooncake" is not a registered KV connector backend in ATOM today, so consumer openai_server instances will fail at startup. Register the backend or use the existing supported connector key.

    kv_config=$(cat <<KVJSON
{
    "kv_role": "kv_consumer",
    "kv_connector": "mooncake",
    "proxy_ip": "${local_ip}",
    "proxy_ping_port": ${DISCOVERY_PORT},
    "http_port": ${consumer_port}
}

+    if: >-
+      !cancelled()
+      && (github.event_name == 'schedule'
+          || inputs.pd-benchmark != false)
+    needs: [benchmark]


+    echo "========== Node Assignment =========="
+    echo "  Prefill: $(get_local_hostname) (${local_ip})"
+    local idx=0
+    while IFS= read -r dh; do
+        [ -z "$dh" ] && continue
+        dip=$(get_remote_ip "$dh")
+        echo "  Decode #${idx}: ${dh} (${dip})"
+        idx=$((idx + 1))


+              ssh -o StrictHostKeyChecking=no "${HOSTS[$i]}" bash -l <<EOF || true
+              docker stop atom-pd-consumer-${i} 2>/dev/null || true
+              docker rm atom-pd-consumer-${i} 2>/dev/null || true
+            EOF


+    kv_config=$(cat <<KVJSON
+{
+    "kv_role": "kv_producer",
+    "kv_connector": "mooncake",
+    "proxy_ip": "${local_ip}",
+    "proxy_ping_port": ${DISCOVERY_PORT},
+    "http_port": ${PRODUCER_PORT}
+}


Copilot

Pull request overview

Copilot reviewed 5 out of 5 changed files in this pull request and generated 7 comments.

+COPY --from=build_mooncake /opt/venv/lib/python3.12/site-packages/mooncake /opt/venv/lib/python3.12/site-packages/mooncake
+COPY --from=build_mooncake /opt/venv/lib/python3.12/site-packages/mooncake-*.dist-info /opt/venv/lib/python3.12/site-packages/
+RUN apt-get update && \
+    apt-get install -y --no-install-recommends librdmacm-dev libibverbs-dev rdma-core && \


+    strategy:
+      fail-fast: false
+      max-parallel: 1
+      matrix:
+        model: ${{ fromJson(needs.load-pd-models.outputs.models_json) }}
+        config: ${{ fromJson(needs.parse-pd-params.outputs.matrix_json) }}
+


+      - name: Setup SSH
+        run: |
+          mkdir -p ~/.ssh && chmod 700 ~/.ssh
+          if [ -n "${{ secrets.PD_SSH_KEY }}" ]; then
+            echo "${{ secrets.PD_SSH_KEY }}" > ~/.ssh/id_rsa
+            chmod 600 ~/.ssh/id_rsa
+          fi
+          IFS=',' read -ra NODES <<< "$PD_NODE_LIST"
+          for node in "${NODES[@]}"; do
+            ssh-keyscan -H "$(echo "$node" | xargs)" >> ~/.ssh/known_hosts 2>/dev/null || true
+          done


+#   PREFILL_ARGS, DECODE_ARGS, PROXY_PORT, PRODUCER_PORT, CONSUMER_PORT,
+#   ATOM_DOCKER_IMAGE, ISL, OSL, CONC, RANDOM_RANGE_RATIO, RESULT_FILENAME


+# ── RDMA NIC detection (ported from mori/docker/ci_run.sh) ─────────
+
+detect_nic_type() {
+    if [[ -n "${MORI_NIC_TYPE:-}" ]]; then echo "$MORI_NIC_TYPE"; return; fi
+    local bnxt=0 mlx5=0 ionic=0
+    if [[ -d /sys/class/infiniband ]]; then
+        for dev in /sys/class/infiniband/*; do
+            local name; name=$(basename "$dev")
+            case "$name" in
+                bnxt_re*) ((bnxt++)) ;; mlx5*) ((mlx5++)) ;; ionic*) ((ionic++)) ;;
+                *)
+                    local drv; drv=$(basename "$(readlink -f "$dev/device/driver" 2>/dev/null)" 2>/dev/null || true)
+                    case "$drv" in bnxt*) ((bnxt++)) ;; mlx5*) ((mlx5++)) ;; ionic*) ((ionic++)) ;; esac ;;
+            esac
+        done
+    fi
+    if (( bnxt >= mlx5 && bnxt >= ionic && bnxt > 0 )); then echo "bnxt"
+    elif (( ionic >= mlx5 && ionic > 0 )); then echo "ionic"
+    else echo "mlx5"; fi
+}


+
+SSH_OPTS="-o StrictHostKeyChecking=no -o ConnectTimeout=5 -o BatchMode=yes"


+      "concurrency": [32, 64, 128, 256, 512, 1024, 2048],
+      "random_range_ratio": 0.8
+    },
+    {
+      "isl": 8192,
+      "osl": 1024,
+      "concurrency": [32, 64, 128, 256, 512, 1024, 2048],


Copilot

Pull request overview

Copilot reviewed 5 out of 5 changed files in this pull request and generated 4 comments.

Comments suppressed due to low confidence (1)

.github/workflows/atom-benchmark.yaml:658

Same heredoc indentation issue here: the EOF terminator is indented, so bash will not close the heredoc correctly. Unindent the terminator (or use <<-EOF with tabs) to ensure remote cleanup commands execute reliably.

              ssh -o StrictHostKeyChecking=no "${HOSTS[$i]}" bash -l <<EOF || true
              docker stop atom-pd-consumer-${i} 2>/dev/null || true
              docker rm atom-pd-consumer-${i} 2>/dev/null || true
            EOF
            done

+            ssh -o StrictHostKeyChecking=no "${HOSTS[$i]}" bash -l <<EOF || true
+            docker stop atom-pd-consumer-${i} 2>/dev/null || true
+            docker rm atom-pd-consumer-${i} 2>/dev/null || true
+          EOF
+          done


+      - name: Install PD dependencies (prefill node)
+        run: |
+          docker exec atom-pd-prefill bash -lc \
+            "pip install msgpack msgspec quart mooncake-transfer-engine"


+    "pip install msgpack msgspec quart mooncake-transfer-engine"
+


+detect_nic_type() {
+    if [[ -n "${MORI_NIC_TYPE:-}" ]]; then echo "$MORI_NIC_TYPE"; return; fi
+    local bnxt=0 mlx5=0 ionic=0
+    if [[ -d /sys/class/infiniband ]]; then
+        for dev in /sys/class/infiniband/*; do
+            local name; name=$(basename "$dev")
+            case "$name" in
+                bnxt_re*) ((bnxt++)) ;; mlx5*) ((mlx5++)) ;; ionic*) ((ionic++)) ;;
+                *)
+                    local drv; drv=$(basename "$(readlink -f "$dev/device/driver" 2>/dev/null)" 2>/dev/null || true)
+                    case "$drv" in bnxt*) ((bnxt++)) ;; mlx5*) ((mlx5++)) ;; ionic*) ((ionic++)) ;; esac ;;
+            esac
+        done
+    fi
+    if (( bnxt >= mlx5 && bnxt >= ionic && bnxt > 0 )); then echo "bnxt"
+    elif (( ionic >= mlx5 && ionic > 0 )); then echo "ionic"
+    else echo "mlx5"; fi
+}
+
+find_host_ibverbs() {
+    for c in /usr/lib64/libibverbs.so.1 /lib/x86_64-linux-gnu/libibverbs.so.1 /usr/lib/x86_64-linux-gnu/libibverbs.so.1; do
+        local resolved; resolved=$(readlink -f "$c" 2>/dev/null || true)
+        if [[ -f "$resolved" ]]; then echo "$resolved"; return; fi
+    done
+}
+
+nic_mount_flags() {
+    local nic_type="$1" flags=()
+    case "$nic_type" in
+        bnxt)
+            local hib; hib=$(find_host_ibverbs)
+            [[ -n "$hib" ]] && flags+=(-v "$hib:/lib/x86_64-linux-gnu/libibverbs.so.1")
+            for lib in /usr/local/lib/libbnxt_re-rdmav*.so; do
+                [[ -f "$lib" ]] && flags+=(-v "$lib:/usr/lib/x86_64-linux-gnu/libibverbs/$(basename "$lib")")
+            done
+            for lib in /usr/local/lib/libbnxt_re.so; do
+                [[ -f "$lib" ]] && flags+=(-v "$lib:/usr/lib/x86_64-linux-gnu/$(basename "$lib")")
+            done
+            [[ -d /etc/libibverbs.d ]] && flags+=(-v /etc/libibverbs.d:/etc/libibverbs.d:ro)
+            ;;
+        ionic)
+            local hib; hib=$(find_host_ibverbs)
+            [[ -n "$hib" ]] && flags+=(-v "$hib:/lib/x86_64-linux-gnu/libibverbs.so.1")
+            for dir in /usr/local/lib /usr/lib/x86_64-linux-gnu; do
+                for lib in "$dir"/libionic*.so; do
+                    if [[ -f "$lib" ]]; then
+                        local real; real=$(readlink -f "$lib")
+                        [[ -f "$real" ]] && flags+=(-v "$real:$real")
+                        flags+=(-v "$lib:/usr/lib/x86_64-linux-gnu/$(basename "$lib")")
+                    fi
+                done
+            done
+            local pdir=/usr/lib/x86_64-linux-gnu/libibverbs
+            if [[ -d "$pdir" ]]; then
+                for lib in "$pdir"/libionic-rdmav*.so; do
+                    [[ -f "$lib" ]] && flags+=(-v "$lib:$lib")
+                done
+            fi
+            [[ -d /etc/libibverbs.d ]] && flags+=(-v /etc/libibverbs.d:/etc/libibverbs.d:ro)
+            ;;
+    esac
+    echo "${flags[@]}"
+}


Copilot

Pull request overview

Copilot reviewed 5 out of 5 changed files in this pull request and generated 4 comments.

Comments suppressed due to low confidence (2)

.github/workflows/atom-benchmark.yaml:511

In the PD workflow container launch, NIC_TYPE can be detected as "ionic" (see case on infiniband devices), but only the bnxt path mounts extra RDMA user-space libraries. Either drop ionic detection here or add the corresponding ionic mount logic (similar to the bnxt mounts) so RDMA-enabled ionic runners work consistently.

          NIC_TYPE="mlx5"
          if [ -d /sys/class/infiniband ]; then
            for dev in /sys/class/infiniband/*; do
              name=$(basename "$dev")
              case "$name" in bnxt_re*) NIC_TYPE="bnxt"; break ;; ionic*) NIC_TYPE="ionic"; break ;; esac
            done
          fi
          if [ "$NIC_TYPE" = "bnxt" ]; then
            for c in /usr/lib64/libibverbs.so.1 /lib/x86_64-linux-gnu/libibverbs.so.1 /usr/lib/x86_64-linux-gnu/libibverbs.so.1; do
              resolved=$(readlink -f "$c" 2>/dev/null || true)

.github/scripts/atom_pd_test.sh:292

In the remote consumer launch path, NIC_TYPE can become "ionic", but only the bnxt branch populates NIC_MOUNTS. This makes the ionic detection misleading and can break RDMA setups that require extra ionic userspace libraries. Either handle the ionic case or simplify detection to only the NIC types you actively support here.

# NIC detection for RDMA userspace libs (ported from mori)
NIC_MOUNTS=""
NIC_TYPE="mlx5"
if [ -d /sys/class/infiniband ]; then
    for dev in /sys/class/infiniband/*; do
        name=\$(basename "\$dev")
        case "\$name" in bnxt_re*) NIC_TYPE="bnxt"; break ;; ionic*) NIC_TYPE="ionic"; break ;; esac
    done
fi
if [ "\$NIC_TYPE" = "bnxt" ]; then
    for c in /usr/lib64/libibverbs.so.1 /lib/x86_64-linux-gnu/libibverbs.so.1 /usr/lib/x86_64-linux-gnu/libibverbs.so.1; do
        resolved=\$(readlink -f "\$c" 2>/dev/null || true)

+# ── RDMA NIC detection (ported from mori/docker/ci_run.sh) ─────────
+
+detect_nic_type() {
+    if [[ -n "${MORI_NIC_TYPE:-}" ]]; then echo "$MORI_NIC_TYPE"; return; fi
+    local bnxt=0 mlx5=0 ionic=0
+    if [[ -d /sys/class/infiniband ]]; then
+        for dev in /sys/class/infiniband/*; do
+            local name; name=$(basename "$dev")
+            case "$name" in
+                bnxt_re*) ((bnxt++)) ;; mlx5*) ((mlx5++)) ;; ionic*) ((ionic++)) ;;
+                *)
+                    local drv; drv=$(basename "$(readlink -f "$dev/device/driver" 2>/dev/null)" 2>/dev/null || true)
+                    case "$drv" in bnxt*) ((bnxt++)) ;; mlx5*) ((mlx5++)) ;; ionic*) ((ionic++)) ;; esac ;;
+            esac
+        done
+    fi
+    if (( bnxt >= mlx5 && bnxt >= ionic && bnxt > 0 )); then echo "bnxt"
+    elif (( ionic >= mlx5 && ionic > 0 )); then echo "ionic"
+    else echo "mlx5"; fi
+}
+
+find_host_ibverbs() {
+    for c in /usr/lib64/libibverbs.so.1 /lib/x86_64-linux-gnu/libibverbs.so.1 /usr/lib/x86_64-linux-gnu/libibverbs.so.1; do
+        local resolved; resolved=$(readlink -f "$c" 2>/dev/null || true)
+        if [[ -f "$resolved" ]]; then echo "$resolved"; return; fi
+    done
+}
+


+    python -m atom.benchmarks.benchmark_serving \
+        --model="$MODEL_PATH" --backend=vllm --base-url="http://localhost:${PROXY_PORT}" \
+        --dataset-name=random \
+        --random-input-len="$ISL" --random-output-len="$OSL" --random-range-ratio="$RANDOM_RANGE_RATIO" \
+        --max-concurrency="$CONC" \
+        --num-prompts="${NUM_PROMPTS_OVERRIDE:-$(( CONC * 10 ))}" \
+        --trust-remote-code \
+        --num-warmups="$(( CONC * 2 ))" \


+COPY --from=build_mooncake /opt/venv/lib/python3.12/site-packages/mooncake /opt/venv/lib/python3.12/site-packages/mooncake
+COPY --from=build_mooncake /opt/venv/lib/python3.12/site-packages/mooncake-*.dist-info /opt/venv/lib/python3.12/site-packages/
+RUN apt-get update && \
+    apt-get install -y --no-install-recommends librdmacm-dev libibverbs-dev rdma-core && \


+    strategy:
+      fail-fast: false
+      max-parallel: 1
+      matrix:
+        model: ${{ fromJson(needs.load-pd-models.outputs.models_json) }}
+        config: ${{ fromJson(needs.parse-pd-params.outputs.matrix_json) }}
+


…e install The custom RCCL .deb (built from a specific commit) conflicts with the version expected by the ROCm apt repo's rocm-hip package. This caused apt-get install rdma-core to fail with unresolvable dependencies. Co-Authored-By: Claude Opus 4 <noreply@anthropic.com>

Copilot

Pull request overview

Copilot reviewed 5 out of 5 changed files in this pull request and generated 8 comments.

Comments suppressed due to low confidence (1)

.github/workflows/atom-benchmark.yaml:809

Appending pd_models.json into the models list here will overwrite existing display-name mappings when the PD model shares the same path/HF last-segment and has no suffix. For example, deepseek-ai/DeepSeek-R1-0528 already exists in models.json, and both PD entries reuse that path, so mapping['deepseek-r1-0528'] will end up pointing at a PD display name and mislabel the standard single-node DeepSeek results on the dashboard. To avoid collisions, ensure PD configs produce a distinct key (e.g., add a suffix in pd_models.json, or update the mapping key logic to prefer prefix when present).

          python3 -c "
          import json
          models = json.loads(open('.github/benchmark/models.json').read())
          # Include PD model configs in dashboard mapping
          try:
              models += json.loads(open('.github/benchmark/pd_models.json').read())
          except FileNotFoundError:
              pass
          mapping = {}
          for m in models:
              display = m.get('display','')
              path = m.get('path','')
              suffix = m.get('suffix','')
              if not display or not path: continue
              hf_name = path.split('/')[-1].lower()
              # HF path-derived key (old data used model_id last segment + optional suffix)
              hf_key = hf_name + suffix.lower() if suffix else hf_name
              mapping[hf_key] = display

+        zip unzip wget gcc make libtool autoconf cmake \
+        librdmacm-dev rdmacm-utils infiniband-diags ibverbs-utils perftest ethtool \
+        libibverbs-dev rdma-core \
+        openssh-server openmpi-bin openmpi-common libopenmpi-dev && \


+COPY --from=build_mooncake /usr/local/lib/libmooncake* /usr/local/lib/
+COPY --from=build_mooncake /usr/local/lib/libetcd_wrapper.so /usr/local/lib/
+COPY --from=build_mooncake /usr/local/lib/libgrpc* /usr/local/lib/
+COPY --from=build_mooncake /usr/local/lib/libprotobuf* /usr/local/lib/
+COPY --from=build_mooncake /usr/local/lib/libabsl* /usr/local/lib/
+COPY --from=build_mooncake /usr/local/lib/libupb* /usr/local/lib/
+COPY --from=build_mooncake /usr/local/lib/libre2* /usr/local/lib/


+COPY --from=build_mooncake /opt/venv/lib/python3.12/site-packages/mooncake-*.dist-info /opt/venv/lib/python3.12/site-packages/
+RUN apt-mark hold rccl rccl-dev rocm-hip && \
+    apt-get update && \
+    apt-get install -y --no-install-recommends librdmacm-dev libibverbs-dev && \


+      - name: Setup SSH
+        run: |
+          mkdir -p ~/.ssh && chmod 700 ~/.ssh
+          if [ -n "${{ secrets.PD_SSH_KEY }}" ]; then
+            echo "${{ secrets.PD_SSH_KEY }}" > ~/.ssh/id_rsa
+            chmod 600 ~/.ssh/id_rsa
+          fi
+          IFS=',' read -ra NODES <<< "$PD_NODE_LIST"
+          for node in "${NODES[@]}"; do
+            ssh-keyscan -H "$(echo "$node" | xargs)" >> ~/.ssh/known_hosts 2>/dev/null || true
+          done


+      - name: Install PD dependencies (prefill node)
+        run: |
+          docker exec atom-pd-prefill bash -lc \
+            "pip install msgpack msgspec quart mooncake-transfer-engine"


+        run: |
+          if [ -d "/models" ]; then model_path="/models/${{ env.MODEL_PATH }}"
+          else model_path="${{ env.MODEL_PATH }}"; fi
+
+          docker exec atom-pd-prefill bash -lc "set -euo pipefail
+            .github/scripts/atom_pd_test.sh launch-all $model_path ${{ inputs.extra_args || '' }}"
+


+PROXY_PORT=${PROXY_PORT:-10001}
+PRODUCER_PORT=${PRODUCER_PORT:-8003}
+CONSUMER_BASE_PORT=${CONSUMER_BASE_PORT:-8004}
+DISCOVERY_PORT=${DISCOVERY_PORT:-36367}
+NUM_DECODE_NODES=${NUM_DECODE_NODES:-1}


+# ── RDMA NIC detection (ported from mori/docker/ci_run.sh) ─────────
+
+detect_nic_type() {
+    if [[ -n "${MORI_NIC_TYPE:-}" ]]; then echo "$MORI_NIC_TYPE"; return; fi
+    local bnxt=0 mlx5=0 ionic=0


Copilot

Pull request overview

Copilot reviewed 5 out of 5 changed files in this pull request and generated 8 comments.

+RUN rm -rf /usr/local/go && \
+    wget -q https://go.dev/dl/go1.22.2.linux-amd64.tar.gz && \
+    tar -C /usr/local -xzf go1.22.2.linux-amd64.tar.gz && \
+    rm go1.22.2.linux-amd64.tar.gz && \
+    go version


+        librdmacm-dev libibverbs-dev \
+        libgflags-dev libgoogle-glog-dev \
+        libjsoncpp-dev libyaml-cpp-dev \
+        libnuma-dev libcurl4-openssl-dev \
+        libunwind-dev libc-ares-dev && \


+          docker exec atom-pd-prefill bash -lc \
+            "pip install msgpack msgspec quart mooncake-transfer-engine"
+


+        run: |
+          if [ -d "/models" ]; then model_path="/models/${{ env.MODEL_PATH }}"
+          else model_path="${{ env.MODEL_PATH }}"; fi
+
+          docker exec atom-pd-prefill bash -lc "set -euo pipefail
+            .github/scripts/atom_pd_test.sh launch-all $model_path ${{ inputs.extra_args || '' }}"


+# ── RDMA NIC detection (ported from mori/docker/ci_run.sh) ─────────
+
+detect_nic_type() {
+    if [[ -n "${MORI_NIC_TYPE:-}" ]]; then echo "$MORI_NIC_TYPE"; return; fi
+    local bnxt=0 mlx5=0 ionic=0
+    if [[ -d /sys/class/infiniband ]]; then
+        for dev in /sys/class/infiniband/*; do
+            local name; name=$(basename "$dev")
+            case "$name" in
+                bnxt_re*) ((bnxt++)) ;; mlx5*) ((mlx5++)) ;; ionic*) ((ionic++)) ;;
+                *)
+                    local drv; drv=$(basename "$(readlink -f "$dev/device/driver" 2>/dev/null)" 2>/dev/null || true)
+                    case "$drv" in bnxt*) ((bnxt++)) ;; mlx5*) ((mlx5++)) ;; ionic*) ((ionic++)) ;; esac ;;
+            esac
+        done
+    fi
+    if (( bnxt >= mlx5 && bnxt >= ionic && bnxt > 0 )); then echo "bnxt"
+    elif (( ionic >= mlx5 && ionic > 0 )); then echo "ionic"
+    else echo "mlx5"; fi
+}
+
+find_host_ibverbs() {
+    for c in /usr/lib64/libibverbs.so.1 /lib/x86_64-linux-gnu/libibverbs.so.1 /usr/lib/x86_64-linux-gnu/libibverbs.so.1; do
+        local resolved; resolved=$(readlink -f "$c" 2>/dev/null || true)
+        if [[ -f "$resolved" ]]; then echo "$resolved"; return; fi
+    done
+}
+


+if docker ps -q -f "name=atom-" 2>/dev/null | grep -q .; then
+    echo "BUSY:docker"; exit 1
+fi
+USED=$(rocm-smi --showmemuse 2>/dev/null | grep "VRAM%" | awk '{print $NF}' | awk '$1 > 0' | wc -l)


+docker run -dt --device=/dev/kfd \$DEVICE_FLAG \$IB_FLAG \
+    \$NIC_MOUNTS \
+    \$MODEL_MOUNT \
+    -w /workspace --ipc=host --group-add video \
+    --shm-size=16G --privileged --cap-add=SYS_PTRACE \
+    --security-opt seccomp=unconfined \
+    --ulimit memlock=-1 --ulimit stack=67108864 \
+    -e ATOM_DISABLE_MMAP=true \
+    -e NCCL_SOCKET_IFNAME=lo \
+    -e AITER_LOG_LEVEL=WARNING \
+    --network=host \
+    --name ${container_name} \
+    ${ATOM_DOCKER_IMAGE:-rocm/atom-dev:latest}


+docker exec ${container_name} bash -lc \
+    "pip install msgpack msgspec quart mooncake-transfer-engine"
+


Copilot AI review requested due to automatic review settings May 13, 2026 08:13

Copilot started reviewing on behalf of ZhangLirong-amd May 13, 2026 08:14 View session

Copilot AI reviewed May 13, 2026

View reviewed changes

Copilot AI review requested due to automatic review settings May 13, 2026 09:44

Copilot AI reviewed May 13, 2026

View reviewed changes

Copilot AI review requested due to automatic review settings May 13, 2026 12:21

Copilot AI reviewed May 13, 2026

View reviewed changes

ZhangLirong-amd force-pushed the mooncake_build branch from 2af9c26 to 22c597e Compare May 14, 2026 05:58

Copilot AI review requested due to automatic review settings May 14, 2026 09:09

ZhangLirong-amd force-pushed the mooncake_build branch from 22c597e to a4fab45 Compare May 14, 2026 09:09

Copilot started reviewing on behalf of ZhangLirong-amd May 14, 2026 09:10 View session

Copilot AI reviewed May 14, 2026

View reviewed changes

ZhangLirong-amd added 6 commits May 15, 2026 16:08

Add mooncake dockerfile build

4f50c89

fix go version

55a2989

support ci benchmark for pd multi node

5feaa80

add pip and nic docker RDMA needed

faf7dae

fix index bug

23e9499

change mi300 to 355

0142da3

ZhangLirong-amd force-pushed the mooncake_build branch from a4fab45 to 0142da3 Compare May 15, 2026 08:08

fix docker build failed

79b4496

Copilot AI review requested due to automatic review settings May 15, 2026 08:40

Copilot started reviewing on behalf of ZhangLirong-amd May 15, 2026 08:42 View session

Copilot AI reviewed May 15, 2026

View reviewed changes

root and others added 2 commits May 15, 2026 10:23

fix build error

b6493fe

Copilot AI review requested due to automatic review settings May 15, 2026 12:46

Copilot started reviewing on behalf of ZhangLirong-amd May 15, 2026 12:48 View session

Copilot AI reviewed May 15, 2026

View reviewed changes

ZhangLirong-amd added 2 commits May 15, 2026 22:36

fix failed

2bf855f

fix some error in builder

a3b838e

Copilot AI review requested due to automatic review settings May 16, 2026 12:44

Copilot started reviewing on behalf of ZhangLirong-amd May 16, 2026 12:45 View session

Copilot AI reviewed May 16, 2026

View reviewed changes

add network

af670be

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Add mooncake dockerfile build#771

Add mooncake dockerfile build#771
ZhangLirong-amd wants to merge 12 commits into
mainfrom
mooncake_build

ZhangLirong-amd commented May 13, 2026 •

edited

Loading

Uh oh!

Copilot AI left a comment

Uh oh!

Copilot AI left a comment

Uh oh!

Copilot AI left a comment

Uh oh!

Copilot AI left a comment

Uh oh!

Copilot AI left a comment

Uh oh!

Copilot AI left a comment

Uh oh!

Copilot AI left a comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

		# PREFILL_ARGS, DECODE_ARGS, PROXY_PORT, PRODUCER_PORT, CONSUMER_PORT,
		# ATOM_DOCKER_IMAGE, ISL, OSL, CONC, RANDOM_RANGE_RATIO, RESULT_FILENAME


		SSH_OPTS="-o StrictHostKeyChecking=no -o ConnectTimeout=5 -o BatchMode=yes"

		"pip install msgpack msgspec quart mooncake-transfer-engine"

		docker exec atom-pd-prefill bash -lc \
		"pip install msgpack msgspec quart mooncake-transfer-engine"

		docker exec ${container_name} bash -lc \
		"pip install msgpack msgspec quart mooncake-transfer-engine"

Conversation

ZhangLirong-amd commented May 13, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Summary

Technical Details

Test Plan

Test Result

Submission Checklist

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

ZhangLirong-amd commented May 13, 2026 •

edited

Loading