[NPU] support multi-agent rollout pipeline by Windfeng8 · Pull Request #298 · vllm-project/vime

Windfeng8 · 2026-06-30T03:14:00Z

Added local token/logprob parsing for compatibility with the latest VIME.Assigned the same to sibling agent samples.

Signed-off-by: Windfeng8 <523758380@qq.com>

read-the-docs-community · 2026-06-30T03:15:02Z

Documentation build overview

📚 vime | 🛠️ Build #33388884 | 📁 Comparing 7104c8f against latest (d16d1dc)

🔍 Preview build

41 files changed · + 1 added · ± 26 modified · - 14 deleted

+ Added

get_started/NPU.html

± Modified

- Deleted

gemini-code-assist

Code Review

This pull request introduces a compatibility helper for extracting token IDs and log probabilities from vLLM rollout responses, updates multi-agent rollouts to share a single rollout_id among sibling samples to prevent over-counting during loss reduction, and adds a training script for Qwen3-4B on NPU. The review feedback highlights several critical improvements: adding defensive checks to prevent TypeError when converting potentially None log probabilities, handling cases where the multi-agent function returns None or empty samples, and replacing hardcoded absolute paths in the shell script with dynamic path resolution to ensure portability.

gemini-code-assist · 2026-06-30T03:15:18Z

+        new_response_log_probs = [
+            float(item.get("logprob", 0.0)) if isinstance(item, dict) else 0.0
+            for item in content_items
+        ]


If item.get("logprob") is explicitly None or missing, calling float(None) will raise a TypeError and crash the rollout pipeline. We should defensively check if the logprob value is not None before converting it to a float.

Suggested change

new_response_log_probs = [

float(item.get("logprob", 0.0)) if isinstance(item, dict) else 0.0

for item in content_items

]

new_response_log_probs = [

float(item.get("logprob")) if isinstance(item, dict) and item.get("logprob") is not None else 0.0

for item in content_items

]

gemini-code-assist · 2026-06-30T03:15:18Z

+export PYTHONPATH="/workspace/issue205/Megatron-Bridge/src:/workspace/issue205/Megatron-Bridge:/workspace/issue205/Megatron-LM:/workspace/issue205/vime-ascend:${PYTHONPATH}"
+export ASCEND_RT_VISIBLE_DEVICES="${ASCEND_RT_VISIBLE_DEVICES:-0,1,2,3,4,5,6,7,8,9,10,11,12,13,14,15}"
+export CUDA_DEVICE_MAX_CONNECTIONS=1
+export RAY_EXPERIMENTAL_NOSET_ASCEND_RT_VISIBLE_DEVICES=1
+export HCCL_HOST_SOCKET_PORT_RANGE=60000-60050
+export HCCL_NPU_SOCKET_PORT_RANGE=61000-61050
+export HYDRA_FULL_ERROR=1
+export MASTER_PORT="${MASTER_PORT:-$(shuf -i 20000-65000 -n 1)}"
+export DISABLE_L2_CACHE=1
+export VLLM_ASCEND_ENABLE_NZ=0
+
+VIME_ROOT="${VIME_ROOT:-/workspace/issue205/vime-ascend}"
+SCRIPT_DIR="${VIME_ROOT}/scripts"
+WEIGHT_DIR="${WEIGHT_DIR:-/home/data/weights/Qwen3-4B}"
+DATA_FILE="${DATA_FILE:-/home/w00893744/dataset/dapo-math-17k.jsonl}"
+LOG_FILE="${LOG_FILE:-/home/w00893744/train_qwen3_4b_multi_agent_vllm.log}"


The script contains hardcoded absolute paths specific to a single user's environment (e.g., /workspace/issue205/... and /home/w00893744/...). This makes the script non-portable and prone to immediate failure on other machines or for other users.

We should dynamically resolve the project root (VIME_ROOT) and workspace directory relative to the script's location, and avoid hardcoding user-specific home directories for DATA_FILE and LOG_FILE.

Suggested change

export PYTHONPATH="/workspace/issue205/Megatron-Bridge/src:/workspace/issue205/Megatron-Bridge:/workspace/issue205/Megatron-LM:/workspace/issue205/vime-ascend:${PYTHONPATH}"

export ASCEND_RT_VISIBLE_DEVICES="${ASCEND_RT_VISIBLE_DEVICES:-0,1,2,3,4,5,6,7,8,9,10,11,12,13,14,15}"

export CUDA_DEVICE_MAX_CONNECTIONS=1

export RAY_EXPERIMENTAL_NOSET_ASCEND_RT_VISIBLE_DEVICES=1

export HCCL_HOST_SOCKET_PORT_RANGE=60000-60050

export HCCL_NPU_SOCKET_PORT_RANGE=61000-61050

export HYDRA_FULL_ERROR=1

export MASTER_PORT="${MASTER_PORT:-$(shuf -i 20000-65000 -n 1)}"

export DISABLE_L2_CACHE=1

export VLLM_ASCEND_ENABLE_NZ=0

VIME_ROOT="${VIME_ROOT:-/workspace/issue205/vime-ascend}"

SCRIPT_DIR="${VIME_ROOT}/scripts"

WEIGHT_DIR="${WEIGHT_DIR:-/home/data/weights/Qwen3-4B}"

DATA_FILE="${DATA_FILE:-/home/w00893744/dataset/dapo-math-17k.jsonl}"

LOG_FILE="${LOG_FILE:-/home/w00893744/train_qwen3_4b_multi_agent_vllm.log}"

SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"

VIME_ROOT="${VIME_ROOT:-$(cd "${SCRIPT_DIR}/.." && pwd)}"

WORKSPACE_DIR="$(cd "${VIME_ROOT}/.." && pwd)"

export PYTHONPATH="${WORKSPACE_DIR}/Megatron-Bridge/src:${WORKSPACE_DIR}/Megatron-Bridge:${WORKSPACE_DIR}/Megatron-LM:${VIME_ROOT}:${PYTHONPATH}"

export ASCEND_RT_VISIBLE_DEVICES="${ASCEND_RT_VISIBLE_DEVICES:-0,1,2,3,4,5,6,7,8,9,10,11,12,13,14,15}"

export CUDA_DEVICE_MAX_CONNECTIONS=1

export RAY_EXPERIMENTAL_NOSET_ASCEND_RT_VISIBLE_DEVICES=1

export HCCL_HOST_SOCKET_PORT_RANGE=60000-60050

export HCCL_NPU_SOCKET_PORT_RANGE=61000-61050

export HYDRA_FULL_ERROR=1

export MASTER_PORT="${MASTER_PORT:-$(shuf -i 20000-65000 -n 1)}"

export DISABLE_L2_CACHE=1

export VLLM_ASCEND_ENABLE_NZ=0

WEIGHT_DIR="${WEIGHT_DIR:-/home/data/weights/Qwen3-4B}"

if [ -z "${DATA_FILE}" ]; then

echo "Error: DATA_FILE environment variable is not set." >&2

exit 1

fi

LOG_FILE="${LOG_FILE:-/tmp/train_qwen3_4b_multi_agent_vllm.log}"

gemini-code-assist · 2026-06-30T03:15:18Z

+    for sibling in samples:
+        sibling.rollout_id = compact_rollout_id
+
    random.shuffle(samples)


If the custom multi-agent function custom_multi_agent_func returns None or an empty list, iterating over samples or calling random.shuffle(samples) will raise a TypeError. We should add a defensive check to ensure samples is not None or empty before processing.

Suggested change

for sibling in samples:

sibling.rollout_id = compact_rollout_id

random.shuffle(samples)

if samples:

for sibling in samples:

if sibling is not None:

sibling.rollout_id = compact_rollout_id

random.shuffle(samples)

else:

samples = []

floatlibai · 2026-06-30T07:37:16Z

+
+source "${SCRIPT_DIR}/models/qwen3-4B.sh"
+
+python "${VIME_ROOT}/train.py" \


Please use ray to submit task instead of using python directly, just ref the gpu script.

floatlibai · 2026-06-30T07:39:19Z

You should put this script under examples/multi_agent folder

Signed-off-by: Windfeng8 <523758380@qq.com>

floatlibai · 2026-07-01T09:29:54Z

+   --balance-data
+)
+
+EVAL_ARGS=(


EVAL_ARGS and WANDB_ARGS can be removed since they are not used.

floatlibai · 2026-07-01T09:35:33Z

Could you add a short PR description covering what changes were made to the multi-agent example and why?

Windfeng8 · 2026-07-07T08:23:31Z

Could you add a short PR description covering what changes were made to the multi-agent example and why?

done

fix: resolve multi-agent

5650fd1

Signed-off-by: Windfeng8 <523758380@qq.com>

gemini-code-assist Bot reviewed Jun 30, 2026

View reviewed changes

Windfeng8 changed the title ~~support multi-agent rollout pipeline~~ [NPU] support multi-agent rollout pipeline Jun 30, 2026

Windfeng8 mentioned this pull request Jun 30, 2026

[RFC] Steps and Test Result for Running Qwen3-4B on NPU(A3) （multi-agent rollout pipeline） #299

Open

floatlibai reviewed Jun 30, 2026

View reviewed changes

fix: run-qwen3-4B-multi-agent-npu.sh

7104c8f

Signed-off-by: Windfeng8 <523758380@qq.com>

floatlibai reviewed Jul 1, 2026

View reviewed changes

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

[NPU] support multi-agent rollout pipeline#298

[NPU] support multi-agent rollout pipeline#298
Windfeng8 wants to merge 2 commits into
vllm-project:ascendfrom
Windfeng8:fix/multi-agent

Windfeng8 commented Jun 30, 2026 •

edited

Loading

Uh oh!

read-the-docs-community Bot commented Jun 30, 2026 •

edited

Loading

Uh oh!

gemini-code-assist Bot left a comment

Uh oh!

gemini-code-assist Bot Jun 30, 2026

Uh oh!

gemini-code-assist Bot Jun 30, 2026

Uh oh!

gemini-code-assist Bot Jun 30, 2026

Uh oh!

floatlibai Jun 30, 2026 •

edited

Loading

Uh oh!

Windfeng8 Jul 1, 2026 •

edited

Loading

Uh oh!

floatlibai Jun 30, 2026 •

edited

Loading

Uh oh!

Windfeng8 Jul 1, 2026

Uh oh!

floatlibai Jul 1, 2026 •

edited

Loading

Uh oh!

floatlibai commented Jul 1, 2026

Uh oh!

Windfeng8 commented Jul 7, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants


		source "${SCRIPT_DIR}/models/qwen3-4B.sh"

		python "${VIME_ROOT}/train.py" \

Uh oh!

Conversation

Windfeng8 commented Jun 30, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

read-the-docs-community Bot commented Jun 30, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Documentation build overview

Uh oh!

gemini-code-assist Bot left a comment

Choose a reason for hiding this comment

Code Review

Uh oh!

gemini-code-assist Bot Jun 30, 2026

Choose a reason for hiding this comment

Uh oh!

gemini-code-assist Bot Jun 30, 2026

Choose a reason for hiding this comment

Uh oh!

gemini-code-assist Bot Jun 30, 2026

Choose a reason for hiding this comment

Uh oh!

floatlibai Jun 30, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Windfeng8 Jul 1, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

floatlibai Jun 30, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Windfeng8 Jul 1, 2026

Choose a reason for hiding this comment

Uh oh!

floatlibai Jul 1, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

floatlibai commented Jul 1, 2026

Uh oh!

Windfeng8 commented Jul 7, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

Windfeng8 commented Jun 30, 2026 •

edited

Loading

read-the-docs-community Bot commented Jun 30, 2026 •

edited

Loading

floatlibai Jun 30, 2026 •

edited

Loading

Windfeng8 Jul 1, 2026 •

edited

Loading

floatlibai Jun 30, 2026 •

edited

Loading

floatlibai Jul 1, 2026 •

edited

Loading