1Cat-vLLM 1.1.0在4×V100 SXM2 16GB上，TP=4跑Qwen3-30B-A3B-AWQ，单请求完全正常，但多个请求并发时报错

[zhaoqingxiaozhao]

🚀 The feature, motivation and pitch

GitHub Issue版本（更详细）：

Environment:

GPU: Tesla V100 SXM2 16GB × 4
1Cat-vLLM: 1.1.0
CUDA: 12.0
Python: 3.12
Model: Qwen3-30B-A3B-AWQ

Problem:
Single request works perfectly. But when 5 concurrent requests are sent simultaneously, the service crashes with:
RuntimeError: Worker failed with error 'Shared memory exceeds 96KB: 99840 bytes'
Launch command:
--tensor-parallel-size 4 --enforce-eager --dtype float16 --max-num-seqs 5
Question: Is there a fix or workaround for this?

Alternatives

你好，请问1Cat-vLLM 1.1.0在4×V100 SXM2 16GB上，TP=4跑Qwen3-30B-A3B-AWQ，单请求完全正常，但多个请求并发时报错：RuntimeError: Worker failed with error 'Shared memory exceeds 96KB: 99840 bytes'，服务直接崩溃重启。请问这个问题有解决方案吗？

Additional context

No response

Before submitting a new issue...

• Make sure you already searched for relevant issues, and asked the chatbot living at the bottom right corner of the documentation page, which can answer lots of frequently asked questions.

[zhaoqingxiaozhao]

Environment:

GPU: Tesla V100 SXM2 16GB × 4
1Cat-vLLM: 1.1.0
CUDA: 12.0
Python: 3.12
Model: Qwen3-30B-A3B-AWQ

Problem:
Single request works perfectly. But when 5 concurrent requests are sent simultaneously, the service crashes with:
RuntimeError: Worker failed with error 'Shared memory exceeds 96KB: 99840 bytes'
Launch command:
--tensor-parallel-size 4 --enforce-eager --dtype float16 --max-num-seqs 5
Question: Is there a fix or workaround for this?

[yangzhuxinyzx]

感谢反馈。这个场景里有两个关键点：

1. 你使用的是 1Cat-vLLM 1.1.0，建议先升级到最新 1.2.0 wheel：
   https://github.com/1CatAI/1Cat-vLLM/releases/tag/v1.2.0

2. 4×V100 16GB 上 --max-num-seqs 5 对 Qwen3-30B-A3B-AWQ 这类 MoE/长上下文场景比较激进，不建议作为稳定 public 配置。V100 的 shared memory 上限是硬件限制，Shared memory exceeds 96KB 说明当前命中的 kernel shape 已超过 SM70 可承受范围。

建议 workaround：

--max-num-seqs 1
--max-num-batched-tokens 4096

如果显存和 TTFT 可接受，再逐步尝试：

--max-num-batched-tokens 8192

1.2.0 已经针对 V100 做了更保守的 long-context memory policy、paged-prefix low-smem 和相关 prefill 路线优化，但 16GB V100 多并发仍然需要保守配置。建议先用 1.2.0 + 低并发参数确认稳定，再逐步放大并发。