vllm/transformer cannot load GLM-4.7-flash due to module name mismatch

[blairyeung]

System Info / 系統信息

Driver Version: 575.57.08 CUDA Driver Version: 12.9 RTX 5090 (Blackwell)
vllm 0.14.1
transformers 5.0.0.dev0 （在4.57.6显示 transformers不支持）

Who can help? / 谁可以帮助到您？

@zRzRzRzRzRzRzR

Information / 问题信息

• The official example scripts / 官方的示例脚本
• My own modified scripts / 我自己修改的脚本和任务

Reproduction / 复现过程

CUDA_VISIBLE_DEVICES=2,3
vllm serve /data2/models/ZhipuAI/GLM-4.7-Flash
--tensor-parallel-size 2
--tool-call-parser glm47
--reasoning-parser glm45
--quantization fp8
--max-model-len 32768
--enable-auto-tool-choice
--served-model-name glm-4.7-flash

---

(Worker_TP0 pid=884060) ERROR 01-26 14:08:37 [multiproc_executor.py:749] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
(Worker_TP0 pid=884060) ERROR 01-26 14:08:37 [multiproc_executor.py:749] File "/home/user/miniconda3/envs/vllm-nightly/lib/python3.12/site-packages/vllm/model_executor/models/utils.py", line 335, in load_weights
(Worker_TP0 pid=884060) ERROR 01-26 14:08:37 [multiproc_executor.py:749] autoloaded_weights = set(self._load_module("", self.module, weights))
(Worker_TP0 pid=884060) ERROR 01-26 14:08:37 [multiproc_executor.py:749] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
(Worker_TP0 pid=884060) ERROR 01-26 14:08:37 [multiproc_executor.py:749] File "/home/user/miniconda3/envs/vllm-nightly/lib/python3.12/site-packages/vllm/model_executor/models/utils.py", line 288, in _load_module
(Worker_TP0 pid=884060) ERROR 01-26 14:08:37 [multiproc_executor.py:749] yield from self._load_module(
(Worker_TP0 pid=884060) ERROR 01-26 14:08:37 [multiproc_executor.py:749] File "/home/user/miniconda3/envs/vllm-nightly/lib/python3.12/site-packages/vllm/model_executor/models/utils.py", line 288, in _load_module
(Worker_TP0 pid=884060) ERROR 01-26 14:08:37 [multiproc_executor.py:749] yield from self._load_module(
(Worker_TP0 pid=884060) ERROR 01-26 14:08:37 [multiproc_executor.py:749] File "/home/user/miniconda3/envs/vllm-nightly/lib/python3.12/site-packages/vllm/model_executor/models/utils.py", line 319, in _load_module
(Worker_TP0 pid=884060) ERROR 01-26 14:08:37 [multiproc_executor.py:749] raise ValueError(msg)
(Worker_TP0 pid=884060) ERROR 01-26 14:08:37 [multiproc_executor.py:749] ValueError: There is no module or parameter named 'model.layers.47' in TransformersMoEForCausalLM

Expected behavior / 期待表现

launches without error

[zRzRzRzRzRzRzR]

transformers要是主分支

[blairyeung]

确实是主分支 (base) git checkout main
Already on 'main'
Your branch is up to date with 'origin/main'.
但是还是有这个bug 是因为blackwell不支持吗？我看4090/A100有跑起来的

[JaredforReal]

尝试从源码安装吧
uv pip install git+https://github.com/huggingface/transformers.git

[wonjerry]

I’m experiencing the same issue. I already installed transformers from the GitHub main branch, but I’m still getting the same error: (uv pip install git+https://github.com/huggingface/transformers.git)

ValueError: There is no module or parameter named 'model.layers.47'

[wonjerry]

I think this issue is related to vllm-project/vllm#32637

[JaredforReal]

vLLM and Transformers are both going through major updates right now. The community is busy working on the fixes and adaptations, so please bear with us for a bit.
@wonjerry @blairyeung

[wonjerry]

@JaredforReal I always appreciate the community and have been using it well. Please share the updates once the fixes are applied. Thank you!

[blairyeung]

vLLM and Transformers are both going through major updates right now. The community is busy working on the fixes and adaptations, so please bear with us for a bit. @wonjerry @blairyeung

thx, tried this but got the same error message :(

[wangyi7099]

I have this error

ValueError: There is no module or parameter named 'model.layers.1.mlp.gate.e_score_correction_bias' in TransformersMoEForCausalLM