[Bug]: Run error GLM-4-32B-0414

[SongXiaoMao]

Your current environment

ERROR 04-16 08:38:03 [core.py:387] RuntimeError: ('Worker failed with error %s, please check the stack trace above for the root cause', 'TypeError : linear(): argument 'input' (position 1) must be Tensor, not tuple\n\nfrom user code:\n File "/home/cheng/anaconda3/envs/VLLM/lib/python3.12/site-packages/vllm/model_executor/models/llama.py", line 360, in forward\n hidden_states, residual = layer(positions, hidden_states, residual)\n File "/home/cheng/anaconda3/envs/VLLM/lib/python3.12/site-packages/vllm/model_executor/models/glm4.py", line 204, in forward\n hidden_states = self.mlp(hidden_states)\n File "/home/cheng/anaconda3/envs/VLLM/lib/python3.12/site-packages/vllm/model_executor/models/llama.py", line 92, in forward\n x, _ = self.gate_up_proj(x)\n File "/home/cheng/anaconda3/envs/VLLM/lib/python3.12/site-packages/vllm/model_executor/layers/linear.py", line 474, in forward\n output_parallel = self.quant_method.apply(self, input_, bias)\n File "/home/cheng/anaconda3/envs/VLLM/lib/python3.12/site-packages/vllm/model_executor/layers/linear.py", line 191, in apply\n return F.linear(x, layer.weight, bias)\n\nSet TORCH_LOGS="+dynamo" and TORCHDYNAMO_VERBOSE=1 for more information\n\n\nYou can suppress this exception and fall back to eager by setting:\n import torch._dynamo\n torch._dynamo.config.suppress_errors = True\n')
ERROR 04-16 08:38:03 [core.py:387]
CRITICAL 04-16 08:38:03 [core_client.py:359] Got fatal signal from worker processes, shutting down. See stack trace above for root cause issue.
已杀死

🐛 Describe the bug

ERROR 04-16 08:38:03 [core.py:387] RuntimeError: ('Worker failed with error %s, please check the stack trace above for the root cause', 'TypeError : linear(): argument 'input' (position 1) must be Tensor, not tuple\n\nfrom user code:\n File "/home/cheng/anaconda3/envs/VLLM/lib/python3.12/site-packages/vllm/model_executor/models/llama.py", line 360, in forward\n hidden_states, residual = layer(positions, hidden_states, residual)\n File "/home/cheng/anaconda3/envs/VLLM/lib/python3.12/site-packages/vllm/model_executor/models/glm4.py", line 204, in forward\n hidden_states = self.mlp(hidden_states)\n File "/home/cheng/anaconda3/envs/VLLM/lib/python3.12/site-packages/vllm/model_executor/models/llama.py", line 92, in forward\n x, _ = self.gate_up_proj(x)\n File "/home/cheng/anaconda3/envs/VLLM/lib/python3.12/site-packages/vllm/model_executor/layers/linear.py", line 474, in forward\n output_parallel = self.quant_method.apply(self, input_, bias)\n File "/home/cheng/anaconda3/envs/VLLM/lib/python3.12/site-packages/vllm/model_executor/layers/linear.py", line 191, in apply\n return F.linear(x, layer.weight, bias)\n\nSet TORCH_LOGS="+dynamo" and TORCHDYNAMO_VERBOSE=1 for more information\n\n\nYou can suppress this exception and fall back to eager by setting:\n import torch._dynamo\n torch._dynamo.config.suppress_errors = True\n')
ERROR 04-16 08:38:03 [core.py:387]
CRITICAL 04-16 08:38:03 [core_client.py:359] Got fatal signal from worker processes, shutting down. See stack trace above for root cause issue.
已杀死

Before submitting a new issue...

• Make sure you already searched for relevant issues, and asked the chatbot living at the bottom right corner of the documentation page, which can answer lots of frequently asked questions.

[ZV-Liu]

ERROR 04-22 13:34:41 [core.py:387] EngineCore hit an exception: Traceback (most recent call last):
ERROR 04-22 13:34:41 [core.py:387] File "/usr/local/lib/python3.10/site-packages/vllm/v1/engine/core.py", line 378, in run_engine_core
ERROR 04-22 13:34:41 [core.py:387] engine_core = EngineCoreProc(*args, **kwargs)
ERROR 04-22 13:34:41 [core.py:387] File "/usr/local/lib/python3.10/site-packages/vllm/v1/engine/core.py", line 320, in init
ERROR 04-22 13:34:41 [core.py:387] super().init(vllm_config, executor_class, log_stats)
ERROR 04-22 13:34:41 [core.py:387] File "/usr/local/lib/python3.10/site-packages/vllm/v1/engine/core.py", line 71, in init
ERROR 04-22 13:34:41 [core.py:387] self._initialize_kv_caches(vllm_config)
ERROR 04-22 13:34:41 [core.py:387] File "/usr/local/lib/python3.10/site-packages/vllm/v1/engine/core.py", line 133, in initialize_kv_caches
ERROR 04-22 13:34:41 [core.py:387] available_gpu_memory = self.model_executor.determine_available_memory()
ERROR 04-22 13:34:41 [core.py:387] File "/usr/local/lib/python3.10/site-packages/vllm/v1/executor/abstract.py", line 66, in determine_available_memory
ERROR 04-22 13:34:41 [core.py:387] output = self.collective_rpc("determine_available_memory")
ERROR 04-22 13:34:41 [core.py:387] File "/usr/local/lib/python3.10/site-packages/vllm/v1/executor/multiproc_executor.py", line 133, in collective_rpc
ERROR 04-22 13:34:41 [core.py:387] raise e
ERROR 04-22 13:34:41 [core.py:387] File "/usr/local/lib/python3.10/site-packages/vllm/v1/executor/multiproc_executor.py", line 122, in collective_rpc
ERROR 04-22 13:34:41 [core.py:387] raise RuntimeError(
ERROR 04-22 13:34:41 [core.py:387] RuntimeError: ('Worker failed with error %s, please check the stack trace above for the root cause', 'TypeError : linear(): argument 'input' (position 1) must be Tensor, not tuple\n\nfrom user code:\n File "/usr/local/lib/python3.10/site-packages/vllm/model_executor/models/llama.py", line 360, in forward\n hidden_states, residual = layer(positions, hidden_states, residual)\n File "/usr/local/lib/python3.10/site-packages/vllm/model_executor/models/glm4.py", line 204, in forward\n hidden_states = self.mlp(hidden_states)\n File "/usr/local/lib/python3.10/site-packages/vllm/model_executor/models/llama.py", line 92, in forward\n x, _ = self.gate_up_proj(x)\n File "/usr/local/lib/python3.10/site-packages/vllm/model_executor/layers/linear.py", line 474, in forward\n output_parallel = self.quant_method.apply(self, input, bias)\n File "/usr/local/lib/python3.10/site-packages/vllm/model_executor/layers/linear.py", line 191, in apply\n return F.linear(x, layer.weight, bias)\n\nSet TORCH_LOGS="+dynamo" and TORCHDYNAMO_VERBOSE=1 for more information\n\n\nYou can suppress this exception and fall back to eager by setting:\n import torch._dynamo\n torch._dynamo.config.suppress_errors = True\n')
ERROR 04-22 13:34:41 [core.py:387]
CRITICAL 04-22 13:34:41 [core_client.py:359] Got fatal signal from worker processes, shutting down. See stack trace above for root cause issue.
Killed

[ccdlhj]

+1

[ZV-Liu]

已解决，vllm升级到0.8.4，并替换最新的glm4.py

[Humputer]

已解决，vllm升级到0.8.4，并替换最新的glm4.py

我的vllm/executor/目录下没有models文件夹

[ZV-Liu]

需要升级到0.8.4

[Humputer]

已解决，vllm升级到0.8.4，并替换最新的glm4.py

我的vllm/executor/目录下没有models文件夹

是0.8.4，是verl镜像的，是在vllm/model_executor/models/，多谢

[fernando-neto-ai]

+1