Fix glm4.py residual bug

[solrex]

Fix residual handling in Glm4DecoderLayer according to https://github.com/huggingface/transformers/blob/main/src/transformers/models/glm4/modeling_glm4.py .

The bug will cause the GLM-4 inference to fail, with the failure reason being:

ERROR 04-15 16:20:02 [core.py:387] EngineCore hit an exception: Traceback (most recent call last):
ERROR 04-15 16:20:02 [core.py:387]   File "/workspace/venv-glm4/lib/python3.10/site-packages/torch/_dynamo/utils.py", line 2586, in run_node
ERROR 04-15 16:20:02 [core.py:387]     return node.target(*args, **kwargs)
ERROR 04-15 16:20:02 [core.py:387] TypeError: linear(): argument 'input' (position 1) must be Tensor, not tuple
ERROR 04-15 16:20:02 [core.py:387] 
ERROR 04-15 16:20:02 [core.py:387] The above exception was the direct cause of the following exception:
ERROR 04-15 16:20:02 [core.py:387] 
ERROR 04-15 16:20:02 [core.py:387] Traceback (most recent call last):
ERROR 04-15 16:20:02 [core.py:387]   File "/workspace/venv-glm4/lib/python3.10/site-packages/torch/_dynamo/utils.py", line 2471, in get_fake_value
ERROR 04-15 16:20:02 [core.py:387]     ret_val = wrap_fake_exception(
ERROR 04-15 16:20:02 [core.py:387]   File "/workspace/venv-glm4/lib/python3.10/site-packages/torch/_dynamo/utils.py", line 2017, in wrap_fake_exception
ERROR 04-15 16:20:02 [core.py:387]     return fn()
ERROR 04-15 16:20:02 [core.py:387]   File "/workspace/venv-glm4/lib/python3.10/site-packages/torch/_dynamo/utils.py", line 2472, in <lambda>
ERROR 04-15 16:20:02 [core.py:387]     lambda: run_node(tx.output, node, args, kwargs, nnmodule)
ERROR 04-15 16:20:02 [core.py:387]   File "/workspace/venv-glm4/lib/python3.10/site-packages/torch/_dynamo/utils.py", line 2604, in run_node
ERROR 04-15 16:20:02 [core.py:387]     raise RuntimeError(make_error_message(e)).with_traceback(
ERROR 04-15 16:20:02 [core.py:387]   File "/workspace/venv-glm4/lib/python3.10/site-packages/torch/_dynamo/utils.py", line 2586, in run_node
ERROR 04-15 16:20:02 [core.py:387]     return node.target(*args, **kwargs)
ERROR 04-15 16:20:02 [core.py:387] RuntimeError: Failed running call_function <built-in function linear>(*((FakeTensor(..., device='cuda:0', size=(s0, 4096), dtype=torch.bfloat16), FakeTensor(..., device='cuda:0', size=(s0, 4096), dtype=torch.bfloat16)), Parameter(FakeTensor(..., device='cuda:0', size=(27392, 4096), dtype=torch.bfloat16)), None), **{}):
ERROR 04-15 16:20:02 [core.py:387] linear(): argument 'input' (position 1) must be Tensor, not tuple
ERROR 04-15 16:20:02 [core.py:387] 
ERROR 04-15 16:20:02 [core.py:387] During handling of the above exception, another exception occurred:
...
ERROR 04-15 16:20:02 [core.py:387] from user code:
ERROR 04-15 16:20:02 [core.py:387]    File "/workspace/venv-glm4/lib/python3.10/site-packages/vllm/model_executor/models/llama.py", line 360, in forward
ERROR 04-15 16:20:02 [core.py:387]     hidden_states, residual = layer(positions, hidden_states, residual)
ERROR 04-15 16:20:02 [core.py:387]   File "/workspace/venv-glm4/lib/python3.10/site-packages/vllm/model_executor/models/glm4.py", line 204, in forward
ERROR 04-15 16:20:02 [core.py:387]     hidden_states = self.mlp(hidden_states)
ERROR 04-15 16:20:02 [core.py:387]   File "/workspace/venv-glm4/lib/python3.10/site-packages/vllm/model_executor/models/llama.py", line 92, in forward
ERROR 04-15 16:20:02 [core.py:387]     x, _ = self.gate_up_proj(x)
ERROR 04-15 16:20:02 [core.py:387]   File "/workspace/venv-glm4/lib/python3.10/site-packages/vllm/model_executor/layers/linear.py", line 474, in forward
ERROR 04-15 16:20:02 [core.py:387]     output_parallel = self.quant_method.apply(self, input_, bias)
ERROR 04-15 16:20:02 [core.py:387]   File "/workspace/venv-glm4/lib/python3.10/site-packages/vllm/model_executor/layers/linear.py", line 191, in apply
ERROR 04-15 16:20:02 [core.py:387]     return F.linear(x, layer.weight, bias)

[github-actions]

👋 Hi! Thank you for contributing to the vLLM project.

💬 Join our developer Slack at https://slack.vllm.ai to discuss your PR in #pr-reviews, coordinate on features in #feat- channels, or join special interest groups in #sig- channels.

Just a reminder: PRs would not trigger full CI run by default. Instead, it would only run fastcheck CI which starts running only a small and essential subset of CI tests to quickly catch errors. You can run other CI tests on top of those by going to your fastcheck build on Buildkite UI (linked in the PR checks section) and unblock them. If you do not have permission to unblock, ping simon-mo or khluu to add you in our Buildkite org.

Once the PR is approved and ready to go, your PR reviewer(s) can run CI to test the changes comprehensively before merging.

To run CI, PR reviewers can either: Add ready label to the PR or enable auto-merge.

🚀

[jeejeelee]

Thank you for your contribution, #16618 is working on fixing this