[Bugfix][XPU] Fix Mamba state pointer overflow by Oxygen56 · Pull Request #48109 · vllm-project/vllm

Oxygen56 · 2026-07-09T09:26:30Z

Purpose

XPU USM device pointers can exceed the positive signed int64 range. The Mamba
align-mode fused copy path stores state tensor and block-table data_ptr()
values in torch.int64 metadata buffers before passing them to Triton kernels,
so assigning a high-bit XPU pointer directly can raise:

ValueError: Overflow when unpacking long long

This preserves the 64-bit pointer pattern by reinterpreting unsigned pointer
values as signed int64 before storing them in the existing metadata buffers. The
Triton kernel ABI stays unchanged.

Duplicate-work check: no open PR currently references #48059 or matches the
state_base_addrs / overflow keywords.

Test Plan

pre-commit run ruff-format --files vllm/v1/worker/mamba_utils.py tests/v1/worker/test_mamba_utils.py
pre-commit run ruff-check --files vllm/v1/worker/mamba_utils.py tests/v1/worker/test_mamba_utils.py
python -m pytest tests/v1/worker/test_mamba_utils.py -q

Test Result

ruff-format passed
ruff-check passed
tests/v1/worker/test_mamba_utils.py: 3 passed, 17 skipped

I could not run the full XPU serving repro locally because this environment does
not have the reported Intel Arc/XPU setup and hybrid checkpoint.

Essential Elements of an Effective PR Description Checklist

The purpose of the PR, such as "Fix some issue (link existing issues this PR will resolve)".
The test plan, such as providing test command.
The test results, such as pasting the results comparison before and after, or e2e results
(Optional) The necessary documentation update, such as updating supported_models.md and examples for a new model.

claude

Claude Code Review

This pull request is from a fork — automated review is disabled. A repository maintainer can comment @claude review to run a one-time review.

Hexide · 2026-07-25T18:40:11Z

I had ValueError: Overflow when unpacking long long error on first prompt when MTP was enabled on XPU.
This changes in this merge request fixed issue for me.

Before

(APIServer pid=1) INFO:     Started server process [1[]
(APIServer pid=1) INFO:     Waiting for application startup.
(APIServer pid=1) INFO:     Application startup complete.
(APIServer pid=1) INFO:     10.0.0.79:36350 - "GET /v1/models HTTP/1.1" 200 OK
(APIServer pid=1) INFO:     10.0.0.79:35448 - "GET /v1/models HTTP/1.1" 200 OK
(APIServer pid=1) INFO:     10.0.0.79:35452 - "POST /v1/chat/completions HTTP/1.1" 200 OK
(Worker_TP0 pid=783) WARNING 07-25 17:50:30 [jit_monitor.py:135[] Triton kernel JIT compilation during inference: _compute_slot_mapping_kernel. This causes a latency spike; consider extending warmup to cover this shape/config.
(Worker_TP0 pid=783) ERROR 07-25 17:50:33 [multiproc_executor.py:1007[]     output = func(*args, **kwargs)
(Worker_TP0 pid=783) ERROR 07-25 17:50:33 [multiproc_executor.py:1007[]              ^^^^^^^^^^^^^^^^^^^^^
(Worker_TP0 pid=783) ERROR 07-25 17:50:33 [multiproc_executor.py:1007[]   File "/opt/venv/lib/python3.12/site-packages/torch/utils/_contextlib.py", line 124, in decorate_context
(Worker_TP0 pid=783) ERROR 07-25 17:50:33 [multiproc_executor.py:1007[]     return func(*args, **kwargs)
(Worker_TP0 pid=783) ERROR 07-25 17:50:33 [multiproc_executor.py:1007[]            ^^^^^^^^^^^^^^^^^^^^^
(Worker_TP0 pid=783) ERROR 07-25 17:50:33 [multiproc_executor.py:1007[]   File "/opt/venv/lib/python3.12/site-packages/vllm/v1/worker/gpu_worker.py", line 1083, in sample_tokens
(Worker_TP0 pid=783) ERROR 07-25 17:50:33 [multiproc_executor.py:1007[]     return self.model_runner.sample_tokens(grammar_output)
(Worker_TP0 pid=783) ERROR 07-25 17:50:33 [multiproc_executor.py:1007[]            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
(Worker_TP0 pid=783) ERROR 07-25 17:50:33 [multiproc_executor.py:1007[]   File "/opt/venv/lib/python3.12/site-packages/torch/utils/_contextlib.py", line 124, in decorate_context
(Worker_TP0 pid=783) ERROR 07-25 17:50:33 [multiproc_executor.py:1007[]     return func(*args, **kwargs)
(Worker_TP0 pid=783) ERROR 07-25 17:50:33 [multiproc_executor.py:1007[]            ^^^^^^^^^^^^^^^^^^^^^
(Worker_TP0 pid=783) ERROR 07-25 17:50:33 [multiproc_executor.py:1007[]   File "/opt/venv/lib/python3.12/site-packages/vllm/v1/worker/gpu_model_runner.py", line 4535, in sample_tokens
(Worker_TP0 pid=783) ERROR 07-25 17:50:33 [multiproc_executor.py:1007[]     self._update_states_after_model_execute(
(Worker_TP0 pid=783) ERROR 07-25 17:50:33 [multiproc_executor.py:1007[]   File "/opt/venv/lib/python3.12/site-packages/vllm/v1/worker/gpu_model_runner.py", line 1570, in _update_states_after_model_execute
(Worker_TP0 pid=783) ERROR 07-25 17:50:33 [multiproc_executor.py:1007[]     mamba_utils.postprocess_mamba_align_gpu(
(Worker_TP0 pid=783) ERROR 07-25 17:50:33 [multiproc_executor.py:1007[]   File "/opt/venv/lib/python3.12/site-packages/vllm/v1/worker/mamba_utils.py", line 1131, in postprocess_mamba_align_gpu
(Worker_TP0 pid=783) ERROR 07-25 17:50:33 [multiproc_executor.py:1007[]     ctx.initialize_from_forward_context(
(Worker_TP0 pid=783) ERROR 07-25 17:50:33 [multiproc_executor.py:1007[]   File "/opt/venv/lib/python3.12/site-packages/vllm/v1/worker/mamba_utils.py", line 649, in initialize_from_forward_context
(Worker_TP0 pid=783) ERROR 07-25 17:50:33 [multiproc_executor.py:1007[]     self.state_base_addrs[idx[] = state.data_ptr()
(Worker_TP0 pid=783) ERROR 07-25 17:50:33 [multiproc_executor.py:1007[]     ~~~~~~~~~~~~~~~~~~~~~^^^^^
(Worker_TP0 pid=783) ERROR 07-25 17:50:33 [multiproc_executor.py:1007[] ValueError: Overflow when unpacking long long
(Worker_TP0 pid=783) ERROR 07-25 17:50:33 [multiproc_executor.py:1007[] Traceback (most recent call last):
(Worker_TP0 pid=783) ERROR 07-25 17:50:33 [multiproc_executor.py:1007[]   File "/opt/venv/lib/python3.12/site-packages/vllm/v1/executor/multiproc_executor.py", line 999, in worker_busy_loop
(Worker_TP0 pid=783) ERROR 07-25 17:50:33 [multiproc_executor.py:1007[]     output = func(*args, **kwargs)
(Worker_TP0 pid=783) ERROR 07-25 17:50:33 [multiproc_executor.py:1007[]              ^^^^^^^^^^^^^^^^^^^^^
(Worker_TP0 pid=783) ERROR 07-25 17:50:33 [multiproc_executor.py:1007[]   File "/opt/venv/lib/python3.12/site-packages/torch/utils/_contextlib.py", line 124, in decorate_context
(Worker_TP0 pid=783) ERROR 07-25 17:50:33 [multiproc_executor.py:1007[]     return func(*args, **kwargs)
(Worker_TP0 pid=783) ERROR 07-25 17:50:33 [multiproc_executor.py:1007[]            ^^^^^^^^^^^^^^^^^^^^^
(Worker_TP0 pid=783) ERROR 07-25 17:50:33 [multiproc_executor.py:1007[]   File "/opt/venv/lib/python3.12/site-packages/vllm/v1/worker/gpu_worker.py", line 1083, in sample_tokens
(Worker_TP0 pid=783) ERROR 07-25 17:50:33 [multiproc_executor.py:1007[]     return self.model_runner.sample_tokens(grammar_output)
(Worker_TP0 pid=783) ERROR 07-25 17:50:33 [multiproc_executor.py:1007[]            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
(Worker_TP0 pid=783) ERROR 07-25 17:50:33 [multiproc_executor.py:1007[]   File "/opt/venv/lib/python3.12/site-packages/torch/utils/_contextlib.py", line 124, in decorate_context
(Worker_TP0 pid=783) ERROR 07-25 17:50:33 [multiproc_executor.py:1007[]     return func(*args, **kwargs)
(Worker_TP0 pid=783) ERROR 07-25 17:50:33 [multiproc_executor.py:1007[]            ^^^^^^^^^^^^^^^^^^^^^
(Worker_TP0 pid=783) ERROR 07-25 17:50:33 [multiproc_executor.py:1007[]   File "/opt/venv/lib/python3.12/site-packages/vllm/v1/worker/gpu_model_runner.py", line 4535, in sample_tokens
(Worker_TP0 pid=783) ERROR 07-25 17:50:33 [multiproc_executor.py:1007[]     self._update_states_after_model_execute(
(Worker_TP0 pid=783) ERROR 07-25 17:50:33 [multiproc_executor.py:1007[]   File "/opt/venv/lib/python3.12/site-packages/vllm/v1/worker/gpu_model_runner.py", line 1570, in _update_states_after_model_execute
(Worker_TP0 pid=783) ERROR 07-25 17:50:33 [multiproc_executor.py:1007[]     mamba_utils.postprocess_mamba_align_gpu(
(Worker_TP0 pid=783) ERROR 07-25 17:50:33 [multiproc_executor.py:1007[]   File "/opt/venv/lib/python3.12/site-packages/vllm/v1/worker/mamba_utils.py", line 1131, in postprocess_mamba_align_gpu
(Worker_TP0 pid=783) ERROR 07-25 17:50:33 [multiproc_executor.py:1007[]     ctx.initialize_from_forward_context(
(Worker_TP0 pid=783) ERROR 07-25 17:50:33 [multiproc_executor.py:1007[]   File "/opt/venv/lib/python3.12/site-packages/vllm/v1/worker/mamba_utils.py", line 649, in initialize_from_forward_context
(Worker_TP0 pid=783) ERROR 07-25 17:50:33 [multiproc_executor.py:1007[]     self.state_base_addrs[idx[] = state.data_ptr()
(Worker_TP0 pid=783) ERROR 07-25 17:50:33 [multiproc_executor.py:1007[]     ~~~~~~~~~~~~~~~~~~~~~^^^^^
(Worker_TP0 pid=783) ERROR 07-25 17:50:33 [multiproc_executor.py:1007[] ValueError: Overflow when unpacking long long

After:

(APIServer pid=1) INFO:     Started server process [1[]
(APIServer pid=1) INFO:     Waiting for application startup.
(APIServer pid=1) INFO:     Application startup complete.
(APIServer pid=1) INFO:     10.0.0.79:47226 - "GET /v1/models HTTP/1.1" 200 OK
(APIServer pid=1) INFO:     10.0.0.79:47226 - "POST /v1/chat/completions HTTP/1.1" 200 OK
(Worker_TP0 pid=783) WARNING 07-25 18:34:56 [jit_monitor.py:135[] Triton kernel JIT compilation during inference: _compute_slot_mapping_kernel. This causes a latency spike; consider extending warmup to cover this shape/config.
(Worker_TP0 pid=783) WARNING 07-25 18:35:00 [jit_monitor.py:135[] Triton kernel JIT compilation during inference: postprocess_mamba_fused_kernel. This causes a latency spike; consider extending warmup to cover this shape/config.
(Worker_TP0 pid=783) WARNING 07-25 18:35:03 [jit_monitor.py:135[] Triton kernel JIT compilation during inference: eagle_prepare_next_token_padded_kernel. This causes a latency spike; consider extending warmup to cover this shape/config.
(Worker_TP0 pid=783) WARNING 07-25 18:35:06 [jit_monitor.py:135[] Triton kernel JIT compilation during inference: eagle_step_slot_mapping_metadata_kernel. This causes a latency spike; consider extending warmup to cover this shape/config.
(Worker_TP0 pid=783) WARNING 07-25 18:35:10 [jit_monitor.py:135[] Triton kernel JIT compilation during inference: fused_moe_kernel. This causes a latency spike; consider extending warmup to cover this shape/config.
(Worker_TP0 pid=783) WARNING 07-25 18:35:16 [jit_monitor.py:135[] Triton kernel JIT compilation during inference: batch_memcpy_kernel. This causes a latency spike; consider extending warmup to cover this shape/config.
(APIServer pid=1) INFO:     10.0.0.79:52838 - "GET /v1/models HTTP/1.1" 200 OK
(Worker_TP0 pid=783) WARNING 07-25 18:35:26 [jit_monitor.py:135[] Triton kernel JIT compilation during inference: expand_kernel. This causes a latency spike; consider extending warmup to cover this shape/config.
(Worker_TP0 pid=783) WARNING 07-25 18:35:29 [jit_monitor.py:135[] Triton kernel JIT compilation during inference: eagle_prepare_inputs_padded_kernel. This causes a latency spike; consider extending warmup to cover this shape/config.
(APIServer pid=1) INFO 07-25 18:35:34 [loggers.py:310[] Engine 000: Avg prompt throughput: 247.9 tokens/s, Avg generation throughput: 0.8 tokens/s, Running: 1 reqs, Waiting: 0 reqs, GPU KV cache usage: 1.9%, Prefix cache hit rate: 0.0%

mayuyuace · 2026-07-28T06:06:44Z

@jikunshang

urakozz · 2026-07-28T19:04:27Z

And now there is a conflict with freshly merged #48110 😐

mayuyuace · 2026-07-29T06:55:32Z

@Oxygen56
Can you rebase your branch?

Oxygen56 · 2026-08-01T14:51:57Z

Rebased onto the latest main.

jikunshang · 2026-08-06T03:18:42Z

/ci run

github-actions · 2026-08-06T03:19:16Z

✅ Triggered Buildkite CI #82608 for commit 5133492ec356.

jikunshang · 2026-08-06T04:41:19Z

/ci retry

github-actions · 2026-08-06T04:41:35Z

✅ Queued 1 failed job(s) for retry in Buildkite CI #82608.

mergify · 2026-08-17T08:03:39Z

This pull request has merge conflicts that must be resolved before it can be
merged. Please rebase the PR, @Oxygen56.

https://docs.github.com/en/pull-requests/collaborating-with-pull-requests/working-with-forks/syncing-a-fork

Signed-off-by: Oxygen56 <jiangth99@163.com>

mayuyuace · 2026-08-18T03:56:25Z

/ci run

github-actions · 2026-08-18T03:56:45Z

❌ @mayuyuace, Only reviewers with write access can use CI commands before CI is delegated to the PR author.

mayuyuace · 2026-08-18T08:26:49Z

/ci run

github-actions · 2026-08-18T08:27:40Z

❌ @mayuyuace, Only reviewers with write access can use CI commands before CI is delegated to the PR author.

jikunshang · 2026-08-18T08:33:26Z

/ci run

github-actions · 2026-08-18T08:33:45Z

✅ Triggered Buildkite CI #84369 for commit 8d88c60b8085.

Signed-off-by: Oxygen56 <jiangth99@163.com> Signed-off-by: Wenhua Cheng <wenhua.cheng@intel.com>

Signed-off-by: Oxygen56 <jiangth99@163.com>

Signed-off-by: Oxygen56 <jiangth99@163.com> Signed-off-by: Wyett <wyettzeng@gmail.com>

Signed-off-by: Oxygen56 <jiangth99@163.com> Signed-off-by: Zhu, Zufang <zufang.zhu@intel.com>

Signed-off-by: Oxygen56 <jiangth99@163.com> Signed-off-by: khushali9 <khushali.desai9@gmail.com>

Signed-off-by: Oxygen56 <jiangth99@163.com>

Oxygen56 requested a review from njhill as a code owner July 9, 2026 09:26

claude Bot reviewed Jul 9, 2026

View reviewed changes

mergify Bot added intel-gpu Related to Intel GPU v1 bug Something isn't working labels Jul 9, 2026

mayuyuace approved these changes Jul 28, 2026

View reviewed changes

jikunshang added the ready ONLY add when PR is ready to merge/full CI is needed label Jul 28, 2026

jikunshang removed the ready ONLY add when PR is ready to merge/full CI is needed label Jul 29, 2026

mayuyuace mentioned this pull request Jul 31, 2026

[Bugfix][XPU] Fix Mamba state pointer overflow #50552

Closed

Oxygen56 force-pushed the fix/xpu-mamba-state-ptr-overflow branch from af11afb to 89b52dd Compare August 1, 2026 14:51

jikunshang approved these changes Aug 6, 2026

View reviewed changes

jikunshang added the verified Run pre-commit for new contributors without triggering other tests label Aug 6, 2026

swakhandekar mentioned this pull request Aug 17, 2026

[XPU][Bugfix] Fix XPU crash with speculative decoding + prefix caching in Mamba models #52186

Closed

4 tasks

mergify Bot added the needs-rebase label Aug 17, 2026

Oxygen56 added 2 commits August 17, 2026 17:57

[Bugfix][XPU] Fix Mamba state pointer overflow

634ee21

Signed-off-by: Oxygen56 <jiangth99@163.com>

test: keep high temporal pointer aligned

8d88c60

Signed-off-by: Oxygen56 <jiangth99@163.com>

Oxygen56 force-pushed the fix/xpu-mamba-state-ptr-overflow branch from 5133492 to 8d88c60 Compare August 17, 2026 09:59

mergify Bot removed the needs-rebase label Aug 17, 2026

mayuyuace mentioned this pull request Aug 18, 2026

[Bug] GDN causal_conv1d rejects mixed spec/non-spec batches — MTP unusable under concurrency vllm-project/vllm-xpu-kernels#510

Closed

jikunshang approved these changes Aug 18, 2026

View reviewed changes

jikunshang merged commit f936a26 into vllm-project:main Aug 19, 2026
88 checks passed

wenhuach21 pushed a commit to wenhuach21/vllm that referenced this pull request Aug 19, 2026

[Bugfix][XPU] Fix Mamba state pointer overflow (vllm-project#48109)

e55c7ec

Signed-off-by: Oxygen56 <jiangth99@163.com> Signed-off-by: Wenhua Cheng <wenhua.cheng@intel.com>

wolfallein mentioned this pull request Aug 19, 2026

[Bug][XPU] Mamba align-mode prefix caching crashes: "Overflow when unpacking long long" storing state.data_ptr() #48059

Closed

zyp2014 pushed a commit to zyp2014/vllm that referenced this pull request Aug 21, 2026

[Bugfix][XPU] Fix Mamba state pointer overflow (vllm-project#48109)

1e6f486

Signed-off-by: Oxygen56 <jiangth99@163.com>

wyettzeng pushed a commit to wyettzeng/vllm that referenced this pull request Aug 21, 2026

[Bugfix][XPU] Fix Mamba state pointer overflow (vllm-project#48109)

933c902

Signed-off-by: Oxygen56 <jiangth99@163.com> Signed-off-by: Wyett <wyettzeng@gmail.com>

zufangzhu pushed a commit to zufangzhu/vllm that referenced this pull request Aug 24, 2026

[Bugfix][XPU] Fix Mamba state pointer overflow (vllm-project#48109)

3aba69b

Signed-off-by: Oxygen56 <jiangth99@163.com> Signed-off-by: Zhu, Zufang <zufang.zhu@intel.com>

Hoborific mentioned this pull request Aug 27, 2026

[Bug]: [XPU] RuntimeError: causal_conv1d does not support spec-decode and non-spec tokens in the same invocation (MTP + chunked prefill) #53928

Open

1 task

khushali9 pushed a commit to khushali9/vllm that referenced this pull request Aug 29, 2026

[Bugfix][XPU] Fix Mamba state pointer overflow (vllm-project#48109)

156873f

Signed-off-by: Oxygen56 <jiangth99@163.com> Signed-off-by: khushali9 <khushali.desai9@gmail.com>

am-cohere pushed a commit to am-cohere/vllm that referenced this pull request Sep 1, 2026

[Bugfix][XPU] Fix Mamba state pointer overflow (vllm-project#48109)

1833d2c

Signed-off-by: Oxygen56 <jiangth99@163.com>

gc-fu mentioned this pull request Sep 3, 2026

0.26.0-b1: every request dies with ValueError: Overflow when unpacking long long in postprocess_mamba_align_gpu (XPU, hybrid GDN + prefix caching) intel/llm-scaler#667

Closed

pmanczak mentioned this pull request Sep 7, 2026

[Bugfix][XPU] Fix KDA RecoverSSM device-pointer overflow #55730

Draft

Uh oh!

Uh oh!

Conversation

Oxygen56 commented Jul 9, 2026 • edited by github-actions Bot Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Purpose

Test Plan

Test Result

Uh oh!

claude Bot left a comment

Choose a reason for hiding this comment

Claude Code Review

Uh oh!

Hexide commented Jul 25, 2026

Uh oh!

mayuyuace commented Jul 28, 2026

Uh oh!

urakozz commented Jul 28, 2026

Uh oh!

mayuyuace commented Jul 29, 2026

Uh oh!

Oxygen56 commented Aug 1, 2026

Uh oh!

jikunshang commented Aug 6, 2026

Uh oh!

github-actions Bot commented Aug 6, 2026

Uh oh!

jikunshang commented Aug 6, 2026

Uh oh!

github-actions Bot commented Aug 6, 2026

Uh oh!

mergify Bot commented Aug 17, 2026

Uh oh!

mayuyuace commented Aug 18, 2026

Uh oh!

github-actions Bot commented Aug 18, 2026

Uh oh!

mayuyuace commented Aug 18, 2026

Uh oh!

github-actions Bot commented Aug 18, 2026

Uh oh!

jikunshang commented Aug 18, 2026

Uh oh!

github-actions Bot commented Aug 18, 2026

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

5 participants

Oxygen56 commented Jul 9, 2026 •

edited by github-actions Bot

Loading