Skip to content

Conversation

@Chen-0210
Copy link
Contributor

Motivation

When the logprobs key is included in the request for Qwen2-VL, the server crashes because input_ids is set to None.
The purpose of this line of code is unclear, and it was introduced in PR #2977.

  File "/usr/local/lib/python3.10/dist-packages/sglang/srt/managers/scheduler.py", line 1827, in run_scheduler_process
    scheduler.event_loop_normal()
  File "/usr/local/lib/python3.10/dist-packages/torch/utils/_contextlib.py", line 116, in decorate_context
    return func(*args, **kwargs)
  File "/usr/local/lib/python3.10/dist-packages/sglang/srt/managers/scheduler.py", line 478, in event_loop_normal
    result = self.run_batch(batch)
  File "/usr/local/lib/python3.10/dist-packages/sglang/srt/managers/scheduler.py", line 1080, in run_batch
    logits_output, next_token_ids = self.tp_worker.forward_batch_generation(
  File "/usr/local/lib/python3.10/dist-packages/sglang/srt/managers/tp_worker.py", line 164, in forward_batch_generation
    logits_output = self.model_runner.forward(forward_batch)
  File "/usr/local/lib/python3.10/dist-packages/sglang/srt/model_executor/model_runner.py", line 795, in forward
    return self.forward_extend(forward_batch)
  File "/usr/local/lib/python3.10/dist-packages/sglang/srt/model_executor/model_runner.py", line 760, in forward_extend
    return self.model.forward(
  File "/usr/local/lib/python3.10/dist-packages/sglang/srt/models/qwen2_vl.py", line 570, in forward
    return self.logits_processor(
  File "/usr/local/lib/python3.10/dist-packages/torch/nn/modules/module.py", line 1736, in _wrapped_call_impl
    return self._call_impl(*args, **kwargs)
  File "/usr/local/lib/python3.10/dist-packages/torch/nn/modules/module.py", line 1747, in _call_impl
    return forward_call(*args, **kwargs)
  File "/usr/local/lib/python3.10/dist-packages/sglang/srt/layers/logits_processor.py", line 165, in forward
    pruned_input_ids.append(input_ids[pt + start_len : pt + extend_len])
TypeError: 'NoneType' object is not subscriptable

Modifications

Removed the line.

Checklist

@Chen-0210 Chen-0210 changed the title remove input_ids = None remove input_ids = None in qwen2-vl Feb 17, 2025
@zhaochenyang20
Copy link
Collaborator

#3635

@Chen-0210 I combied this in #3635 and will give you credit, thanks for help!

@Chen-0210 Chen-0210 deleted the qwen2-vl-bugfix branch October 23, 2025 11:43
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants