[Bug Fix] Fix naive_block_assignment always defaulting to False due to arg misalignment#33848
Conversation
Signed-off-by: Runkai Tao <rt572@physics.rutgers.edu>
There was a problem hiding this comment.
Code Review
This pull request correctly fixes a bug where naive_block_assignment was always defaulting to False due to a positional argument being misaligned. The fix involves changing the function call to use a keyword argument, which is the right approach. I've added a suggestion to make the function signature more robust to prevent similar issues in the future.
| adapter_enabled: torch.Tensor, | ||
| expert_map: torch.Tensor | None = None, | ||
| pad_sorted_ids: bool = False, | ||
| naive_block_assignment: bool = False, |
There was a problem hiding this comment.
This bug was caused by passing a positional argument that was misinterpreted. To prevent this class of bugs, consider making optional boolean flags keyword-only by adding a * in the function signature before them (e.g., before pad_sorted_ids). This would enforce keyword arguments for all subsequent parameters and would have raised a TypeError for the original buggy call.
varun-sundar-rabindranath
left a comment
There was a problem hiding this comment.
Thanks for the fix @RunkaiTao
cc @jeejeelee
|
cc @mgoin @robertgshaw2-redhat can you please take a look. Thanks 🙌 |
… to arg misalignment (vllm-project#33848) Signed-off-by: Runkai Tao <rt572@physics.rutgers.edu>
… to arg misalignment (vllm-project#33848) Signed-off-by: Runkai Tao <rt572@physics.rutgers.edu> Signed-off-by: Eldar Kurtic <research@neuralmagic.com>
… to arg misalignment (vllm-project#33848) Signed-off-by: Runkai Tao <rt572@physics.rutgers.edu>
… to arg misalignment (vllm-project#33848) Signed-off-by: Runkai Tao <rt572@physics.rutgers.edu>
Purpose
Fix a bug that
naive_block_assignmentalways defaulting to False due to arg misalignment.Test Result
gpt-oss 120b max_loras=8, concurrency=1
before
after
Essential Elements of an Effective PR Description Checklist
supported_models.mdandexamplesfor a new model.