Skip to content

Commit beafd5e

Browse files
MengqingCaoepwalsh
authored andcommitted
[Bugfix][Logprobs] Fix logprobs op to support more backend (vllm-project#21591)
Signed-off-by: MengqingCao <[email protected]>
1 parent 4fc7e93 commit beafd5e

File tree

1 file changed

+3
-1
lines changed

1 file changed

+3
-1
lines changed

vllm/v1/sample/ops/logprobs.py

Lines changed: 3 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -4,8 +4,10 @@
44

55
import torch
66

7+
from vllm.platforms import current_platform
78

8-
@torch.compile(dynamic=True)
9+
10+
@torch.compile(dynamic=True, backend=current_platform.simple_compile_backend)
911
def batched_count_greater_than(x: torch.Tensor,
1012
values: torch.Tensor) -> torch.Tensor:
1113
"""

0 commit comments

Comments
 (0)