Skip to content

[None][feat] Enable 2 DSv4 perf optimizations by default #14120

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
lfr-0531 merged 1 commit into from
May 18, 2026
Merged

[None][feat] Enable 2 DSv4 perf optimizations by default #14120

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion tensorrt_llm/_torch/custom_ops/torch_custom_ops.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -1530,7 +1530,7 @@ def _(
# deep_gemm_gen_tuning_buckets is imported from ..utils

_USE_FUSED_FP8_QUANT_PACK = os.environ.get("TRTLLM_FUSED_FP8_QUANT_PACK",
"0") == "1"
"1") == "1"


def _fp8_quantize_1x128_ue8m0(input: torch.Tensor, tactic: int):
Expand Down
2 changes: 1 addition & 1 deletion tensorrt_llm/_torch/modules/attention.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -2195,7 +2195,7 @@ def forward_impl_with_deepseek_v4(self,
# with q_b_proj + q_b_layernorm. The indexer is launched on a
# dedicated stream and still uses a different aux stream for its
# internal q-proj/weights-proj split.
_v4_extra_overlap = (os.environ.get("TRTLLM_MLA_EXTRA_OVERLAP", "0")
_v4_extra_overlap = (os.environ.get("TRTLLM_MLA_EXTRA_OVERLAP", "1")
== "1" and self.compressor is not None
and self.aux_stream is not None)

Expand Down
Loading