Skip to content

[https://nvbugs/6329052][fix] Add attn_backend: FLASHINFER and model_kwargs: {num_hidden_layers: 4} to… #15464

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
tensorrt-cicd wants to merge 2 commits into from
Closed

[https://nvbugs/6329052][fix] Add attn_backend: FLASHINFER and model_kwargs: {num_hidden_layers: 4} to… #15464

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
eb0b727
[nvbugs/6329052][fix] Drop deepseek_v3_lite_bf16 disagg-workers test …
tensorrt-cicd Jun 17, 2026
62e70c6
[nvbugs/6329052][fix] Use FLASHINFER + 4 layers for DeepSeek-V3-Lite …
tensorrt-cicd Jun 17, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 3 additions & 0 deletions tests/integration/defs/disaggregated/test_configs/disagg_config_cache_reuse_deepseek_v3.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -2,9 +2,12 @@ hostname: localhost
model: DeepSeek-V3-Lite/bf16
free_gpu_memory_fraction: 0.15
backend: pytorch
attn_backend: FLASHINFER
cuda_graph_config: null
disable_overlap_scheduler: true
enable_autotuner: false
model_kwargs:
num_hidden_layers: 4
Comment on lines +5 to +10

@coderabbitai coderabbitai Bot Jun 17, 2026

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Potential issue | 🟠 Major

🧩 Analysis chain

🏁 Script executed:

#!/bin/bash
# Verify which YAML the conditional test uses, and whether it references cache_reuse config.

set -euo pipefail

echo "== Mapping for deepseek_v3_lite_bf16_conditional =="
rg -n -C3 'deepseek_v3_lite_bf16_conditional|disagg_config_conditional_deepseek_v3\.yaml' \
  tests/integration/defs/disaggregated/test_disaggregated.py

echo
echo "== Does conditional config reference cache_reuse config? =="
rg -n -C3 'disagg_config_cache_reuse_deepseek_v3\.yaml|attn_backend|num_hidden_layers' \
  tests/integration/defs/disaggregated/test_configs/disagg_config_conditional_deepseek_v3.yaml \
  tests/integration/defs/disaggregated/test_configs/disagg_config_cache_reuse_deepseek_v3.yaml

Repository: NVIDIA/TensorRT-LLM

Length of output: 2961

🏁 Script executed:

cat -n tests/integration/defs/disaggregated/test_configs/disagg_config_conditional_deepseek_v3.yaml

Repository: NVIDIA/TensorRT-LLM

Length of output: 1138

Changes applied to incorrect config file; conditional test uses a different configuration.

The modified disagg_config_cache_reuse_deepseek_v3.yaml (lines 5 and 9–10) is not used by test_disaggregated_deepseek_v3_lite_bf16_conditional. That test uses disagg_config_conditional_deepseek_v3.yaml (confirmed at test_disaggregated.py:282–283), which has a completely different structure with no attn_backend or model_kwargs fields.

To fix the failing conditional test, apply the same changes (attn_backend: FLASHINFER and model_kwargs.num_hidden_layers: 4) to tests/integration/defs/disaggregated/test_configs/disagg_config_conditional_deepseek_v3.yaml instead.

🤖 Prompt for AI Agents 
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In
`@tests/integration/defs/disaggregated/test_configs/disagg_config_cache_reuse_deepseek_v3.yaml`
around lines 5 - 10, The configuration changes (attn_backend: FLASHINFER and
model_kwargs.num_hidden_layers: 4) were applied to the wrong file. The test
test_disaggregated_deepseek_v3_lite_bf16_conditional actually uses
disagg_config_conditional_deepseek_v3.yaml, not
disagg_config_cache_reuse_deepseek_v3.yaml. Revert the changes made to
disagg_config_cache_reuse_deepseek_v3.yaml and instead apply the same two
modifications (adding attn_backend: FLASHINFER and setting
model_kwargs.num_hidden_layers to 4) to the
disagg_config_conditional_deepseek_v3.yaml file to fix the failing test.

Source: Coding guidelines

context_servers:
num_instances: 1
tensor_parallel_size: 1
Expand Down
1 change: 0 additions & 1 deletion tests/integration/test_lists/qa/llm_function_core.txt
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -924,7 +924,6 @@ disaggregated/test_disaggregated_single_gpu.py::test_disaggregated_simple_qwen3[
disaggregated/test_disaggregated_single_gpu.py::test_disaggregated_spec_dec_batch_slot_limit[False-False-EAGLE3-LLaMA3.1-Instruct-8B-Llama-3.1-8B-Instruct]
disaggregated/test_disaggregated_single_gpu.py::test_disaggregated_spec_dec_batch_slot_limit[True-False-EAGLE3-LLaMA3.1-Instruct-8B-Llama-3.1-8B-Instruct]
disaggregated/test_workers.py::test_workers_conditional_disaggregation[TinyLlama-1.1B-Chat-v1.0]
disaggregated/test_workers.py::test_workers_conditional_disaggregation_deepseek_v3_lite_bf16[DeepSeek-V3-Lite-bf16]

@StanleySun639 StanleySun639 Jun 24, 2026

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why is this case deleted from QA test list?

disaggregated/test_workers.py::test_workers_kv_cache_aware_router[TinyLlama-1.1B-Chat-v1.0]
disaggregated/test_workers.py::test_workers_kv_cache_aware_router_deepseek_v3_lite_bf16[DeepSeek-V3-Lite-bf16]
disaggregated/test_workers.py::test_workers_kv_cache_aware_router_eviction[TinyLlama-1.1B-Chat-v1.0]
Expand Down
2 changes: 0 additions & 2 deletions tests/integration/test_lists/waives.txt
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -146,7 +146,6 @@ examples/visual_gen/test_visual_gen_multi_gpu.py::test_wan22_t2v_lpips_against_g
examples/visual_gen/test_visual_gen_multi_gpu.py::test_wan22_t2v_lpips_against_golden_tp[cfg2_tp2] SKIP (https://nvbugs/6329227)
examples/visual_gen/test_visual_gen_multi_gpu.py::test_wan22_t2v_lpips_against_golden_tp[tp2] SKIP (https://nvbugs/6329227)
examples/visual_gen/test_visual_gen_multi_gpu.py::test_wan22_t2v_lpips_against_golden_tp[tp2_ulysses2] SKIP (https://nvbugs/6329227)
full:A100/disaggregated/test_workers.py::test_workers_conditional_disaggregation_deepseek_v3_lite_bf16[DeepSeek-V3-Lite-bf16] SKIP (https://nvbugs/6329052)
full:A100X/llmapi/test_llm_examples.py::test_llmapi_speculative_decoding_mtp SKIP (https://nvbugs/6287561)
full:A100X/unittest/llmapi/test_llm_pytorch.py -m "part0" SKIP (https://nvbugs/6416249)
full:B200/accuracy/test_llm_api_pytorch.py::TestDeepSeekR1::test_nvfp4_multi_gpus[throughput_pp4_mtp] SKIP (https://nvbugs/5970614)
Expand Down Expand Up @@ -219,7 +218,6 @@ full:L40S/accuracy/test_llm_api_autodeploy.py::TestNemotronH::test_auto_dtype[tr
full:L40S/accuracy/test_llm_api_pytorch.py::TestQwen3_30B_A3B::test_fp8[latency-torch_compile=True] SKIP (https://nvbugs/6276841)
full:L40S/accuracy/test_llm_api_pytorch_multimodal.py::TestExaone4_5_33B::test_auto_dtype[forced_chunked_prefill] SKIP (https://nvbugs/6327149)
full:L40S/accuracy/test_llm_api_pytorch_multimodal.py::TestExaone4_5_33B::test_auto_dtype[full_budget] SKIP (https://nvbugs/6327149)
full:L40S/disaggregated/test_workers.py::test_workers_conditional_disaggregation_deepseek_v3_lite_bf16[DeepSeek-V3-Lite-bf16] SKIP (https://nvbugs/6329052)
full:RTXPro6000D/accuracy/test_llm_api_pytorch.py::TestDeepSeekV3Lite::test_nvfp4_4gpus[moe_backend=CUTLASS-mtp_nextn=0-ep4-fp8kv=True-attention_dp=True-cuda_graph=True-overlap_scheduler=True-low_precision_combine=False-torch_compile=False] SKIP (https://nvbugs/5948435)
full:RTXPro6000D/accuracy/test_llm_api_pytorch.py::TestDeepSeekV3Lite::test_nvfp4_4gpus[moe_backend=CUTLASS-mtp_nextn=0-ep4-fp8kv=True-attention_dp=True-cuda_graph=True-overlap_scheduler=True-low_precision_combine=False-torch_compile=True] SKIP (https://nvbugs/5961814)
full:RTXPro6000D/accuracy/test_llm_api_pytorch.py::TestDeepSeekV3Lite::test_nvfp4_4gpus[moe_backend=CUTLASS-mtp_nextn=2-ep4-fp8kv=True-attention_dp=True-cuda_graph=True-overlap_scheduler=True-low_precision_combine=False-torch_compile=False] SKIP (https://nvbugs/5961814)
Expand Down
Loading