fix(recipes): pin transformers==4.57.6 in deepseek-v32-fp4 perf jobs

[dagil-nvidia]

Summary

• QA (DYN-2878) reported recipes/deepseek-v32-fp4/trtllm/disagg-kv-router/perf.yaml exiting with TokenizerError: Failed to load tokenizer 'nvidia/DeepSeek-V3.2-NVFP4' against nvcr.io/nvstaging/ai-dynamo/tensorrtllm-runtime:1.1.0-rc4.
• Root cause is not missing tokenizer files (the NVIDIA repo has shipped tokenizer.json + tokenizer_config.json since 2026-01-02). It's a silent transformers upgrade past native model_type=deepseek_v32 support.
• Fix: exact-pin transformers==4.57.6 (the version verified to load the deepseek_v32 tokenizer) on the pip install aiperf==0.6.0 line in both perf.yaml files. Same change in disagg-kv-router and agg-round-robin.

Root cause

1. nvcr.io/.../tensorrtllm-runtime:1.1.0-rc4 ships transformers==4.55.0 (per NVIDIA/TensorRT-LLM v1.1.0rc4 requirements.txt).
2. The perf.yaml install step pip install aiperf==0.6.0 upgrades transformers to satisfy aiperf 0.6.0's transformers>=4.56.0 floor. With default pip resolution and no upper bound, this picks the latest matching release — currently 5.7.0.
3. transformers 5.x has no native support for model_type=deepseek_v32 (still pending in huggingface/transformers#41251 and #42767). AutoTokenizer.from_pretrained() reads config.json first and raises AttributeError: 'PreTrainedConfig' object has no attribute 'max_position_embeddings' before it ever opens tokenizer.json.
4. aiperf catches the exception in aiperf/common/tokenizer.py and re-raises as TokenizerError: Failed to load tokenizer '<name>' — exactly the QA error.

This regressed silently when transformers 5.0 shipped, with no change in this repo.

Fix

Exact-pin to a transformers version verified to load the deepseek_v32 tokenizer:

pip install "aiperf==0.6.0" "transformers==4.57.6"

4.57.6 is the latest 4.x release. The exact pin is deterministic across job re-runs and removes any dependency on what <5 happens to resolve to in the future.

Verification (reproduced locally)

Setup	transformers ends up at	aiperf tokenizer load
rc4 base + pip install aiperf==0.6.0	5.7.0	FAIL — Failed to load tokenizer 'nvidia/DeepSeek-V3.2-NVFP4' (caused by AttributeError: 'PreTrainedConfig' object has no attribute 'max_position_embeddings')
rc4 base + pip install "aiperf==0.6.0" "transformers==4.57.6"	4.57.6	OK — LlamaTokenizerFast, vocab=128000, 'Hello world' -> [19923, 2058]

Note: the original version of this PR (changing --tokenizer to deepseek-ai/DeepSeek-V3.2) did not fix the failure — both repos carry model_type=deepseek_v32 and fail identically on transformers 5.x. That commit has been replaced.

Where Should the Reviewer Start?

• recipes/deepseek-v32-fp4/trtllm/disagg-kv-router/perf.yaml (line 52)
• recipes/deepseek-v32-fp4/trtllm/agg-round-robin/perf.yaml (line 52)

Diff is +14/-2 across the two files: the install line plus a 6-line explanatory comment.

Related Issues

Fixes DYN-2878

Test Plan

• CI passes
• QA re-runs kubectl apply -f recipes/deepseek-v32-fp4/trtllm/disagg-kv-router/perf.yaml -n <ns> against dynamo-nscale-dev-cluster and confirms the perf job exits with code 0
• QA also validates recipes/deepseek-v32-fp4/trtllm/agg-round-robin/perf.yaml since it had the same bug

Follow-ups (out of scope here)

The same pip install aiperf==0.6.0 pattern (without a transformers pin) appears in recipes/qwen3-vl-30b/vllm/agg-embedding-cache/perf.yaml. That recipe doesn't hit the bug today (qwen3 is supported in transformers >= 4.56) but would benefit from the same defensive pin. Worth a follow-up PR.

[coderabbitai]

Walkthrough

Updated tokenizer references in performance configuration files for DeepSeek model recipes, changing the tokenizer identifier from the NVIDIA-namespaced version to the DeepSeek-AI namespaced version in two separate benchmark configurations.

Changes

Cohort / File(s)	Summary
Tokenizer Configuration Updates recipes/deepseek-v32-fp4/trtllm/agg-round-robin/perf.yaml, recipes/deepseek-v32-fp4/trtllm/disagg-kv-router/perf.yaml	Updated --tokenizer argument in aiperf profile commands from nvidia/DeepSeek-V3.2-NVFP4 to deepseek-ai/DeepSeek-V3.2 for both warmup and trace-replay phases, while keeping model identifiers unchanged.

Estimated code review effort

🎯 2 (Simple) | ⏱️ ~8 minutes

🚥 Pre-merge checks | ✅ 4 | ❌ 1

❌ Failed checks (1 warning)

Check name	Status	Explanation	Resolution
Title check	⚠️ Warning	The PR title mentions pinning transformers==4.57.6, but the actual changes involve updating tokenizer references from 'nvidia/DeepSeek-V3.2-NVFP4' to 'deepseek-ai/DeepSeek-V3.2' and adding a transformers pin. The title is misleading about the primary change.	Update the title to accurately reflect the main change: e.g., 'fix(recipes): use upstream tokenizer and pin transformers for deepseek-v32-fp4 perf jobs' or focus on either the tokenizer change or the transformers pin as the primary concern.

✅ Passed checks (4 passed)

Check name	Status	Explanation
Docstring Coverage	✅ Passed	No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check.
Linked Issues check	✅ Passed	Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check	✅ Passed	Check skipped because no linked issues were found for this pull request.
Description check	✅ Passed	The PR description is comprehensive and follows the template with all required sections: Overview (Summary), Details (Root cause and Fix), Where to start, Related Issues, and Test Plan. It thoroughly explains the issue, root cause analysis, verification, and follow-ups.

_{✏️ Tip: You can configure your own custom pre-merge checks in the settings.}

---

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

• X
• Mastodon
• Reddit
• LinkedIn

_{Comment @coderabbitai help to get the list of available commands and usage tips.}

[nealvaidya]

The nvidia/DeepSeek-V3.2-NVFP4 HuggingFace repo is a quantized weights-only checkpoint and ships no tokenizer files, so passing --tokenizer nvidia/DeepSeek-V3.2-NVFP4 to aiperf causes the perf job to fail at startup with Failed to load tokenizer 'nvidia/DeepSeek-V3.2-NVFP4'

This is not true, the repo has the tokenizer files:

• https://huggingface.co/nvidia/DeepSeek-V3.2-NVFP4/blob/main/tokenizer.json
• https://huggingface.co/nvidia/DeepSeek-V3.2-NVFP4/blob/main/tokenizer_config.json

Are we sure this fixes the raised issue

[dagil-nvidia]

You were right — the original premise was wrong, and the original tokenizer-rename did not fix the raised issue. I dug in and reproduced the actual failure. The PR has been replaced with the real fix; force-pushed to 5a41d3edcd.

What was actually happening:

1. nvcr.io/.../tensorrtllm-runtime:1.1.0-rc4 ships transformers==4.55.0 (per the v1.1.0rc4 requirements.txt).
2. The perf.yaml line pip install aiperf==0.6.0 upgrades transformers to satisfy aiperf's transformers>=4.56.0 floor. Without an upper bound, default pip resolution picks the latest — currently 5.7.0.
3. transformers 5.x has no native support for model_type=deepseek_v32 (still pending in huggingface/transformers#41251 and #42767). AutoTokenizer.from_pretrained() reads config.json first and raises AttributeError: 'PreTrainedConfig' object has no attribute 'max_position_embeddings' before it ever opens tokenizer.json. aiperf wraps it as TokenizerError: Failed to load tokenizer '<name>'.
4. Both nvidia/DeepSeek-V3.2-NVFP4 and deepseek-ai/DeepSeek-V3.2 have model_type=deepseek_v32 in their config.json, so the original tokenizer rename produced the exact same error.

Reproduced end-to-end locally:

Setup	transformers ends up at	aiperf load
transformers==4.55.0 + pip install aiperf==0.6.0	5.7.0	FAIL — same error as DYN-2878
transformers==4.55.0 + pip install "aiperf==0.6.0" "transformers>=4.56.0,<5"	4.57.6	OK — LlamaTokenizerFast, vocab=128000

New fix: pin transformers>=4.56.0,<5 on the install line. The lower bound mirrors pyproject.toml so the perf job runs against the same transformers contract as the rest of dynamo; the upper bound prevents the silent upgrade past native deepseek_v32 support.

Diff is now +14/-2: install line plus a 6-line explanatory comment in each of the two perf.yaml files. PR title and body updated to match.