[Bugfix] DeepSeek V4: support transformers >= 4.57 normalized compress_ratios

[dparikh79]

Purpose

Closes #42741.

DeepseekV4Attention.__init__ reads config.compress_ratios[layer_id] directly (vllm/model_executor/models/deepseek_v4.py:960). transformers >= 4.57 reshapes the legacy JSON field on DeepseekV4Config.__init__ into layer_types (list[str]) + compress_rates (dict[str, int]) and stops exposing compress_ratios. Every DSV4 checkpoint then fails to load on vLLM >= 0.20.2:

AttributeError: 'DeepseekV4Config' object has no attribute 'compress_ratios'.
Did you mean: 'compress_rates'?

Read from the normalized fields when compress_ratios is absent. The per-layer ratio is reconstructed via the issue author's documented 1-to-1 mapping:

compress_ratios[i] == compress_rates.get(layer_types[i], 0)

The existing max(1, ...) clamp at the end keeps the downstream invariant (compress ratio is never 0, see line 1019 where self.compress_ratio > 1 gates compress_rope_theta) intact for both paths. Legacy configs with compress_ratios keep the original code path, so anyone pinning a pre-4.57 transformers stack sees no behavior change.

Duplicate-work check

gh issue view 42741 --repo vllm-project/vllm --comments    # 0 comments
gh pr list --repo vllm-project/vllm --state all --search "42741 in:body"
gh pr list --repo vllm-project/vllm --state all --search "DeepSeek V4 compress_ratios"

No existing PR addresses this bug.

Test Plan

The CUDA / GPU code path that DSV4 actually exercises is not reachable on a CPU dev box, but the resolution logic is small enough to validate at the Python level with a synthetic config. Twenty assertions covering both schemas:

• Bug repro under the old code: AttributeError on a normalized-only config.
• Legacy schema (with compress_ratios): 8 layer indices, fixed values match the pre-fix path.
• Normalized schema (layer_types + compress_rates): 8 layer indices, fixed values match the legacy schema for the same logical mapping.
• Layer type missing from compress_rates: falls back to 1 via the max(1, ...) clamp.
• compress_rates is None: same fallback (covered by config.compress_rates or {}).

All twenty cases pass under the fix.

Test Result

=== bug reproduces under OLD code on normalized-only config ===
  AttributeError reproduced: 'types.SimpleNamespace' object has no attribute 'compress_ratios'

=== fixed code: legacy schema unchanged ===
  [OK ] legacy layer 0..7   (1, 4, 4, 128, 128, 1, 1, 1)

=== fixed code: normalized schema produces same values as legacy ===
  [OK ] normalized layer 0..7   (1, 4, 4, 128, 128, 1, 1, 1)

=== fixed code: layer_type missing from compress_rates falls back to 1 ===
  [OK ] full_attention -> 1
  [OK ] unknown_type -> 1

=== fixed code: compress_rates is None handled ===
  [OK ] None rates -> 1

ruff check and ruff format --check clean on the modified file.

Related work

Same DSV4 bug-report set:

• [Bug]: DeepSeek V4 load_weights UnboundLocalError: 'name_mapped' when expert mapping has no match #42769 / PR [Bugfix] DeepSeek V4: skip expert tensor when no mapping matches #42804: name_mapped UnboundLocalError in expert loading.
• [Bug]: DeepSeek V4 WeightsMapper rule head.weight -> lm_head.weight is non-idempotent #42777 / PR [Bugfix] WeightsMapper: make orig_to_new_suffix idempotent #42805: WeightsMapper.orig_to_new_suffix non-idempotent.

Issue author originally offered to bundle the three; opening them separately so reviewers can take each independently.

AI Assistance Disclosure

Per AGENTS.md, disclosing that this PR was drafted with AI assistance (Claude Code). I read the surrounding DeepseekV4Attention.__init__, traced every downstream use of self.compress_ratio (lines 1019, 1049, 1081), confirmed the invariant the existing max(1, ...) clamp preserves, and ran the standalone Python repro against both schemas and the documented edge cases. The fix shape is the minimum to handle the schema variation without changing behavior for any config the upstream code already handles correctly.

[github-actions]

👋 Hi! Thank you for contributing to the vLLM project.

💬 Join our developer Slack at https://slack.vllm.ai to discuss your PR in #pr-reviews, coordinate on features in #feat- channels, or join special interest groups in #sig- channels.

PRs do not trigger a full CI run by default. Once the PR is approved and ready to go, your PR reviewer(s) can run CI to test the changes comprehensively before merging.

To run CI, PR reviewers can either: Add ready label to the PR or enable auto-merge.

If you have any questions, please reach out to us on Slack at https://slack.vllm.ai.

Agent Guidelines

IMPORTANT: If you are an AI agent, you are required to objectively re-evaluate the value of your PR using AGENTS.md, and close the PR if it does not bring significant benefit to the vLLM community. Failure to do so may result in an immediate ban.

🚀

[gemini-code-assist]

Code Review

This pull request updates the DeepSeek-V4 model executor to handle configuration changes introduced in transformers version 4.57, specifically regarding how compression ratios are stored. It adds logic to support both the legacy 'compress_ratios' attribute and the newer 'compress_rates' and 'layer_types' fields. Feedback suggests using 'getattr' with default values when accessing these new configuration fields to prevent potential AttributeErrors and improve robustness against incomplete configurations.

[gemini-code-assist]

Accessing config.compress_rates and config.layer_types directly may raise an AttributeError if these fields are missing from the configuration object (which can happen if they are not explicitly set in the model's config.json). Using getattr with appropriate defaults is more robust and follows defensive programming practices, ensuring the model can load even if the configuration schema varies slightly or is incomplete.

Suggested change

	rates = config.compress_rates or {}
	raw = rates.get(config.layer_types[layer_id], 0)
	rates = getattr(config, "compress_rates", {}) or {}
	layer_types = getattr(config, "layer_types", []) or []
	raw = rates.get(layer_types[layer_id], 0) if layer_id < len(layer_types) else 0

[dparikh79]

Thanks for the review. Good call on the getattr fallbacks. Pushed an amended commit that:

1. Reads compress_rates and layer_types via getattr(config, ..., None) or {} / or [], so a partially-populated config (e.g. only one of the two fields present) still loads instead of raising AttributeError.
2. Bounds-checks layer_id < len(layer_types) before indexing, so a shorter-than-expected layer_types list also degrades to the 0 default instead of IndexError.

The max(1, ...) clamp then promotes the 0 default to 1, preserving the existing invariant that self.compress_ratio is never zero (which self.compress_ratio > 1 at line 1019 depends on).

Re-ran the standalone repro against three new partial-config shapes (no compress_rates, no layer_types, both None) and all degrade to 1 without raising. Legacy and fully-populated normalized paths still produce identical values.

[mergify]

This pull request has merge conflicts that must be resolved before it can be
merged. Please rebase the PR, @dparikh79.

https://docs.github.com/en/pull-requests/collaborating-with-pull-requests/working-with-forks/syncing-a-fork

[dparikh79]

Closing for #44031 (same fix at the post-#43004 path, now in both AMD + NVIDIA forks).