Repository navigation
Conversation
|
/ci run |
|
❌ This PR is 1 commit behind upstream |
ec657d5 to
5148805
Compare
|
/ci run |
|
❌ This PR is 2 commits behind upstream |
The V4 / V4.1 model packages read architecture fields straight off the config while building layers, with no default in the read, so a checkpoint whose config.json omits one fails deep inside layer construction with a bare AttributeError naming only the attribute -- with no hint that the fix is a config.json key. Explain it from the config classes instead. The message names the field, the model type and the file. No list of "required" fields: the set differs by architecture, platform and enabled feature, so a fixed list is both incomplete (head_dim, index_n_heads, the Engram fields) and partly wrong (V4.1 reaches V4's swiglu_limit read; num_hash_layers is supplied by both V4.1 MoE paths). A list that advertises the wrong fields sends people to edit keys that do not matter. The hook only runs when normal attribute lookup fails, so a complete checkpoint is unaffected and hasattr()/getattr(..., default) probes keep deciding fallbacks exactly as before. Signed-off-by: Ao Shen <aoshen@inferact.ai> Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Signed-off-by: aoshen02 <aoshen@inferact.ai>
5148805 to
7f29eea
Compare
|
/ci run |
|
❌ This PR is 2 commits behind upstream |
|
This pull request has merge conflicts that must be resolved before it can be |
|
Closing as we plan to delete this file in #60823 |
Summary
vllm/models/deepseek_v4/**reads six fields straight off the config objectwhile building layers, with no default anywhere in the stack:
A checkpoint whose
config.jsonomits one of them fails deep inside layerconstruction with:
which names the attribute but not the file it belongs in. This PR raises from
the config classes instead, naming
config.jsonand the full required set.Why not defaults
Because a default would be a guess at the model's shape.
index_topksets thesparse selection size,
compress_ratiosthe per-layer cache type,o_groupsthe o-proj grouping — a wrong value builds a model that does not match the
weights and fails later, or worse, silently produces bad output. An explicit
error is the correct behaviour for a checkpoint that does not say.
Cost
The hook is
__getattr__, which Python only calls when normal lookup fails, soa complete checkpoint pays nothing and behaves exactly as before. Unrelated
missing attributes keep raising a plain
AttributeError(name), socopy,to_dictand the transformers machinery are unaffected — covered by a test.Applies to both
DeepseekV4ConfigandDeepseekV41Config(V4.1 flattenstext_configonto the top level and has the same six fields missing when thecheckpoint omits them).
Not a duplicate
gh pr list --search "DeepseekV4Config"returns only #44031, which normalizescompress_ratiosfor transformers >= 4.57 inside the model files — a differentproblem (a field that is present but shaped differently). No open PR or issue
covers the missing-field failure mode.
Test plan
CPU-only. Covers each required field on both config classes, unrelated
attributes still raising plainly, and a complete config surviving
deepcopyand
to_dict.No model-quality impact: no change for any checkpoint that loads today.
AI assistance
Developed with AI assistance (Claude). The missing-field list came from reading
every bare
config.<field>access in the DeepSeek-V4/V4.1 model packages, andwas confirmed by constructing both config classes and checking each attribute. I
reviewed every changed line, ran the tests above, and I am the accountable
submitter.
🤖 Generated with Claude Code
Review rounds
A later adversarial round, told about the deleted list and asked for new
defects, went after the
__getattr__itself: copy/deepcopy, pickle protocols0-5, cloudpickle and
ForkingPickler(these configs cross the process boundaryto workers), probes on a pre-
__init__instance,hasattrfallbacks,attribute_mapandget_text_config()delegation, anddataclassesfieldintrospection. It executed all of them against transformers 5.12.1: both
DeepseekV4ConfigandDeepseekV41Configpass, and no caller matches the olderror text. No new defects.