conversion : fix Nemotron 3.5 Lightning layers

[danbev]

Overview

This commit contains a fix for the conversion of NVIDIA Nemotron 3.5 Lightning which currently incorrectly converts when using a transformers version later than 5.5.1.

Additional information

When converting using convert the transformers version is 5.13.1 and this produces the following:

WARNING:gguf.gguf_writer:Duplicated key name 'nemotron_h_moe.attention.head_count_kv', overwriting it with new value [0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0] of type ARRAY

This does not happen with transformers 5.5.1. The reason seems to be that the configuration is different in later versions, for example when using 5.13.1 the configuration block looks like this:

transformers 5.13.1
raw has layers_block_type: True
autoconfig has layers_block_type: True
autoconfig layers_block_type: [
'linear_attention',
'moe',
'linear_attention',
'moe',
'linear_attention',
'full_attention',
'moe',
...
]

And with 5.5.1 we get:

transformers 5.5.1
raw has layers_block_type: True
autoconfig has layers_block_type: True
autoconfig layers_block_type: [
'mamba',
'moe',
'mamba',
'moe',
'mamba',
'attention',
'moe'
...
]

In our conversion script we only match for attention, not full attention which is causing this issue.

With the changes in this commit the output with transformers 5.13.1 will be:

(venv) $ gguf-dump models/NVIDIA-Nemotron-3.5-Lightning-30B-A3B-BF16.gguf | grep head_count_kv
INFO:gguf-dump:* Loading: models/NVIDIA-Nemotron-3.5-Lightning-30B-A3B-BF16.gguf
     29: [INT32]    |       52 | nemotron_h_moe.attention.head_count_kv = [0, 0, 0, 0, 0, 2, ...]

Resolves: #27718
Refs: https://github.com/ggml-org/convert/actions/runs/32949047680/job/98116096069#step:5:2391

Requirements

• I have read and agree with the contributing guidelines
• AI usage disclosure: No

[danbev]

I'm looking at enabling convert to be used for private/day-0 conversions so I avoid making this type of mistake in the future.