Skip to content

[Bugfix][Model] Nemotron 3.5 Super VL: support the native transformers config - #64506

Merged
tomeras91 merged 4 commits into
vllm-project:mainfrom
roikoren755:bugfix/support-nemotron-h-omni-without-trust-remote-code
Oct 29, 2026
Merged

[Bugfix][Model] Nemotron 3.5 Super VL: support the native transformers config#64506
tomeras91 merged 4 commits into
vllm-project:mainfrom
roikoren755:bugfix/support-nemotron-h-omni-without-trust-remote-code

Conversation

@roikoren755

@roikoren755 roikoren755 commented Oct 29, 2026 •
edited
Loading

Copy link
Copy Markdown
Contributor

Overview

Lets NemotronH_Omni_Reasoning_V3 checkpoints (Nemotron 3.5 Super VL) run without --trust-remote-code on transformers ≥ 5.18.

Related to #63470. Alternative to #64030 (see Details).

Claims

  • nvidia/NVIDIA-Nemotron-3.5-Super-VL-120B-A12B-* serves without --trust-remote-code when transformers ≥ 5.18 is installed.
  • --trust-remote-code, older transformers versions, and the existing Nano VL / Nano Omni checkpoints behave as before.
  • CI now initializes the model from transformers' built-in nemotron_h_omni config.
  • supported_models.md lists the architecture and the transformers ≥ 5.18 requirement for running without the flag.

Validation

Serving: vllm serve nvidia/NVIDIA-Nemotron-3.5-Super-VL-120B-A12B-NVFP4, with and without --trust-remote-code, on transformers 5.18.0. Both load and serve correctly. Without the fix, the run without the flag fails at model init with:

AttributeError: 'NemotronH_Omni_Reasoning_V3_Config' object has no attribute 'vit_hidden_size'. Did you mean: 'vision_hidden_size'?

Test:

pytest tests/models/test_initialization.py -k "NemotronH_Omni_Reasoning_V3 or NemotronH_Nano_Omni_Reasoning_V3 or NemotronH_Nano_VL_V2" -v
  • Before the fix (model file from main):
============================= test session starts ==============================
platform linux -- Python 3.12.3, pytest-9.1.1, pluggy-1.6.0 -- <repo>/.venv/bin/python3
cachedir: .pytest_cache
hypothesis profile 'default'
rootdir: <repo>
configfile: pyproject.toml
plugins: anyio-4.14.2, buildkite-test-collector-0.1.9, hypothesis-6.165.2, asyncio-1.4.0, cov-7.1.0, forked-1.7.5, mock-3.15.1, rerunfailures-16.4, shard-0.1.2, timeout-2.4.0, schemathesis-4.24.3, typeguard-4.6.0
asyncio: mode=Mode.STRICT, debug=False, asyncio_default_fixture_loop_scope=None, asyncio_default_test_loop_scope=function
collecting ... collected 421 items / 420 deselected / 1 selected
Running 1 items in this shard: tests/models/test_initialization.py::test_can_initialize_large_subset[NemotronH_Omni_Reasoning_V3]

tests/models/test_initialization.py::test_can_initialize_large_subset[NemotronH_Omni_Reasoning_V3] FAILED [100%]

=================================== FAILURES ===================================
________ test_can_initialize_large_subset[NemotronH_Omni_Reasoning_V3] _________

model_arch = 'NemotronH_Omni_Reasoning_V3'
monkeypatch = <_pytest.monkeypatch.MonkeyPatch object at 0x7505f427d250>

    @pytest.mark.parametrize("model_arch", OTHER_MODEL_ARCH_LIST)
    def test_can_initialize_large_subset(model_arch: str, monkeypatch: pytest.MonkeyPatch):
        """Test initializing large subset of supported models.
    
        This test covers the complement of the tests covered in the "small subset"
        test.
        """
        if model_arch in ("HYV4ForCausalLM", "HYV4MTPModel"):
            from vllm.platforms import current_platform
    
            if current_platform.is_rocm():
                pytest.skip("HY V4 ROCm initialization requires #54405")
    
>       can_initialize(model_arch, monkeypatch, HF_EXAMPLE_MODELS)

tests/models/test_initialization.py:273: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
tests/utils.py:1829: in wrapper
    raise original_exception
tests/utils.py:1772: in wrapper
    func(*args, **kwargs)
    ^^^^^^^^^^^^^^^^^
tests/models/test_initialization.py:221: in can_initialize
    LLM(
vllm/entrypoints/llm.py:341: in __init__
    self.llm_engine = LLMEngine.from_engine_args(
    ^^^^^^^^^^^^^^^^^
vllm/v1/engine/llm_engine.py:191: in from_engine_args
    return cls(
vllm/v1/engine/llm_engine.py:116: in __init__
    self.engine_core = EngineCoreClient.make_client(
    ^^^^^^^^^^^^^^^^^
vllm/v1/engine/core_client.py:129: in make_client
    return SyncMPClient(
    ^^^^^^^^^^^^^^^^^
vllm/tracing/otel.py:175: in sync_wrapper
    return func(*args, **kwargs)
    ^^^^^^^^^^^^^^^^^
vllm/v1/engine/core_client.py:966: in __init__
    super().__init__(
vllm/v1/engine/core_client.py:699: in __init__
    with launch_core_engines(
    ^^^^^^^^^^^^^^^^^
/usr/lib/python3.12/contextlib.py:144: in __exit__
    next(self.gen)
vllm/v1/engine/utils.py:1393: in launch_core_engines
    wait_for_engine_startup(
    ^^^^^^^^^^^^^^^^^
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 

>   raise RuntimeError(
    ^^^^^^^^^^^^^^^^^
        "Engine core initialization failed. "
        "See root cause above. "
        f"Failed core proc(s): {finished}"
        + (
            f", failed frontend proc(s): {failed_frontend_procs}"
            if failed_frontend_procs
            else ""
        )
    )
E   RuntimeError: Engine core initialization failed. See root cause above. Failed core proc(s): {}

vllm/v1/engine/utils.py:1473: RuntimeError
----------------------------- Captured stdout call -----------------------------
Fork a new process to run a test 3802713
Fork a new process to run a test 0
(MainProcess pid=3802713) INFO 10-29 04:37:53 [startup.py:96] non-default args: {'load_format': 'dummy', 'max_model_len': 4096, 'gpu_memory_utilization': 0.8, 'disable_log_stats': True, 'hf_overrides': functools.partial(<function dummy_hf_overrides at 0x7506cf3ae200>, model_arch='NemotronH_Omni_Reasoning_V3', exist_overrides={'text_config': {'layers_block_type': ['linear_attention', 'full_attention']}}, use_original_num_layers=True), 'model_impl': 'vllm', 'model': 'nvidia/NVIDIA-Nemotron-3.5-Super-VL-120B-A12B-BF16'}
(MainProcess pid=3802713) INFO 10-29 04:39:15 [model.py:744] Resolved architecture: NemotronH_Omni_Reasoning_V3
(MainProcess pid=3802713) INFO 10-29 04:39:15 [model.py:2170] Using max model len 4096
(MainProcess pid=3802713) WARNING 10-29 04:39:15 [model.py:1116] Model does not support mm_device_do_normalize, forcing mm_device_do_normalize = False.
(MainProcess pid=3802713) INFO 10-29 04:39:17 [scheduler.py:310] Chunked prefill is enabled with max_num_batched_tokens=16384.
(MainProcess pid=3802713) INFO 10-29 04:39:17 [config.py:766] Mamba cache mode is set to 'align' for NemotronH_Omni_Reasoning_V3 by default when prefix caching is enabled
(MainProcess pid=3802713) INFO 10-29 04:39:17 [kernel.py:480] Final IR op priority after setting platform defaults: IrOpPriorityConfig(rms_norm=['native'], fused_add_rms_norm=['native'], gelu_and_mul_sparse=['triton', 'native'])
(EngineCore pid=3812221) INFO 10-29 04:39:19 [core.py:136] Initializing a V1 LLM engine (v0.32.1rc1.dev531+g9efeeb9aa) with config: model='nvidia/NVIDIA-Nemotron-3.5-Super-VL-120B-A12B-BF16', speculative_config=None, tokenizer='nvidia/NVIDIA-Nemotron-3.5-Super-VL-120B-A12B-BF16', skip_tokenizer_init=False, tokenizer_mode=auto, revision=main, tokenizer_revision=main, trust_remote_code=False, dtype=torch.bfloat16, max_seq_len=4096, download_dir=None, load_format=dummy, tensor_parallel_size=1, pipeline_parallel_size=1, data_parallel_size=1, decode_context_parallel_size=1, dcp_comm_backend=ag_rs, disable_custom_all_reduce=False, quantization=None, quantization_config=None, enforce_eager=False, aux_output_config=AuxOutputConfig(enable_return_routed_experts=False, max_bytes=None), kv_cache_dtype=auto, device_config=cuda, structured_outputs_config=StructuredOutputsConfig(backend='auto', disable_any_whitespace=False, disable_additional_properties=False, reasoning_parser='', reasoning_parser_plugin='', enable_in_reasoning=False), observability_config=ObservabilityConfig(show_hidden_metrics_for_version=None, otlp_traces_endpoint=None, collect_detailed_traces=None, per_request_spec_decode_metrics='none', kv_cache_metrics=False, kv_cache_metrics_sample=0.01, custom_histogram_buckets=None, cudagraph_metrics=False, enable_layerwise_nvtx_tracing=False, enable_mfu_metrics=False, enable_mm_processor_stats=False, enable_logging_iteration_details=False, jit_monitor_mode='warn', jit_monitor_verbose=False), seed=0, served_model_name=nvidia/NVIDIA-Nemotron-3.5-Super-VL-120B-A12B-BF16, enable_prefix_caching=True, enable_chunked_prefill=True, pooler_config=None, compilation_config={'mode': <CompilationMode.VLLM_COMPILE: 3>, 'debug_dump_path': None, 'cache_dir': '', 'compile_cache_save_format': 'binary', 'backend': 'inductor', 'custom_ops': ['none'], 'ir_enable_torch_wrap': True, 'splitting_ops': ['vllm::unified_attention_with_output', 'vllm::unified_mla_attention_with_output', 'vllm::mamba_mixer2', 'vllm::mamba_mixer', 'vllm::short_conv', 'vllm::qwen4_exp_ple_short_conv', 'vllm::qwen4_exp_qsa_with_output', 'vllm::qwen4_exp_qsa_prepare_with_output', 'vllm::linear_attention', 'vllm::qwen_gdn_attention_core', 'vllm::qwen_gdn_attention_core_fused_norm_packed', 'vllm::gdn_attention_core_xpu', 'vllm::olmo_hybrid_gdn_full_forward', 'vllm::sparse_attn_indexer', 'vllm::rocm_aiter_sparse_attn_indexer', 'vllm::deepseek_v4_attention', 'vllm::hpc_rope_norm_forward', 'vllm::unified_kv_cache_update', 'vllm::unified_mla_kv_cache_update'], 'compile_mm_encoder': False, 'cudagraph_mm_encoder': False, 'encoder_cudagraph_token_budgets': [], 'encoder_cudagraph_max_vision_items_per_batch': 0, 'encoder_cudagraph_max_frames_per_batch': None, 'cudagraph_decoder_replay': True, 'decoder_replay_cudagraph_capture_sizes': [], 'decoder_replay_trim_threshold': 768, 'compile_sizes': [], 'compile_ranges_endpoints': [16384], 'inductor_compile_config': {'enable_auto_functionalized_v2': False, 'combo_kernels': True, 'benchmark_combo_kernel': True}, 'inductor_passes': {}, 'cudagraph_mode': <CUDAGraphMode.FULL_AND_PIECEWISE: (2, 1)>, 'cudagraph_num_of_warmups': 1, 'cudagraph_capture_sizes': [1, 2, 4, 8, 16, 24, 32, 40, 48, 56, 64, 72, 80, 88, 96, 104, 112, 120, 128, 136, 144, 152, 160, 168, 176, 184, 192, 200, 208, 216, 224, 232, 240, 248, 256, 272, 288, 304, 320, 336, 352, 368, 384, 400, 416, 432, 448, 464, 480, 496, 512, 528, 544, 560, 576, 592, 608, 624, 640, 656, 672, 688, 704, 720, 736, 752, 768, 784, 800, 816, 832, 848, 864, 880, 896, 912, 928, 944, 960, 976, 992, 1008, 1024], 'cudagraph_copy_inputs': False, 'cudagraph_specialize_lora': True, 'use_inductor_graph_partition': False, 'pass_config': {'fuse_norm_quant': False, 'fuse_act_quant': False, 'fuse_attn_quant': False, 'fuse_allreduce_rms': False, 'enable_qk_norm_rope_fusion': False, 'fuse_rope_kvcache_cat_mla': False, 'fuse_act_padding': False, 'fuse_qk_norm_rope_kvcache': False}, 'max_cudagraph_capture_size': 1024, 'dynamic_shapes_config': {'type': <DynamicShapesType.BACKED: 'backed'>, 'evaluate_guards': False, 'assume_32_bit_indexing': False}, 'local_cache_dir': None, 'fast_moe_cold_start': False, 'static_all_moe_layers': []}, kernel_config=KernelConfig(ir_op_priority=IrOpPriorityConfig(rms_norm=['native'], fused_add_rms_norm=['native'], gelu_and_mul_sparse=['triton', 'native']), enable_flashinfer_autotune=True, enable_rocm_segmented_attn_autotune=False, enable_cutedsl_warmup=True, enable_jit_warmup=True, monokernel=False, moe_backend='auto', sparse_indexer_topk_backend='auto', linear_backend='auto', linear_backend_per_quant=None)
(EngineCore pid=3812221) INFO 10-29 04:39:22 [parallel_state.py:1929] world_size=1 rank=0 local_rank=0 distributed_init_method=file:///tmp/vllm_dist_0c42f96f096b4c2fb6f9f0aea327c549 backend=nccl
(EngineCore pid=3812221) INFO 10-29 04:39:24 [parallel_state.py:2385] rank 0 in world size 1 is assigned as DP rank 0, PP rank 0, PCP rank 0, TP rank 0, ETP rank 0, EP rank 0, EPLB rank N/A
(EngineCore pid=3812221) INFO 10-29 04:39:24 [gpu_worker.py:513] Using V2 Model Runner
(EngineCore pid=3812221) INFO 10-29 04:39:25 [model_runner.py:399] Loading model from scratch...
(EngineCore pid=3812221) INFO 10-29 04:39:29 [cuda.py:508] Using FLASHINFER attention backend out of potential backends: ['FLASHINFER', 'FLASH_ATTN', 'TRITON_ATTN', 'FLEX_ATTENTION'].
(EngineCore pid=3812221) INFO 10-29 04:39:30 [cuda.py:567] Using backend AttentionBackendEnum.FLASH_ATTN for vit attention
(EngineCore pid=3812221) INFO 10-29 04:39:30 [mm_encoder_attention.py:372] Using AttentionBackendEnum.FLASH_ATTN for MMEncoderAttention.
(EngineCore pid=3812221) ERROR 10-29 04:39:31 [core.py:1576] EngineCore failed to start.
(EngineCore pid=3812221) ERROR 10-29 04:39:31 [core.py:1576] Traceback (most recent call last):
(EngineCore pid=3812221) ERROR 10-29 04:39:31 [core.py:1576]   File "<repo>/vllm/v1/engine/core.py", line 1547, in run_engine_core
(EngineCore pid=3812221) ERROR 10-29 04:39:31 [core.py:1576]     engine_core = EngineCoreProc(*args, engine_index=dp_rank, **kwargs)
(EngineCore pid=3812221) ERROR 10-29 04:39:31 [core.py:1576]                   ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
(EngineCore pid=3812221) ERROR 10-29 04:39:31 [core.py:1576]   File "<repo>/vllm/tracing/otel.py", line 175, in sync_wrapper
(EngineCore pid=3812221) ERROR 10-29 04:39:31 [core.py:1576]     return func(*args, **kwargs)
(EngineCore pid=3812221) ERROR 10-29 04:39:31 [core.py:1576]            ^^^^^^^^^^^^^^^^^^^^^
(EngineCore pid=3812221) ERROR 10-29 04:39:31 [core.py:1576]   File "<repo>/vllm/v1/engine/core.py", line 1224, in __init__
(EngineCore pid=3812221) ERROR 10-29 04:39:31 [core.py:1576]     super().__init__(
(EngineCore pid=3812221) ERROR 10-29 04:39:31 [core.py:1576]   File "<repo>/vllm/v1/engine/core.py", line 147, in __init__
(EngineCore pid=3812221) ERROR 10-29 04:39:31 [core.py:1576]     self.model_executor = executor_class(vllm_config)
(EngineCore pid=3812221) ERROR 10-29 04:39:31 [core.py:1576]                           ^^^^^^^^^^^^^^^^^^^^^^^^^^^
(EngineCore pid=3812221) ERROR 10-29 04:39:31 [core.py:1576]   File "<repo>/vllm/tracing/otel.py", line 175, in sync_wrapper
(EngineCore pid=3812221) ERROR 10-29 04:39:31 [core.py:1576]     return func(*args, **kwargs)
(EngineCore pid=3812221) ERROR 10-29 04:39:31 [core.py:1576]            ^^^^^^^^^^^^^^^^^^^^^
(EngineCore pid=3812221) ERROR 10-29 04:39:31 [core.py:1576]   File "<repo>/vllm/v1/executor/abstract.py", line 120, in __init__
(EngineCore pid=3812221) ERROR 10-29 04:39:31 [core.py:1576]     self._init_executor()
(EngineCore pid=3812221) ERROR 10-29 04:39:31 [core.py:1576]   File "<repo>/vllm/v1/executor/uniproc_executor.py", line 74, in _init_executor
(EngineCore pid=3812221) ERROR 10-29 04:39:31 [core.py:1576]     self.driver_worker.load_model()
(EngineCore pid=3812221) ERROR 10-29 04:39:31 [core.py:1576]   File "<repo>/vllm/v1/worker/gpu_worker.py", line 601, in load_model
(EngineCore pid=3812221) ERROR 10-29 04:39:31 [core.py:1576]     self.model_runner.load_model(load_dummy_weights=load_dummy_weights)
(EngineCore pid=3812221) ERROR 10-29 04:39:31 [core.py:1576]   File "<repo>/vllm/v1/worker/gpu/model_runner.py", line 403, in load_model
(EngineCore pid=3812221) ERROR 10-29 04:39:31 [core.py:1576]     self.model = model or model_loader.load_model(
(EngineCore pid=3812221) ERROR 10-29 04:39:31 [core.py:1576]                           ^^^^^^^^^^^^^^^^^^^^^^^^
(EngineCore pid=3812221) ERROR 10-29 04:39:31 [core.py:1576]   File "<repo>/vllm/tracing/otel.py", line 175, in sync_wrapper
(EngineCore pid=3812221) ERROR 10-29 04:39:31 [core.py:1576]     return func(*args, **kwargs)
(EngineCore pid=3812221) ERROR 10-29 04:39:31 [core.py:1576]            ^^^^^^^^^^^^^^^^^^^^^
(EngineCore pid=3812221) ERROR 10-29 04:39:31 [core.py:1576]   File "<repo>/vllm/model_executor/model_loader/base_loader.py", line 76, in load_model
(EngineCore pid=3812221) ERROR 10-29 04:39:31 [core.py:1576]     model = self.create_model(
(EngineCore pid=3812221) ERROR 10-29 04:39:31 [core.py:1576]             ^^^^^^^^^^^^^^^^^^
(EngineCore pid=3812221) ERROR 10-29 04:39:31 [core.py:1576]   File "<repo>/vllm/model_executor/model_loader/base_loader.py", line 53, in create_model
(EngineCore pid=3812221) ERROR 10-29 04:39:31 [core.py:1576]     model = initialize_model(
(EngineCore pid=3812221) ERROR 10-29 04:39:31 [core.py:1576]             ^^^^^^^^^^^^^^^^^
(EngineCore pid=3812221) ERROR 10-29 04:39:31 [core.py:1576]   File "<repo>/vllm/tracing/otel.py", line 175, in sync_wrapper
(EngineCore pid=3812221) ERROR 10-29 04:39:31 [core.py:1576]     return func(*args, **kwargs)
(EngineCore pid=3812221) ERROR 10-29 04:39:31 [core.py:1576]            ^^^^^^^^^^^^^^^^^^^^^
(EngineCore pid=3812221) ERROR 10-29 04:39:31 [core.py:1576]   File "<repo>/vllm/model_executor/model_loader/utils.py", line 101, in initialize_model
(EngineCore pid=3812221) ERROR 10-29 04:39:31 [core.py:1576]     model = model_class(vllm_config=vllm_config, prefix=prefix)
(EngineCore pid=3812221) ERROR 10-29 04:39:31 [core.py:1576]             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
(EngineCore pid=3812221) ERROR 10-29 04:39:31 [core.py:1576]   File "<repo>/vllm/model_executor/models/nano_nemotron_vl.py", line 1000, in __init__
(EngineCore pid=3812221) ERROR 10-29 04:39:31 [core.py:1576]     vit_hidden_size = config.vit_hidden_size
(EngineCore pid=3812221) ERROR 10-29 04:39:31 [core.py:1576]                       ^^^^^^^^^^^^^^^^^^^^^^
(EngineCore pid=3812221) ERROR 10-29 04:39:31 [core.py:1576]   File "<repo>/.venv/lib/python3.12/site-packages/transformers/configuration_utils.py", line 525, in __getattribute__
(EngineCore pid=3812221) ERROR 10-29 04:39:31 [core.py:1576]     return super().__getattribute__(key)
(EngineCore pid=3812221) ERROR 10-29 04:39:31 [core.py:1576]            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
(EngineCore pid=3812221) ERROR 10-29 04:39:31 [core.py:1576]   File "<repo>/.venv/lib/python3.12/site-packages/transformers/integrations/heterogeneity/configuration_utils.py", line 312, in __getattribute__
(EngineCore pid=3812221) ERROR 10-29 04:39:31 [core.py:1576]     return super().__getattribute__(key)
(EngineCore pid=3812221) ERROR 10-29 04:39:31 [core.py:1576]            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
(EngineCore pid=3812221) ERROR 10-29 04:39:31 [core.py:1576] AttributeError: 'NemotronH_Omni_Reasoning_V3_Config' object has no attribute 'vit_hidden_size'. Did you mean: 'vision_hidden_size'?
(MainProcess pid=3802713) INFO 10-29 04:39:31 [utils.py:628] [shutdown] Process manager: send sigterm to process EngineCore
----------------------------- Captured stderr call -----------------------------

Fetching 22 files:   0%|          | 0/22 [00:00<?, ?it/s]
Fetching 22 files: 100%|██████████| 22/22 [00:00<00:00, 1405.83it/s]
Process EngineCore:
Traceback (most recent call last):
  File "/usr/lib/python3.12/multiprocessing/process.py", line 314, in _bootstrap
    self.run()
  File "/usr/lib/python3.12/multiprocessing/process.py", line 108, in run
    self._target(*self._args, **self._kwargs)
  File "<repo>/vllm/v1/engine/core.py", line 1580, in run_engine_core
    raise e
  File "<repo>/vllm/v1/engine/core.py", line 1547, in run_engine_core
    engine_core = EngineCoreProc(*args, engine_index=dp_rank, **kwargs)
                  ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "<repo>/vllm/tracing/otel.py", line 175, in sync_wrapper
    return func(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^
  File "<repo>/vllm/v1/engine/core.py", line 1224, in __init__
    super().__init__(
  File "<repo>/vllm/v1/engine/core.py", line 147, in __init__
    self.model_executor = executor_class(vllm_config)
                          ^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "<repo>/vllm/tracing/otel.py", line 175, in sync_wrapper
    return func(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^
  File "<repo>/vllm/v1/executor/abstract.py", line 120, in __init__
    self._init_executor()
  File "<repo>/vllm/v1/executor/uniproc_executor.py", line 74, in _init_executor
    self.driver_worker.load_model()
  File "<repo>/vllm/v1/worker/gpu_worker.py", line 601, in load_model
    self.model_runner.load_model(load_dummy_weights=load_dummy_weights)
  File "<repo>/vllm/v1/worker/gpu/model_runner.py", line 403, in load_model
    self.model = model or model_loader.load_model(
                          ^^^^^^^^^^^^^^^^^^^^^^^^
  File "<repo>/vllm/tracing/otel.py", line 175, in sync_wrapper
    return func(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^
  File "<repo>/vllm/model_executor/model_loader/base_loader.py", line 76, in load_model
    model = self.create_model(
            ^^^^^^^^^^^^^^^^^^
  File "<repo>/vllm/model_executor/model_loader/base_loader.py", line 53, in create_model
    model = initialize_model(
            ^^^^^^^^^^^^^^^^^
  File "<repo>/vllm/tracing/otel.py", line 175, in sync_wrapper
    return func(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^
  File "<repo>/vllm/model_executor/model_loader/utils.py", line 101, in initialize_model
    model = model_class(vllm_config=vllm_config, prefix=prefix)
            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "<repo>/vllm/model_executor/models/nano_nemotron_vl.py", line 1000, in __init__
    vit_hidden_size = config.vit_hidden_size
                      ^^^^^^^^^^^^^^^^^^^^^^
  File "<repo>/.venv/lib/python3.12/site-packages/transformers/configuration_utils.py", line 525, in __getattribute__
    return super().__getattribute__(key)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "<repo>/.venv/lib/python3.12/site-packages/transformers/integrations/heterogeneity/configuration_utils.py", line 312, in __getattribute__
    return super().__getattribute__(key)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
AttributeError: 'NemotronH_Omni_Reasoning_V3_Config' object has no attribute 'vit_hidden_size'. Did you mean: 'vision_hidden_size'?
--------------------------- Captured stdout teardown ---------------------------
gpu memory used/total (GiB): 0=0.71/179.06; 
Done waiting for free GPU memory on devices devices=[0] (threshold='device=0: 0.100') dur_s=0.00
=============================== warnings summary ===============================
tests/models/test_initialization.py::test_can_initialize_large_subset[NemotronH_Omni_Reasoning_V3]
  <repo>/tests/utils.py:1762: DeprecationWarning: This process (pid=3797052) is multi-threaded, use of fork() may lead to deadlocks in the child.
    pid = os.fork()

-- Docs: https://docs.pytest.org/en/stable/how-to/capture-warnings.html
=========================== short test summary info ============================
FAILED tests/models/test_initialization.py::test_can_initialize_large_subset[NemotronH_Omni_Reasoning_V3]
=========== 1 failed, 420 deselected, 1 warning in 106.05s (0:01:46) ===========
  • After the fix:
============================= test session starts ==============================
platform linux -- Python 3.12.3, pytest-9.1.1, pluggy-1.6.0 -- <repo>/.venv/bin/python3
cachedir: .pytest_cache
hypothesis profile 'default'
rootdir: <repo>
configfile: pyproject.toml
plugins: anyio-4.14.2, buildkite-test-collector-0.1.9, hypothesis-6.165.2, asyncio-1.4.0, cov-7.1.0, forked-1.7.5, mock-3.15.1, rerunfailures-16.4, shard-0.1.2, timeout-2.4.0, schemathesis-4.24.3, typeguard-4.6.0
asyncio: mode=Mode.STRICT, debug=False, asyncio_default_fixture_loop_scope=None, asyncio_default_test_loop_scope=function
collecting ... collected 421 items / 418 deselected / 3 selected
Running 3 items in this shard: tests/models/test_initialization.py::test_can_initialize_large_subset[NemotronH_Nano_VL_V2], tests/models/test_initialization.py::test_can_initialize_large_subset[NemotronH_Omni_Reasoning_V3], tests/models/test_initialization.py::test_can_initialize_large_subset[NemotronH_Nano_Omni_Reasoning_V3]

tests/models/test_initialization.py::test_can_initialize_large_subset[NemotronH_Nano_VL_V2] PASSED [ 33%]
tests/models/test_initialization.py::test_can_initialize_large_subset[NemotronH_Omni_Reasoning_V3] PASSED [ 66%]
tests/models/test_initialization.py::test_can_initialize_large_subset[NemotronH_Nano_Omni_Reasoning_V3] PASSED [100%]

=============================== warnings summary ===============================
tests/models/test_initialization.py::test_can_initialize_large_subset[NemotronH_Nano_VL_V2]
tests/models/test_initialization.py::test_can_initialize_large_subset[NemotronH_Omni_Reasoning_V3]
tests/models/test_initialization.py::test_can_initialize_large_subset[NemotronH_Nano_Omni_Reasoning_V3]
  <repo>/tests/utils.py:1762: DeprecationWarning: This process (pid=3791232) is multi-threaded, use of fork() may lead to deadlocks in the child.
    pid = os.fork()

-- Docs: https://docs.pytest.org/en/stable/how-to/capture-warnings.html
========== 3 passed, 418 deselected, 3 warnings in 102.11s (0:01:42) ===========

No model eval: the change only selects which config field the projector size comes from, so weights and computation are unchanged.

Details

Root cause. Transformers 5.18 added a built-in nemotron_h_omni config. The Super VL checkpoints declare "model_type": "nemotron_h_omni", so AutoConfig loads that class without remote code. The built-in class renames vit_hidden_size to vision_hidden_size, but NemotronH_Nano_VL_V2.__init__ read config.vit_hidden_size.

Fix. Read vision_hidden_size, the name the built-in transformers config uses, and fall back to vit_hidden_size, which the remote-code configs use:

Config Field
Transformers built-in nemotron_h_omni (≥ 5.18) vision_hidden_size
Checkpoint remote code (--trust-remote-code) vit_hidden_size
Nano VL / Nano Omni checkpoints vit_hidden_size

Test. The NemotronH_Omni_Reasoning_V3 registry entry was an offline placeholder. It now points at nvidia/NVIDIA-Nemotron-3.5-Super-VL-120B-A12B-BF16 without trust_remote_code, gated on transformers ≥ 5.18.0. The built-in NemotronHConfig derives hybrid_override_pattern from layers_block_type and doesn't allow setting it, so the 2-layer override sets layers_block_type instead. The "Basic Models (Extra Initialization)" CI step selects this test.

Why not #64030. #64030 copies the remote-code config and image processor into vLLM, about 520 lines. Transformers ≥ 5.18 already provides the config, and serving goes through NanoNemotronVLProcessor either way, so vLLM only needs this one field read. This matches #60623, which removed configs vLLM had copied once Transformers shipped them.

Limitations.

  • Older transformers: on transformers < 5.18, --trust-remote-code is still required.
  • Audio: the built-in config also renames sound_config → audio_config, which vLLM doesn't read. The Super VL checkpoints have no audio, but a future checkpoint that loads with the built-in config and includes audio would have audio switched off with no error. Left for a follow-up.
  • Recipe: the Super VL recipe keeps --trust-remote-code until a release includes this fix. Its pinned image (v0.31.0) doesn't have it.
Pull Request Checklist

  • I used vLLM's /pr-checklist skill. (Mandatory for agents, optional for humans).

  • AI assistance was used during the creation of this PR.

  • Design Fit: Minimizes impact on core components, reuses existing functionality, and justifies added complexity.

  • Testing and Validation: Validates the change and ensures any added tests are meaningful and reliable, with CI coverage or documented CI resource constraints and validation performed outside CI.

  • Code Quality and Style: Keeps code and comments clear and concise, and updates relevant documentation and examples.

  • Pull Request Contents: Includes a brief summary and relevant links, supports claims with evidence, explains root causes and implementation trade-offs, and follows the contributing guide.

🤖 Generated with Claude Code

roikoren755 and others added 3 commits October 26, 2026 15:38
@roikoren755 @claude
[Model] NemotronH Omni: read vision_hidden_size, fall back to vit_hid…
9efeeb9 
…den_size

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Signed-off-by: Roi Koren <roik@nvidia.com>
@roikoren755 @claude
[CI] Test NemotronH Omni native config with Nemotron 3.5 Super VL
9d839d9 
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Signed-off-by: Roi Koren <roik@nvidia.com>
@roikoren755 @claude
[Doc] Add Nemotron 3.5 Super VL to supported models
e56b29a 
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Signed-off-by: Roi Koren <roik@nvidia.com>
claude[bot]
claude Bot reviewed Oct 29, 2026
View reviewed changes

@claude claude Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Claude Code Review

This pull request is from a fork — automated review is disabled. A repository maintainer can comment @claude review to run a one-time review.

@mergify

mergify Bot commented Oct 29, 2026

Copy link
Copy Markdown
Contributor

Documentation preview: https://vllm--64506.org.readthedocs.build/en/64506/

@mergify mergify Bot added documentation Improvements or additions to documentation bug Something isn't working labels Oct 29, 2026
@roikoren755 roikoren755 mentioned this pull request Oct 29, 2026
Open
1 task done
tomeras91
tomeras91 approved these changes Oct 29, 2026
View reviewed changes

@tomeras91 tomeras91 left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@tomeras91
tomeras91 enabled auto-merge (squash) October 29, 2026 11:46
@tomeras91

tomeras91 commented Oct 29, 2026

Copy link
Copy Markdown
Member

/ci run

@github-actions github-actions Bot added the ready ONLY add when PR is ready to merge/full CI is needed label Oct 29, 2026
@github-actions

github-actions Bot commented Oct 29, 2026

Copy link
Copy Markdown

✅ Triggered Buildkite CI #98278 for commit 7ce77a52c646.

ℹ️ This PR is 274 commits behind upstream main. Merged upstream main into this branch as 7ce77a52c646 before starting CI. Pull the branch before pushing more changes.

@tomeras91 tomeras91 mentioned this pull request Oct 29, 2026
Closed
4 of 6 tasks
@tomeras91
tomeras91 merged commit 510d15c into vllm-project:main Oct 29, 2026
125 checks passed
@roikoren755
roikoren755 deleted the bugfix/support-nemotron-h-omni-without-trust-remote-code branch October 29, 2026 14:25
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@claude claude[bot] claude[bot] left review comments
@tomeras91 tomeras91 tomeras91 approved these changes
@DarkLight1337 DarkLight1337 Awaiting requested review from DarkLight1337 DarkLight1337 is a code owner
@ywang96 ywang96 Awaiting requested review from ywang96 ywang96 is a code owner
@AndreasKaratzas AndreasKaratzas Awaiting requested review from AndreasKaratzas AndreasKaratzas is a code owner

Assignees

No one assigned

Labels

bug Something isn't working documentation Improvements or additions to documentation ready ONLY add when PR is ready to merge/full CI is needed

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

3 participants

@roikoren755 @tomeras91 @vllm-agent