[Bug]: Qwen3-VL-4B 'Tensor' object has no attribute 'get'. Did you mean: 'det'?

[justzhanghong]

⚙️ Your current environment

The output of python collect_env.py

### Environment Information ###
Operating System: `Linux-5.4.0-125-generic-x86_64-with-glibc2.35`
Python Version: `3.12.11 (main, Jun  4 2025, 08:56:18) [GCC 11.4.0]`
llm-compressor Version: `0.8.1`
compressed-tensors Version: `0.12.3a20251013`
transformers Version: `4.57.1`
torch Version: `2.8.0+cu128`
CUDA Devices: `['NVIDIA A100-SXM4-80GB']`
AMD Devices: `None`

🐛 Describe the bug

/usr/local/lib/python3.12/dist-packages/torch/cuda/init.py:63: FutureWarning: The pynvml package is deprecated. Please install nvidia-ml-py instead. If you did not install pynvml directly, please report this to the maintainers of the package that installed pynvml for you.
import pynvml # type: ignore[import]
torch_dtype is deprecated! Use dtype instead!
Loading checkpoint shards: 100%|██████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 2/2 [00:34<00:00, 17.45s/it]
Generating test split: 3532 examples [00:02, 1485.15 examples/s]
Map: 100%|███████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 12/12 [00:00<00:00, 12.18 examples/s]
2025-10-17T00:56:56.307532-0700 | reset | INFO - Compression lifecycle reset
2025-10-17T00:56:56.315648-0700 | _create_default_logger | INFO - Logging all LLM Compressor modifier-level logs to sparse_logs/17-10-2025_00.56.56.log
2025-10-17T00:56:56.317732-0700 | from_modifiers | INFO - Creating recipe from modifiers
2025-10-17T00:56:56.375590-0700 | initialize | INFO - Compression lifecycle initialized for 1 modifiers
2025-10-17T00:56:56.375803-0700 | IndependentPipeline | INFO - Inferred SequentialPipeline for GPTQModifier
huggingface/tokenizers: The current process just got forked, after parallelism has already been used. Disabling parallelism to avoid deadlocks...
To disable this warning, you can either:
- Avoid using tokenizers before the fork if possible
- Explicitly set the environment variable TOKENIZERS_PARALLELISM=(true | false)
huggingface/tokenizers: The current process just got forked, after parallelism has already been used. Disabling parallelism to avoid deadlocks...
To disable this warning, you can either:
- Avoid using tokenizers before the fork if possible
- Explicitly set the environment variable TOKENIZERS_PARALLELISM=(true | false)
huggingface/tokenizers: The current process just got forked, after parallelism has already been used. Disabling parallelism to avoid deadlocks...
To disable this warning, you can either:
- Avoid using tokenizers before the fork if possible
- Explicitly set the environment variable TOKENIZERS_PARALLELISM=(true | false)
huggingface/tokenizers: The current process just got forked, after parallelism has already been used. Disabling parallelism to avoid deadlocks...
To disable this warning, you can either:
- Avoid using tokenizers before the fork if possible
- Explicitly set the environment variable TOKENIZERS_PARALLELISM=(true | false)
huggingface/tokenizers: The current process just got forked, after parallelism has already been used. Disabling parallelism to avoid deadlocks...
To disable this warning, you can either:
- Avoid using tokenizers before the fork if possible
- Explicitly set the environment variable TOKENIZERS_PARALLELISM=(true | false)
huggingface/tokenizers: The current process just got forked, after parallelism has already been used. Disabling parallelism to avoid deadlocks...
To disable this warning, you can either:
- Avoid using tokenizers before the fork if possible
- Explicitly set the environment variable TOKENIZERS_PARALLELISM=(true | false)
huggingface/tokenizers: The current process just got forked, after parallelism has already been used. Disabling parallelism to avoid deadlocks...
To disable this warning, you can either:
- Avoid using tokenizers before the fork if possible
- Explicitly set the environment variable TOKENIZERS_PARALLELISM=(true | false)
huggingface/tokenizers: The current process just got forked, after parallelism has already been used. Disabling parallelism to avoid deadlocks...
To disable this warning, you can either:
- Avoid using tokenizers before the fork if possible
- Explicitly set the environment variable TOKENIZERS_PARALLELISM=(true | false)
Traceback (most recent call last):
File "/nfs/AE/zhanghong/llm-compressor-0.8.1/examples/multimodal_vision/zh_qwen_3_vl_4b_example.py", line 85, in
oneshot(
File "/nfs/AE/zhanghong/dev/llm-compressor/src/llmcompressor/entrypoints/oneshot.py", line 330, in oneshot
one_shot()
File "/nfs/AE/zhanghong/dev/llm-compressor/src/llmcompressor/entrypoints/oneshot.py", line 158, in call
self.apply_recipe_modifiers(
File "/nfs/AE/zhanghong/dev/llm-compressor/src/llmcompressor/entrypoints/oneshot.py", line 201, in apply_recipe_modifiers
pipeline(
File "/nfs/AE/zhanghong/dev/llm-compressor/src/llmcompressor/pipelines/independent/pipeline.py", line 45, in call
pipeline(model, dataloader, dataset_args)
File "/nfs/AE/zhanghong/dev/llm-compressor/src/llmcompressor/pipelines/sequential/pipeline.py", line 71, in call
subgraphs = trace_subgraphs(model, sample_input, sequential_targets, ignore)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/nfs/AE/zhanghong/dev/llm-compressor/src/llmcompressor/pipelines/sequential/helpers.py", line 125, in trace_subgraphs
tracer.trace(
File "/usr/local/lib/python3.12/dist-packages/transformers/utils/fx.py", line 1316, in trace
self.graph = super().trace(root, concrete_args=concrete_args)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.12/dist-packages/torch/fx/_symbolic_trace.py", line 850, in trace
(self.create_arg(fn(*args)),),
^^^^^^^^^
File "/usr/local/lib/python3.12/dist-packages/transformers/utils/generic.py", line -1, in wrapper
File "/usr/local/lib/python3.12/dist-packages/transformers/utils/fx.py", line 681, in getattr
return HFAttribute(self, k)
^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.12/dist-packages/transformers/utils/fx.py", line 700, in init
self.install_metadata(getattr(self.root._metadata, attr))
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
AttributeError: 'Tensor' object has no attribute 'get'. Did you mean: 'det'?

🛠️ Steps to reproduce

No response

[dsikka]

Hi @justzhanghong do you mind sending your full compression script that you're using?

[brian-dellabetta]

This should be resolved by

• Fix meta tensor materialization in SequentialPipeline #1930

I am unable to run Qwen 3 VL on main due to an error during subgraph tracing. On this branch, it works for me

[dsikka]

Hi @justzhanghong - Do you mind using llmcompressor main as well? llmcompressor 0.8.1 requires compressed-tessors 0.12.2 and it seems like you're using a nightly version for compressed-tensors

I was able to generate the models on llmcompressor main using the following script:

import torch
from datasets import load_dataset
from transformers import AutoProcessor, Qwen3VLForConditionalGeneration

from llmcompressor import oneshot
from llmcompressor.modeling import replace_modules_for_calibration
from llmcompressor.modifiers.quantization import GPTQModifier
from llmcompressor.utils import dispatch_for_generation

# NOTE: Requires a minimum of transformers 4.57.0

MODEL_ID = "Qwen/Qwen3-VL-8B-Instruct"


# Load model.
model = Qwen3VLForConditionalGeneration.from_pretrained(MODEL_ID, torch_dtype="auto")
processor = AutoProcessor.from_pretrained(MODEL_ID)

DATASET_ID = "neuralmagic/calibration"
NUM_CALIBRATION_SAMPLES = 20
MAX_SEQUENCE_LENGTH = 8192

ds = load_dataset(DATASET_ID, name="LLM", split=f"train[:{NUM_CALIBRATION_SAMPLES}]")


def preprocess_function(example):
    messgages = []
    for message in example["messages"]:
        messgages.append(
            {
                "role": message["role"],
                "content": [{"type": "text", "text": message["content"]}],
            }
        )

    return processor.apply_chat_template(
        messgages,
        return_tensors="pt",
        padding=False,
        truncation=True,
        max_length=MAX_SEQUENCE_LENGTH,
        tokenize=True,
        add_special_tokens=False,
        return_dict=True,
        add_generation_prompt=False,
    )


ds = ds.map(preprocess_function, batched=False, remove_columns=ds.column_names)


def data_collator(batch):
    assert len(batch) == 1
    return {
        key: (
            torch.tensor(value)
            if key != "pixel_values"
            else torch.tensor(value, dtype=torch.bfloat16).squeeze(0)
        )
        for key, value in batch[0].items()
    }


recipe = GPTQModifier(
    targets="Linear",
    scheme="W4A16",
    ignore=[
        "re:.*lm_head",
        "re:visual.*",
        "re:model.visual.*",
    ],
)

# Apply quantization.
oneshot(
    model=model,
    recipe=recipe,
    max_seq_length=MAX_SEQUENCE_LENGTH,
    num_calibration_samples=NUM_CALIBRATION_SAMPLES,
    dataset=ds,
    data_collator=data_collator,
)

print("========== SAMPLE GENERATION ==============")
dispatch_for_generation(model)
input_ids = processor(text="Hello my name is", return_tensors="pt").input_ids.to("cuda")
output = model.generate(input_ids, max_new_tokens=20)
print(processor.decode(output[0]))
print("==========================================")


# Save to disk in compressed-tensors format.
SAVE_DIR = MODEL_ID.rstrip("/").split("/")[-1] + "-W4A16"
model.save_pretrained(SAVE_DIR)
processor.save_pretrained(SAVE_DIR)

[justzhanghong]

thank.