Convnext model fails to lower on QNN

[GregoryComer]

🐛 Describe the bug

The convnext_small model from torchvision does not successfully lower on the QNN backend. It fails with an error - "number of dimensions in the tensor input does not match the length of the desired ordering of dimensions i.e. input.dim() = 3 is not equal to len(dims) = 4". The error is from in torch, but the model is able to lower and run without delegation, so it appears that there may be some problem with a transformation made during QNN lowering. This same error message shows up in a few different models, so it would be nice to resolve it.

Output excerpt:

[INFO] [Qnn ExecuTorch]: Running level=3 optimization.
[QNN Partitioner Op Support]: aten.linear.default | True
[QNN Partitioner Op Support]: aten.view_copy.default | True
[QNN Partitioner Op Support]: aten.permute_copy.default | True
[QNN Partitioner Op Support]: aten.native_layer_norm.default | True
[QNN Partitioner Op Support]: aten.permute_copy.default | True
[QNN Partitioner Op Support]: aten.adaptive_avg_pool2d.default | True
[QNN Partitioner Op Support]: aten.add.Tensor | True
E0907 21:42:05.974000 30617 site-packages/torch/_subclasses/fake_tensor.py:2755] failed while attempting to run meta for aten.permute.default
E0907 21:42:05.974000 30617 site-packages/torch/_subclasses/fake_tensor.py:2755] Traceback (most recent call last):
E0907 21:42:05.974000 30617 site-packages/torch/_subclasses/fake_tensor.py:2755]   File "/home/gregory/miniconda3/envs/executorch/lib/python3.12/site-packages/torch/_subclasses/fake_tensor.py", line 2751, in _dispatch_impl
E0907 21:42:05.974000 30617 site-packages/torch/_subclasses/fake_tensor.py:2755]     r = func(*args, **kwargs)
E0907 21:42:05.974000 30617 site-packages/torch/_subclasses/fake_tensor.py:2755]         ^^^^^^^^^^^^^^^^^^^^^
E0907 21:42:05.974000 30617 site-packages/torch/_subclasses/fake_tensor.py:2755]   File "/home/gregory/miniconda3/envs/executorch/lib/python3.12/site-packages/torch/_ops.py", line 840, in __call__
E0907 21:42:05.974000 30617 site-packages/torch/_subclasses/fake_tensor.py:2755]     return self._op(*args, **kwargs)
E0907 21:42:05.974000 30617 site-packages/torch/_subclasses/fake_tensor.py:2755]            ^^^^^^^^^^^^^^^^^^^^^^^^^
E0907 21:42:05.974000 30617 site-packages/torch/_subclasses/fake_tensor.py:2755] RuntimeError: permute(sparse_coo): number of dimensions in the tensor input does not match the length of the desired ordering of dimensions i.e. input.dim() = 3 is not equal to len(dims) = 4

This can be reproduced with the following test case command or standalone script.

python -m executorch.backends.test.suite.runner models --flow qnn --filter "test_convnext_small_qnn_float32$"

Standalone repro:

from typing import Tuple

import executorch
import torch
import torchvision

from executorch.backends.qualcomm.utils.utils import (
    generate_qnn_executorch_compiler_spec,
    generate_htp_compiler_spec,
    QcomChipset,
    to_edge_transform_and_lower_to_qnn,
)

inputs = (torch.randn(1, 3, 224, 224),)
model = torchvision.models.convnext_small().eval()

ep = torch.export.export(model, inputs)

backend_options = generate_htp_compiler_spec(
    use_fp16=True,
)

compile_spec = generate_qnn_executorch_compiler_spec(
    soc_model=QcomChipset.SM8650,
    backend_options=backend_options,
)

model = to_edge_transform_and_lower_to_qnn(
    model,
    inputs,
    compile_spec
).to_executorch()

Note that running the backend test case requires executorch's python bindings to be built with the QNN backend. An example build command is below, Note that it will still need the library paths to be set up properly as described in the ET QNN docs.

CMAKE_ARGS="-DEXECUTORCH_BUILD_QNN=ON -DQNN_SDK_ROOT=$QNN_SDK_ROOT" ./install_executorch.sh --editable

Versions

Commit fbda3a9, x86-64 simulator, WSL

cc @cccclai @winskuo-quic @shewu-quic @cbilgin

[cccclai]

@haowhsu-quic can you help with triaging this issue? Note that it's reported from the backend tester, and we can cherry pick changes after branch cut. One thing to note is that, we don't need to have the full model lowered. As long as the repro command work, that's good.

[haowhsu-quic]

Will try to fix it asap.

[cccclai]

It seems happens to the mul tensor, and the op itself tensor shape is different. The mul node shape is 1, 768, 7, 7, but the input tensor shape is 768, 1, 1

node:  aten_mul_tensor_35 node:  aten_mul_tensor_35 node.args:  (p_features_7_2_layer_scale, aten_permute_copy_default_79) node.kwargs:  {} node.target:  <EdgeOpOverload: aten.mul.Tensor>: schema = aten::mul.Tensor(Tensor self, Tensor other) -> Tensor node.op:  call_function node.meta['val'].shape:  torch.Size([1, 768, 7, 7])

input_node:  p_features_7_2_layer_scale index:  0 input_node.name:  p_features_7_2_layer_scale input_node.target:  p_features_7_2_layer_scale input_node.op:  placeholder input_node.meta['val'].shape:  torch.Size([768, 1, 1])