请问可否提供pth模型转onnx的脚本

[Zong-my]

非常棒的工作，请问可否提供pth模型转onnx的脚本呢？

[HaoZhang534]

你或许可以参考这个,DINO应该可以用相似的方法 IDEA-Research/detrex#192

[Sencc]

请问你成功转换onnx了吗

[Zong-my]

请问你成功转换onnx了吗

未成功

[Sencc]

@Zong-my 我最近使用mmdeploy可以成功转换出onnx，但是它的检测速度比pytorch慢很多。

[wuzujiong]

@Zong-my 我最近使用mmdeploy可以成功转换出onnx，但是它的检测速度比pytorch慢很多。

我也是，速度差了10倍

[Sencc]

@wuzujiong 使用tensorrt加速后，速度正常

[pof1423]

I succeeded in converting to onnx. I recommend converting it to torch.jit.trace first and then exporting it to onnx.

[wuzujiong]

@wuzujiong 使用tensorrt加速后，速度正常

可以介绍一下用的是什么环境导出来.engine 的吗？我在windows 上转engine 转不成功；
PS D:\DL\mmdeploy-main\dino_crack_detections> trtexec --onnx=end2end.onnx --plugins=./mmdeploy_tensorrt_ops.dll --dumpProfile --separateProfileRun
&&&& RUNNING TensorRT.trtexec [TensorRT v8400] # C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v11.3\bin\trtexec.exe --onnx=end2end.onnx --plugins=./mmdeploy_tensorrt_ops.dll --dumpProfile --separateProfileRun
[02/25/2024-10:09:05] [I] === Model Options ===
[02/25/2024-10:09:05] [I] Format: ONNX
[02/25/2024-10:09:05] [I] Model: end2end.onnx
[02/25/2024-10:09:05] [I] Output:
[02/25/2024-10:09:05] [I] === Build Options ===
[02/25/2024-10:09:05] [I] Max batch: explicit batch
[02/25/2024-10:09:05] [I] Memory Pools: workspace: default, dlaSRAM: default, dlaLocalDRAM: default, dlaGlobalDRAM: default
[02/25/2024-10:09:05] [I] minTiming: 1
[02/25/2024-10:09:05] [I] avgTiming: 8
[02/25/2024-10:09:05] [I] Precision: FP32
[02/25/2024-10:09:05] [I] LayerPrecisions:
[02/25/2024-10:09:05] [I] Calibration:
[02/25/2024-10:09:05] [I] Refit: Disabled
[02/25/2024-10:09:05] [I] Sparsity: Disabled
[02/25/2024-10:09:05] [I] Safe mode: Disabled
[02/25/2024-10:09:05] [I] DirectIO mode: Disabled
[02/25/2024-10:09:05] [I] Restricted mode: Disabled
[02/25/2024-10:09:05] [I] Save engine:
[02/25/2024-10:09:05] [I] Load engine:
[02/25/2024-10:09:05] [I] Profiling verbosity: 0
[02/25/2024-10:09:05] [I] Tactic sources: Using default tactic sources
[02/25/2024-10:09:05] [I] timingCacheMode: local
[02/25/2024-10:09:05] [I] timingCacheFile:
[02/25/2024-10:09:05] [I] Input(s)s format: fp32:CHW
[02/25/2024-10:09:05] [I] Output(s)s format: fp32:CHW
[02/25/2024-10:09:05] [I] Input build shapes: model
[02/25/2024-10:09:05] [I] Input calibration shapes: model
[02/25/2024-10:09:05] [I] === System Options ===
[02/25/2024-10:09:05] [I] Device: 0
[02/25/2024-10:09:05] [I] DLACore:
[02/25/2024-10:09:05] [I] Plugins: ./mmdeploy_tensorrt_ops.dll
[02/25/2024-10:09:05] [I] === Inference Options ===
[02/25/2024-10:09:05] [I] Batch: Explicit
[02/25/2024-10:09:05] [I] Input inference shapes: model
[02/25/2024-10:09:05] [I] Iterations: 10
[02/25/2024-10:09:05] [I] Duration: 3s (+ 200ms warm up)
[02/25/2024-10:09:05] [I] Sleep time: 0ms
[02/25/2024-10:09:05] [I] Idle time: 0ms
[02/25/2024-10:09:05] [I] Streams: 1
[02/25/2024-10:09:05] [I] ExposeDMA: Disabled
[02/25/2024-10:09:05] [I] Data transfers: Enabled
[02/25/2024-10:09:05] [I] Spin-wait: Disabled
[02/25/2024-10:09:05] [I] Multithreading: Disabled
[02/25/2024-10:09:05] [I] CUDA Graph: Disabled
[02/25/2024-10:09:05] [I] Separate profiling: Enabled
[02/25/2024-10:09:05] [I] Time Deserialize: Disabled
[02/25/2024-10:09:05] [I] Time Refit: Disabled
[02/25/2024-10:09:05] [I] Skip inference: Disabled
[02/25/2024-10:09:05] [I] Inputs:
[02/25/2024-10:09:05] [I] === Reporting Options ===
[02/25/2024-10:09:05] [I] Verbose: Disabled
[02/25/2024-10:09:05] [I] Averages: 10 inferences
[02/25/2024-10:09:05] [I] Percentile: 99
[02/25/2024-10:09:05] [I] Dump refittable layers:Disabled
[02/25/2024-10:09:05] [I] Dump output: Disabled
[02/25/2024-10:09:05] [I] Profile: Enabled
[02/25/2024-10:09:05] [I] Export timing to JSON file:
[02/25/2024-10:09:05] [I] Export output to JSON file:
[02/25/2024-10:09:05] [I] Export profile to JSON file:
[02/25/2024-10:09:05] [I]
[02/25/2024-10:09:05] [I] === Device Information ===
[02/25/2024-10:09:05] [I] Selected Device: NVIDIA GeForce RTX 3080
[02/25/2024-10:09:05] [I] Compute Capability: 8.6
[02/25/2024-10:09:05] [I] SMs: 68
[02/25/2024-10:09:05] [I] Compute Clock Rate: 1.785 GHz
[02/25/2024-10:09:05] [I] Device Global Memory: 10239 MiB
[02/25/2024-10:09:05] [I] Shared Memory per SM: 100 KiB
[02/25/2024-10:09:05] [I] Memory Bus Width: 320 bits (ECC disabled)
[02/25/2024-10:09:05] [I] Memory Clock Rate: 9.501 GHz
[02/25/2024-10:09:05] [I]
[02/25/2024-10:09:05] [I] TensorRT version: 8.4.0
[02/25/2024-10:09:05] [I] Loading supplied plugin library: ./mmdeploy_tensorrt_ops.dll
[02/25/2024-10:09:06] [I] [TRT] [MemUsageChange] Init CUDA: CPU +519, GPU +0, now: CPU 12446, GPU 1346 (MiB)
[02/25/2024-10:09:07] [I] [TRT] [MemUsageSnapshot] Begin constructing builder kernel library: CPU 12651 MiB, GPU 1346 MiB
[02/25/2024-10:09:08] [I] [TRT] [MemUsageSnapshot] End constructing builder kernel library: CPU 13086 MiB, GPU 1468 MiB
[02/25/2024-10:09:08] [I] Start parsing network model
[02/25/2024-10:09:08] [I] [TRT] ----------------------------------------------------------------
[02/25/2024-10:09:08] [I] [TRT] Input filename: end2end.onnx
[02/25/2024-10:09:08] [I] [TRT] ONNX IR version: 0.0.6
[02/25/2024-10:09:08] [I] [TRT] Opset version: 11
[02/25/2024-10:09:08] [I] [TRT] Producer name: pytorch
[02/25/2024-10:09:08] [I] [TRT] Producer version: 1.11.0
[02/25/2024-10:09:08] [I] [TRT] Domain:
[02/25/2024-10:09:08] [I] [TRT] Model version: 0
[02/25/2024-10:09:08] [I] [TRT] Doc string:
[02/25/2024-10:09:08] [I] [TRT] ----------------------------------------------------------------
[02/25/2024-10:09:08] [W] [TRT] onnx2trt_utils.cpp:365: Your ONNX model has been generated with INT64 weights, while TensorRT does not natively support INT64. Attempting to cast down to INT32.
[02/25/2024-10:09:08] [I] [TRT] No importer registered for op: TRTInstanceNormalization. Attempting to import as plugin.
[02/25/2024-10:09:08] [I] [TRT] Searching for plugin: TRTInstanceNormalization, plugin_version: 1, plugin_namespace:
[02/25/2024-10:09:08] [I] [TRT] Successfully created plugin: TRTInstanceNormalization
[02/25/2024-10:09:08] [I] [TRT] No importer registered for op: TRTInstanceNormalization. Attempting to import as plugin.
[02/25/2024-10:09:08] [I] [TRT] Searching for plugin: TRTInstanceNormalization, plugin_version: 1, plugin_namespace:
[02/25/2024-10:09:08] [I] [TRT] Successfully created plugin: TRTInstanceNormalization
[02/25/2024-10:09:08] [I] [TRT] No importer registered for op: TRTInstanceNormalization. Attempting to import as plugin.
[02/25/2024-10:09:08] [I] [TRT] Searching for plugin: TRTInstanceNormalization, plugin_version: 1, plugin_namespace:
[02/25/2024-10:09:08] [I] [TRT] Successfully created plugin: TRTInstanceNormalization
[02/25/2024-10:09:08] [I] [TRT] No importer registered for op: TRTInstanceNormalization. Attempting to import as plugin.
[02/25/2024-10:09:08] [I] [TRT] Searching for plugin: TRTInstanceNormalization, plugin_version: 1, plugin_namespace:
[02/25/2024-10:09:08] [I] [TRT] Successfully created plugin: TRTInstanceNormalization
[02/25/2024-10:09:08] [E] [TRT] ModelImporter.cpp:748: While parsing node number 160 [Range -> "onnx::Reshape_864"]:
[02/25/2024-10:09:08] [E] [TRT] ModelImporter.cpp:749: --- Begin node ---
[02/25/2024-10:09:08] [E] [TRT] ModelImporter.cpp:750: input: "onnx::Range_8968"
input: "onnx::Range_862"
input: "onnx::Range_8968"
output: "onnx::Reshape_864"
name: "Range_160"
op_type: "Range"

[02/25/2024-10:09:08] [E] [TRT] ModelImporter.cpp:751: --- End node ---
[02/25/2024-10:09:08] [E] [TRT] ModelImporter.cpp:754: ERROR: ModelImporter.cpp:162 In function parseGraph:
[6] Invalid Node - Range_160
All inputs to range should be initializers.
[02/25/2024-10:09:08] [E] Failed to parse onnx file
[02/25/2024-10:09:08] [I] Finish parsing network model
[02/25/2024-10:09:08] [E] Parsing model failed
[02/25/2024-10:09:08] [E] Failed to create engine from model.
[02/25/2024-10:09:08] [E] Engine set up failed

[shuchang0714]

@wuzujiong 使用tensorrt加速后，速度正常

请问下你dino转trt fp16的精度能对齐嘛

[sherryhsy]

@Zong-my 我最近使用mmdeploy可以成功转换出onnx，但是它的检测速度比pytorch慢很多。

请问是怎么转换的呢，我尝试maskdino转onnx，转换后输出结果是错误的

[wulongjian]

I succeeded in converting to onnx. I recommend converting it to torch.jit.trace first and then exporting it to onnx.

hello、Could ou share your conversion code and environment configuration? I didn't succeed just use torch.jit.trace(model, img)

[pof1423]

I succeeded in converting to onnx. I recommend converting it to torch.jit.trace first and then exporting it to onnx.

hello、Could ou share your conversion code and environment configuration? I didn't succeed just use torch.jit.trace(model, img)

In ms_deform_attn.py change line 124 to 126 as output = ms_deform_attn_core_pytorch(value, input_spatial_shapes, sampling_locations, attention_weights)

and import ms_deform_attn_core_pytorch with adding from ..functions.ms_deform_attn_func import ms_deform_attn_core_pytorch

[Ochre-amber]

@wuzujiong @Sencc
你好，我尝试用mmdeploy导出onnx但是失败了

from mmdeploy.apis import torch2onnx
from mmdeploy.backend.sdk.export_info import export2SDK

img = 'demo/demo.jpg'
work_dir = 'mmdeploy_models/mmdet/onnx'
save_file = 'end2end.onnx'
deploy_cfg = '../mmdeploy/configs/mmdet/detection/detection_onnxruntime_dynamic.py'
model_cfg = 'configs/dino/dino-4scale_r50_8xb2-12e_coco.py'
model_checkpoint = ('dino-4scale_r50_8xb2-12e_coco_20221202_182705-55b2bba2.pth')
device = 'cuda'

# 1. convert model to onnx
torch2onnx(img, work_dir, save_file, deploy_cfg, model_cfg,
           model_checkpoint, device)

# 2. extract pipeline info for inference by MMDeploy SDK
export2SDK(deploy_cfg, model_cfg, work_dir, pth=model_checkpoint,
           device=device)

没有输出的模型

D:\Coding\anaconda\envs\tensoryolo\python.exe D:\Programs\last\project\mmdetection\to_onnx.py 
05/05 11:34:44 - mmengine - WARNING - Failed to search registry with scope "mmdet" in the "Codebases" registry tree. As a workaround, the current "Codebases" registry in "mmdeploy" is used to build instance. This may cause unexpected failure when running the built modules. Please check whether "mmdet" is a correct scope, or whether the registry is initialized.
05/05 11:34:44 - mmengine - WARNING - Failed to search registry with scope "mmdet" in the "mmdet_tasks" registry tree. As a workaround, the current "mmdet_tasks" registry in "mmdeploy" is used to build instance. This may cause unexpected failure when running the built modules. Please check whether "mmdet" is a correct scope, or whether the registry is initialized.
Loads checkpoint by local backend from path: dino-4scale_r50_8xb2-12e_coco_20221202_182705-55b2bba2.pth
05/05 11:34:46 - mmengine - WARNING - DeprecationWarning: get_onnx_config will be deprecated in the future. 
05/05 11:34:46 - mmengine - INFO - Export PyTorch model to ONNX: mmdeploy_models/mmdet/onnx\end2end.onnx.
D:\Coding\anaconda\envs\tensoryolo\lib\site-packages\mmdeploy\core\optimizers\function_marker.py:160: TracerWarning: Converting a tensor to a Python integer might cause the trace to be incorrect. We can't record the data flow of Python values, so this value will be treated as a constant in the future. This means that the trace might not generalize to other inputs!
  ys_shape = tuple(int(s) for s in ys.shape)
D:\Programs\last\project\mmdetection\mmdet\models\detectors\deformable_detr.py:157: TracerWarning: Converting a tensor to a Python boolean might cause the trace to be incorrect. We can't record the data flow of Python values, so this value will be treated as a constant in the future. This means that the trace might not generalize to other inputs!
  s[0] == input_img_h and s[1] == input_img_w for s in img_shape_list
D:\Programs\last\project\mmdetection\mmdet\models\detectors\deformable_detr.py:156: TracerWarning: Converting a tensor to a Python boolean might cause the trace to be incorrect. We can't record the data flow of Python values, so this value will be treated as a constant in the future. This means that the trace might not generalize to other inputs!
  same_shape_flag = all([
D:\Programs\last\project\mmdetection\mmdet\models\layers\transformer\deformable_detr_layers.py:103: TracerWarning: Iterating over a tensor might cause the trace to be incorrect. Passing a tensor of different shape won't change the number of iterations executed (and might lead to errors or silently give incorrect results).
  for lvl, (H, W) in enumerate(spatial_shapes):
D:\Coding\anaconda\envs\tensoryolo\lib\site-packages\mmdeploy\pytorch\functions\linspace.py:15: TracerWarning: Converting a tensor to a Python boolean might cause the trace to be incorrect. We can't record the data flow of Python values, so this value will be treated as a constant in the future. This means that the trace might not generalize to other inputs!
  if steps == 1:
D:\Coding\anaconda\envs\tensoryolo\lib\site-packages\torch\functional.py:504: UserWarning: torch.meshgrid: in an upcoming release, it will be required to pass the indexing argument. (Triggered internally at C:\cb\pytorch_1000000000000\work\aten\src\ATen\native\TensorShape.cpp:3527.)
  return _VF.meshgrid(tensors, **kwargs)  # type: ignore[attr-defined]
D:\Coding\anaconda\envs\tensoryolo\lib\site-packages\mmcv\ops\multi_scale_deform_attn.py:335: TracerWarning: Converting a tensor to a Python boolean might cause the trace to be incorrect. We can't record the data flow of Python values, so this value will be treated as a constant in the future. This means that the trace might not generalize to other inputs!
  assert (spatial_shapes[:, 0] * spatial_shapes[:, 1]).sum() == num_value
D:\Coding\anaconda\envs\tensoryolo\lib\site-packages\mmcv\ops\multi_scale_deform_attn.py:351: TracerWarning: Converting a tensor to a Python boolean might cause the trace to be incorrect. We can't record the data flow of Python values, so this value will be treated as a constant in the future. This means that the trace might not generalize to other inputs!
  if reference_points.shape[-1] == 2:
D:\Coding\anaconda\envs\tensoryolo\lib\site-packages\mmcv\ops\multi_scale_deform_attn.py:136: TracerWarning: Iterating over a tensor might cause the trace to be incorrect. Passing a tensor of different shape won't change the number of iterations executed (and might lead to errors or silently give incorrect results).
  value_list = value.split([H_ * W_ for H_, W_ in value_spatial_shapes],
D:\Coding\anaconda\envs\tensoryolo\lib\site-packages\mmcv\ops\multi_scale_deform_attn.py:140: TracerWarning: Iterating over a tensor might cause the trace to be incorrect. Passing a tensor of different shape won't change the number of iterations executed (and might lead to errors or silently give incorrect results).
  for level, (H_, W_) in enumerate(value_spatial_shapes):
D:\Programs\last\project\mmdetection\mmdet\models\detectors\deformable_detr.py:489: TracerWarning: Iterating over a tensor might cause the trace to be incorrect. Passing a tensor of different shape won't change the number of iterations executed (and might lead to errors or silently give incorrect results).
  for lvl, HW in enumerate(spatial_shapes):
D:\Programs\last\project\mmdetection\mmdet\models\detectors\deformable_detr.py:490: TracerWarning: Iterating over a tensor might cause the trace to be incorrect. Passing a tensor of different shape won't change the number of iterations executed (and might lead to errors or silently give incorrect results).
  H, W = HW
D:\Coding\anaconda\envs\tensoryolo\lib\site-packages\mmdeploy\pytorch\functions\topk.py:28: TracerWarning: torch.tensor results are registered as constants in the trace. You can safely ignore this warning if you use this function to create tensors out of constant variables that would be the same every time you call this function. In any other case, this might cause the trace to be incorrect.
  k = torch.tensor(k, device=input.device, dtype=torch.long)
D:\Programs\last\project\mmdetection\mmdet\models\layers\transformer\dino_layers.py:74: TracerWarning: Converting a tensor to a Python boolean might cause the trace to be incorrect. We can't record the data flow of Python values, so this value will be treated as a constant in the future. This means that the trace might not generalize to other inputs!
  if reference_points.shape[-1] == 4:
D:\Programs\last\project\mmdetection\mmdet\models\layers\transformer\utils.py:80: TracerWarning: Converting a tensor to a Python boolean might cause the trace to be incorrect. We can't record the data flow of Python values, so this value will be treated as a constant in the future. This means that the trace might not generalize to other inputs!
  if coord_tensor.size(-1) == 2:
D:\Programs\last\project\mmdetection\mmdet\models\layers\transformer\utils.py:82: TracerWarning: Converting a tensor to a Python boolean might cause the trace to be incorrect. We can't record the data flow of Python values, so this value will be treated as a constant in the future. This means that the trace might not generalize to other inputs!
  elif coord_tensor.size(-1) == 4:
D:\Coding\anaconda\envs\tensoryolo\lib\site-packages\mmcv\ops\multi_scale_deform_attn.py:357: TracerWarning: Converting a tensor to a Python boolean might cause the trace to be incorrect. We can't record the data flow of Python values, so this value will be treated as a constant in the future. This means that the trace might not generalize to other inputs!
  elif reference_points.shape[-1] == 4:
D:\Programs\last\project\mmdetection\mmdet\models\layers\transformer\dino_layers.py:101: TracerWarning: Converting a tensor to a Python boolean might cause the trace to be incorrect. We can't record the data flow of Python values, so this value will be treated as a constant in the future. This means that the trace might not generalize to other inputs!
  assert reference_points.shape[-1] == 4
D:\Programs\last\project\mmdetection\mmdet\models\detectors\dino.py:277: TracerWarning: Using len to get tensor shape might cause the trace to be incorrect. Recommended usage would be tensor.shape[0]. Passing a tensor of different shape might lead to errors or silently give incorrect results.
  if len(query) == self.num_queries:
D:\Programs\last\project\mmdetection\mmdet\models\detectors\dino.py:286: TracerWarning: Using len to get tensor shape might cause the trace to be incorrect. Recommended usage would be tensor.shape[0]. Passing a tensor of different shape might lead to errors or silently give incorrect results.
  hidden_states=inter_states, references=list(references))
D:\Programs\last\project\mmdetection\mmdet\models\detectors\dino.py:286: TracerWarning: Iterating over a tensor might cause the trace to be incorrect. Passing a tensor of different shape won't change the number of iterations executed (and might lead to errors or silently give incorrect results).
  hidden_states=inter_states, references=list(references))
D:\Programs\last\project\mmdetection\mmdet\models\dense_heads\deformable_detr_head.py:124: TracerWarning: Converting a tensor to a Python boolean might cause the trace to be incorrect. We can't record the data flow of Python values, so this value will be treated as a constant in the future. This means that the trace might not generalize to other inputs!
  if reference.shape[-1] == 4:
D:\Coding\anaconda\envs\tensoryolo\lib\site-packages\mmdeploy\codebase\mmdet\models\dense_heads\detr_head.py:31: TracerWarning: Using len to get tensor shape might cause the trace to be incorrect. Recommended usage would be tensor.shape[0]. Passing a tensor of different shape might lead to errors or silently give incorrect results.
  max_per_img = self.test_cfg.get('max_per_img', len(cls_scores[0]))
D:\Coding\anaconda\envs\tensoryolo\lib\site-packages\mmdeploy\pytorch\functions\topk.py:28: TracerWarning: torch.tensor results are registered as constants in the trace. You can safely ignore this warning if you use this function to create tensors out of constant variables that would be the same every time you call this function. In any other case, this might cause the trace to be incorrect.
  k = torch.tensor(k, device=input.device, dtype=torch.long)

Process finished with exit code -1073741819 (0xC0000005)

请问您是怎么导出的？

[Sencc]

@wuzujiong @Sencc 你好，我尝试用mmdeploy导出onnx但是失败了

from mmdeploy.apis import torch2onnx
from mmdeploy.backend.sdk.export_info import export2SDK

img = 'demo/demo.jpg'
work_dir = 'mmdeploy_models/mmdet/onnx'
save_file = 'end2end.onnx'
deploy_cfg = '../mmdeploy/configs/mmdet/detection/detection_onnxruntime_dynamic.py'
model_cfg = 'configs/dino/dino-4scale_r50_8xb2-12e_coco.py'
model_checkpoint = ('dino-4scale_r50_8xb2-12e_coco_20221202_182705-55b2bba2.pth')
device = 'cuda'

# 1. convert model to onnx
torch2onnx(img, work_dir, save_file, deploy_cfg, model_cfg,
           model_checkpoint, device)

# 2. extract pipeline info for inference by MMDeploy SDK
export2SDK(deploy_cfg, model_cfg, work_dir, pth=model_checkpoint,
           device=device)

没有输出的模型

D:\Coding\anaconda\envs\tensoryolo\python.exe D:\Programs\last\project\mmdetection\to_onnx.py 
05/05 11:34:44 - mmengine - WARNING - Failed to search registry with scope "mmdet" in the "Codebases" registry tree. As a workaround, the current "Codebases" registry in "mmdeploy" is used to build instance. This may cause unexpected failure when running the built modules. Please check whether "mmdet" is a correct scope, or whether the registry is initialized.
05/05 11:34:44 - mmengine - WARNING - Failed to search registry with scope "mmdet" in the "mmdet_tasks" registry tree. As a workaround, the current "mmdet_tasks" registry in "mmdeploy" is used to build instance. This may cause unexpected failure when running the built modules. Please check whether "mmdet" is a correct scope, or whether the registry is initialized.
Loads checkpoint by local backend from path: dino-4scale_r50_8xb2-12e_coco_20221202_182705-55b2bba2.pth
05/05 11:34:46 - mmengine - WARNING - DeprecationWarning: get_onnx_config will be deprecated in the future. 
05/05 11:34:46 - mmengine - INFO - Export PyTorch model to ONNX: mmdeploy_models/mmdet/onnx\end2end.onnx.
D:\Coding\anaconda\envs\tensoryolo\lib\site-packages\mmdeploy\core\optimizers\function_marker.py:160: TracerWarning: Converting a tensor to a Python integer might cause the trace to be incorrect. We can't record the data flow of Python values, so this value will be treated as a constant in the future. This means that the trace might not generalize to other inputs!
  ys_shape = tuple(int(s) for s in ys.shape)
D:\Programs\last\project\mmdetection\mmdet\models\detectors\deformable_detr.py:157: TracerWarning: Converting a tensor to a Python boolean might cause the trace to be incorrect. We can't record the data flow of Python values, so this value will be treated as a constant in the future. This means that the trace might not generalize to other inputs!
  s[0] == input_img_h and s[1] == input_img_w for s in img_shape_list
D:\Programs\last\project\mmdetection\mmdet\models\detectors\deformable_detr.py:156: TracerWarning: Converting a tensor to a Python boolean might cause the trace to be incorrect. We can't record the data flow of Python values, so this value will be treated as a constant in the future. This means that the trace might not generalize to other inputs!
  same_shape_flag = all([
D:\Programs\last\project\mmdetection\mmdet\models\layers\transformer\deformable_detr_layers.py:103: TracerWarning: Iterating over a tensor might cause the trace to be incorrect. Passing a tensor of different shape won't change the number of iterations executed (and might lead to errors or silently give incorrect results).
  for lvl, (H, W) in enumerate(spatial_shapes):
D:\Coding\anaconda\envs\tensoryolo\lib\site-packages\mmdeploy\pytorch\functions\linspace.py:15: TracerWarning: Converting a tensor to a Python boolean might cause the trace to be incorrect. We can't record the data flow of Python values, so this value will be treated as a constant in the future. This means that the trace might not generalize to other inputs!
  if steps == 1:
D:\Coding\anaconda\envs\tensoryolo\lib\site-packages\torch\functional.py:504: UserWarning: torch.meshgrid: in an upcoming release, it will be required to pass the indexing argument. (Triggered internally at C:\cb\pytorch_1000000000000\work\aten\src\ATen\native\TensorShape.cpp:3527.)
  return _VF.meshgrid(tensors, **kwargs)  # type: ignore[attr-defined]
D:\Coding\anaconda\envs\tensoryolo\lib\site-packages\mmcv\ops\multi_scale_deform_attn.py:335: TracerWarning: Converting a tensor to a Python boolean might cause the trace to be incorrect. We can't record the data flow of Python values, so this value will be treated as a constant in the future. This means that the trace might not generalize to other inputs!
  assert (spatial_shapes[:, 0] * spatial_shapes[:, 1]).sum() == num_value
D:\Coding\anaconda\envs\tensoryolo\lib\site-packages\mmcv\ops\multi_scale_deform_attn.py:351: TracerWarning: Converting a tensor to a Python boolean might cause the trace to be incorrect. We can't record the data flow of Python values, so this value will be treated as a constant in the future. This means that the trace might not generalize to other inputs!
  if reference_points.shape[-1] == 2:
D:\Coding\anaconda\envs\tensoryolo\lib\site-packages\mmcv\ops\multi_scale_deform_attn.py:136: TracerWarning: Iterating over a tensor might cause the trace to be incorrect. Passing a tensor of different shape won't change the number of iterations executed (and might lead to errors or silently give incorrect results).
  value_list = value.split([H_ * W_ for H_, W_ in value_spatial_shapes],
D:\Coding\anaconda\envs\tensoryolo\lib\site-packages\mmcv\ops\multi_scale_deform_attn.py:140: TracerWarning: Iterating over a tensor might cause the trace to be incorrect. Passing a tensor of different shape won't change the number of iterations executed (and might lead to errors or silently give incorrect results).
  for level, (H_, W_) in enumerate(value_spatial_shapes):
D:\Programs\last\project\mmdetection\mmdet\models\detectors\deformable_detr.py:489: TracerWarning: Iterating over a tensor might cause the trace to be incorrect. Passing a tensor of different shape won't change the number of iterations executed (and might lead to errors or silently give incorrect results).
  for lvl, HW in enumerate(spatial_shapes):
D:\Programs\last\project\mmdetection\mmdet\models\detectors\deformable_detr.py:490: TracerWarning: Iterating over a tensor might cause the trace to be incorrect. Passing a tensor of different shape won't change the number of iterations executed (and might lead to errors or silently give incorrect results).
  H, W = HW
D:\Coding\anaconda\envs\tensoryolo\lib\site-packages\mmdeploy\pytorch\functions\topk.py:28: TracerWarning: torch.tensor results are registered as constants in the trace. You can safely ignore this warning if you use this function to create tensors out of constant variables that would be the same every time you call this function. In any other case, this might cause the trace to be incorrect.
  k = torch.tensor(k, device=input.device, dtype=torch.long)
D:\Programs\last\project\mmdetection\mmdet\models\layers\transformer\dino_layers.py:74: TracerWarning: Converting a tensor to a Python boolean might cause the trace to be incorrect. We can't record the data flow of Python values, so this value will be treated as a constant in the future. This means that the trace might not generalize to other inputs!
  if reference_points.shape[-1] == 4:
D:\Programs\last\project\mmdetection\mmdet\models\layers\transformer\utils.py:80: TracerWarning: Converting a tensor to a Python boolean might cause the trace to be incorrect. We can't record the data flow of Python values, so this value will be treated as a constant in the future. This means that the trace might not generalize to other inputs!
  if coord_tensor.size(-1) == 2:
D:\Programs\last\project\mmdetection\mmdet\models\layers\transformer\utils.py:82: TracerWarning: Converting a tensor to a Python boolean might cause the trace to be incorrect. We can't record the data flow of Python values, so this value will be treated as a constant in the future. This means that the trace might not generalize to other inputs!
  elif coord_tensor.size(-1) == 4:
D:\Coding\anaconda\envs\tensoryolo\lib\site-packages\mmcv\ops\multi_scale_deform_attn.py:357: TracerWarning: Converting a tensor to a Python boolean might cause the trace to be incorrect. We can't record the data flow of Python values, so this value will be treated as a constant in the future. This means that the trace might not generalize to other inputs!
  elif reference_points.shape[-1] == 4:
D:\Programs\last\project\mmdetection\mmdet\models\layers\transformer\dino_layers.py:101: TracerWarning: Converting a tensor to a Python boolean might cause the trace to be incorrect. We can't record the data flow of Python values, so this value will be treated as a constant in the future. This means that the trace might not generalize to other inputs!
  assert reference_points.shape[-1] == 4
D:\Programs\last\project\mmdetection\mmdet\models\detectors\dino.py:277: TracerWarning: Using len to get tensor shape might cause the trace to be incorrect. Recommended usage would be tensor.shape[0]. Passing a tensor of different shape might lead to errors or silently give incorrect results.
  if len(query) == self.num_queries:
D:\Programs\last\project\mmdetection\mmdet\models\detectors\dino.py:286: TracerWarning: Using len to get tensor shape might cause the trace to be incorrect. Recommended usage would be tensor.shape[0]. Passing a tensor of different shape might lead to errors or silently give incorrect results.
  hidden_states=inter_states, references=list(references))
D:\Programs\last\project\mmdetection\mmdet\models\detectors\dino.py:286: TracerWarning: Iterating over a tensor might cause the trace to be incorrect. Passing a tensor of different shape won't change the number of iterations executed (and might lead to errors or silently give incorrect results).
  hidden_states=inter_states, references=list(references))
D:\Programs\last\project\mmdetection\mmdet\models\dense_heads\deformable_detr_head.py:124: TracerWarning: Converting a tensor to a Python boolean might cause the trace to be incorrect. We can't record the data flow of Python values, so this value will be treated as a constant in the future. This means that the trace might not generalize to other inputs!
  if reference.shape[-1] == 4:
D:\Coding\anaconda\envs\tensoryolo\lib\site-packages\mmdeploy\codebase\mmdet\models\dense_heads\detr_head.py:31: TracerWarning: Using len to get tensor shape might cause the trace to be incorrect. Recommended usage would be tensor.shape[0]. Passing a tensor of different shape might lead to errors or silently give incorrect results.
  max_per_img = self.test_cfg.get('max_per_img', len(cls_scores[0]))
D:\Coding\anaconda\envs\tensoryolo\lib\site-packages\mmdeploy\pytorch\functions\topk.py:28: TracerWarning: torch.tensor results are registered as constants in the trace. You can safely ignore this warning if you use this function to create tensors out of constant variables that would be the same every time you call this function. In any other case, this might cause the trace to be incorrect.
  k = torch.tensor(k, device=input.device, dtype=torch.long)

Process finished with exit code -1073741819 (0xC0000005)

请问您是怎么导出的？

要看一下onnx的版本，大于1.8才行

[Sencc]

@wuzujiong 使用tensorrt加速后，速度正常

请问下你dino转trt fp16的精度能对齐嘛

没有做具体测试，我感觉差不多

[Ochre-amber]

@Sencc onnx版本为1.16.0，似乎不是这个问题。

pip show onnx

Name: onnx
Version: 1.16.0
Summary: Open Neural Network Exchange
Home-page: 
Author: 
Author-email: ONNX Contributors <[email protected]>
License: Apache License v2.0
Location: d:\coding\anaconda\envs\tensoryolo\lib\site-packages
Requires: numpy, protobuf
Required-by: mmdeploy, onnx-graphsurgeon, onnx-simplifier

具体环境为：

05/05 09:53:24 - mmengine - INFO - **********Environmental information**********
05/05 09:53:28 - mmengine - INFO - sys.platform: win32
05/05 09:53:28 - mmengine - INFO - Python: 3.8.18 | packaged by conda-forge | (default, Dec 23 2023, 17:17:17) [MSC v.1929 64 bit (AMD64)]
05/05 09:53:28 - mmengine - INFO - CUDA available: True
05/05 09:53:28 - mmengine - INFO - MUSA available: False
05/05 09:53:28 - mmengine - INFO - numpy_random_seed: 2147483648
05/05 09:53:28 - mmengine - INFO - GPU 0: NVIDIA GeForce RTX 4060 Laptop GPU
05/05 09:53:28 - mmengine - INFO - CUDA_HOME: C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v12.1
05/05 09:53:28 - mmengine - INFO - NVCC: Cuda compilation tools, release 12.1, V12.1.105
05/05 09:53:28 - mmengine - INFO - MSVC: 用于 x64 的 Microsoft (R) C/C++ 优化编译器 19.39.33523 版
05/05 09:53:28 - mmengine - INFO - GCC: n/a
05/05 09:53:28 - mmengine - INFO - PyTorch: 2.1.2
05/05 09:53:28 - mmengine - INFO - PyTorch compiling details: PyTorch built with:
  - C++ Version: 199711
  - MSVC 192930151
  - Intel(R) Math Kernel Library Version 2020.0.2 Product Build 20200624 for Intel(R) 64 architecture applications
  - Intel(R) MKL-DNN v3.1.1 (Git Hash 64f6bcbcbab628e96f33a62c3e975f8535a7bde4)
  - OpenMP 2019
  - LAPACK is enabled (usually provided by MKL)
  - CPU capability usage: AVX2
  - CUDA Runtime 12.1
  - NVCC architecture flags: -gencode;arch=compute_50,code=sm_50;-gencode;arch=compute_60,code=sm_60;-gencode;arch=compute_61,code=sm_61;-gencode;arch=compute_70,code=sm_70;-gencode;arch=compute_75,code=sm_75;-gencode;arch=compute_80,code=sm_80;-gencode;arch=compute_86,code=sm_86;-gencode;arch=compute_90,code=sm_90
  - CuDNN 8.8.1  (built against CUDA 12.0)
  - Magma 2.5.4
  - Build settings: BLAS_INFO=mkl, BUILD_TYPE=Release, CUDA_VERSION=12.1, CUDNN_VERSION=8.8.1, CXX_COMPILER=C:/cb/pytorch_1000000000000/work/tmp_bin/sccache-cl.exe, CXX_FLAGS=/DWIN32 /D_WINDOWS /GR /EHsc /bigobj /FS -DUSE_PTHREADPOOL -DNDEBUG -DUSE_KINETO -DLIBKINETO_NOCUPTI -DLIBKINETO_NOROCTRACER -DUSE_FBGEMM -DUSE_XNNPACK -DSYMBOLICATE_MOBILE_DEBUG_HANDLE /utf-8 /wd4624 /wd4068 /wd4067 /wd4267 /wd4661 /wd4717 /wd4244 /wd4804 /wd4273, LAPACK_INFO=mkl, PERF_WITH_AVX=1, PERF_WITH_AVX2=1, PERF_WITH_AVX512=1, TORCH_DISABLE_GPU_ASSERTS=OFF, TORCH_VERSION=2.1.2, USE_CUDA=ON, USE_CUDNN=ON, USE_EXCEPTION_PTR=1, USE_GFLAGS=OFF, USE_GLOG=OFF, USE_MKL=ON, USE_MKLDNN=ON, USE_MPI=OFF, USE_NCCL=OFF, USE_NNPACK=OFF, USE_OPENMP=ON, USE_ROCM=OFF,

05/05 09:53:28 - mmengine - INFO - TorchVision: 0.16.2
05/05 09:53:28 - mmengine - INFO - OpenCV: 4.8.1
05/05 09:53:28 - mmengine - INFO - MMEngine: 0.10.4
05/05 09:53:28 - mmengine - INFO - MMCV: 2.1.0
05/05 09:53:28 - mmengine - INFO - MMCV Compiler: MSVC 192930148
05/05 09:53:28 - mmengine - INFO - MMCV CUDA Compiler: 12.1
05/05 09:53:28 - mmengine - INFO - MMDeploy: 1.0.0+bc75c9d
05/05 09:53:28 - mmengine - INFO -

05/05 09:53:28 - mmengine - INFO - **********Backend information**********
05/05 09:53:29 - mmengine - INFO - tensorrt:    8.6.1
05/05 09:53:29 - mmengine - INFO - tensorrt custom ops: Available
05/05 09:53:29 - mmengine - INFO - ONNXRuntime: 1.8.1
05/05 09:53:29 - mmengine - INFO - ONNXRuntime-gpu:     None
05/05 09:53:29 - mmengine - INFO - ONNXRuntime custom ops:      Available
05/05 09:53:29 - mmengine - INFO - pplnn:       None
05/05 09:53:29 - mmengine - INFO - ncnn:        None
05/05 09:53:29 - mmengine - INFO - snpe:        None
05/05 09:53:29 - mmengine - INFO - openvino:    None
05/05 09:53:29 - mmengine - INFO - torchscript: 2.1.2+cu121
05/05 09:53:29 - mmengine - INFO - torchscript custom ops:      NotAvailable
05/05 09:53:29 - mmengine - INFO - rknn-toolkit:        None
05/05 09:53:29 - mmengine - INFO - rknn-toolkit2:       None
05/05 09:53:29 - mmengine - INFO - ascend:      None
05/05 09:53:29 - mmengine - INFO - coreml:      None
05/05 09:53:29 - mmengine - INFO - tvm: None
05/05 09:53:29 - mmengine - INFO - vacc:        None
05/05 09:53:29 - mmengine - INFO -

05/05 09:53:29 - mmengine - INFO - **********Codebase information**********
05/05 09:53:29 - mmengine - INFO - mmdet:       3.3.0
05/05 09:53:29 - mmengine - INFO - mmseg:       None
05/05 09:53:29 - mmengine - INFO - mmcls:       None
05/05 09:53:29 - mmengine - INFO - mmocr:       None
05/05 09:53:29 - mmengine - INFO - mmedit:      None
05/05 09:53:29 - mmengine - INFO - mmdet3d:     None
05/05 09:53:29 - mmengine - INFO - mmpose:      None
05/05 09:53:29 - mmengine - INFO - mmrotate:    None
05/05 09:53:29 - mmengine - INFO - mmaction:    None
05/05 09:53:29 - mmengine - INFO - mmrazor:     None

请问可能是环境配置问题嘛？您的环境是什么样的？

[Sencc]

@Ochre-amber 这是我的环境

`05/07 11:06:53 - mmengine - INFO - TorchVision: 0.14.1+cu117
05/07 11:06:53 - mmengine - INFO - OpenCV: 4.9.0
05/07 11:06:53 - mmengine - INFO - MMEngine: 0.10.2
05/07 11:06:53 - mmengine - INFO - MMCV: 2.1.0
05/07 11:06:53 - mmengine - INFO - MMCV Compiler: GCC 9.3
05/07 11:06:53 - mmengine - INFO - MMCV CUDA Compiler: 11.7
05/07 11:06:53 - mmengine - INFO - MMDeploy: 1.3.1+44ebd17
05/07 11:06:53 - mmengine - INFO -

05/07 11:06:53 - mmengine - INFO - Backend information
05/07 11:06:54 - mmengine - INFO - tensorrt: 8.6.1
05/07 11:06:54 - mmengine - INFO - tensorrt custom ops: Available
05/07 11:06:54 - mmengine - INFO - ONNXRuntime: None
05/07 11:06:54 - mmengine - INFO - ONNXRuntime-gpu: 1.16.3
05/07 11:06:54 - mmengine - INFO - ONNXRuntime custom ops: Available
05/07 11:06:54 - mmengine - INFO - pplnn: None
05/07 11:06:54 - mmengine - INFO - ncnn: None
05/07 11:06:54 - mmengine - INFO - snpe: None
05/07 11:06:54 - mmengine - INFO - openvino: None
05/07 11:06:54 - mmengine - INFO - torchscript: 1.13.1+cu117
05/07 11:06:54 - mmengine - INFO - torchscript custom ops: NotAvailable
05/07 11:06:54 - mmengine - INFO - rknn-toolkit: None
05/07 11:06:54 - mmengine - INFO - rknn-toolkit2: None
05/07 11:06:54 - mmengine - INFO - ascend: None
05/07 11:06:54 - mmengine - INFO - coreml: None
05/07 11:06:54 - mmengine - INFO - tvm: None
05/07 11:06:54 - mmengine - INFO - vacc: None
05/07 11:06:54 - mmengine - INFO - `

[chaojilelegogogo]

非常棒的工作，请问可否提供pth模型转onnx的脚本呢？

你成功了吗？我按照上面的几种方法都失败了。

[elesun2018]

@wuzujiong 使用tensorrt加速后，速度正常

你好，请问你所说的tensorrt加速后，速度正常，这个速度正常是dino的tensorrt推理速度回到了dino的pytorch推理速度？还是说dino经过onnx然后tensorrt加速后推理速度就远远大于pytorch推理速度呢？感谢感谢！

[elesun2018]

@wuzujiong 使用tensorrt加速后，速度正常

您好，能麻烦加你一个联系方式吗，现在一个项目非常需要dino的tensorrt部署，我们能提供满意的报酬

[Sencc]

@elesun2018 前面有个兄弟提到他转换了onnx后，推理速度比pytorch还慢，但是把dino转成了tensorrt格式后，推理速度比两者都快了很多。另外，dino的tensorrt部署使用mmdetection框架即可，我已成功运行