ai.onnxruntime.OrtException: This tensor is not representable in Java, it's too big

[anandvsr]

get This tensor is not representable in Java, it's too big, while inference fasterrcnn_resnet50_fpn model in onnx.

export_onnx.py (Python Source):

    model = torchvision.models.detection.fasterrcnn_resnet50_fpn(pretrained=True)
    x = [torch.rand(3, 1970, 1080)]
    torch.onnx.export(model, x, "faster_rcnn.onnx", opset_version = 11)

build.gradle (Gradle Source):

    dependencies {
          implementation group: 'com.microsoft.onnxruntime', name: 'onnxruntime', version: '1.7.0'
    }

Pridiction.java (Java Source):

47| try(OrtSession session = env.createSession(modelPath.toFile().getAbsolutePath(), 
48|                    new OrtSession.SessionOptions())) {
49|
50|     OnnxTensor t1 = OnnxTensor.createTensor(env, sourceData, dimensions);
51|     OrtSession.Result result = session.run(Collections.singletonMap("image.1", t1));
52|
53|     for (int i = 0; i < result.size(); i++) {
54|         System.out.println(result.get(i).getValue());
55|     }
56|
57|     result.close();
58|     t1.close();
59| }

Exception while inference:

ai.onnxruntime.OrtException: This tensor is not representable in Java, it's too big - shape = [0, 4]
	at ai.onnxruntime.TensorInfo.makeCarrier(TensorInfo.java:171)
	at ai.onnxruntime.OnnxTensor.getValue(OnnxTensor.java:99)
	at onnx.predict.Pridiction.main(Pridiction.java:54)

Refered sites:

       1. https://www.onnxruntime.ai/docs/reference/api/java-api.html
       2. https://pytorch.org/vision/stable/models.html#faster-r-cnn

[Craigacp]

Well that's a misleading error message, as the check is for shape validation, not just that it's too big. That should be fixed.

However the shape is reported as being [0,4]. I don't think that's valid because it won't contain any elements, and so some kind of exception is expected (or I suppose it could return an empty array). Does this image contain anything that's been detected? If so, what is returned by the Python ONNX Runtime API for that image?

[anandvsr]

Prediction via Onnx runtime in python:

Why the inference return empty array?
please help to clarify!

Prediction in PyTorch (Python source):

[Craigacp]

I don't know why the output differs between pytorch & ORT. Maybe someone who has a better understanding of the internals could take a look. cc @pranavsharma

[anandvsr]

Issue Resolved!

The Exception

    ai.onnxruntime.OrtException: This tensor is not representable in Java, it's too big - shape = [0, 4]
	    at ai.onnxruntime.TensorInfo.makeCarrier(TensorInfo.java:171)
	    at ai.onnxruntime.OnnxTensor.getValue(OnnxTensor.java:99)
	    at onnx.predict.Pridiction.main(Pridiction.java:54)

Raise when it won't contain any elements!
Thank you @Craigacp

export_onnx.py (Python Source):

The output differ while the shape was fixed!

To resolve that, export the model with dynamic size

w, h = 1080, 1970
x = [torch.rand(3, w, h)]
onnx_file = resources/model/fasterrcnn_resnet50_fpn.onnx"
torch.onnx.export(model, x, onnx_file,
                  input_names=['input'],
                  output_names=['output'],
                  opset_version=11,
                  dynamic_axes={'input': {1: 'height', 2: 'width'}},
                  enable_onnx_checker=True)

Prediction via Onnx runtime after dynamic size in python:

PS:

torchvision.transforms.functional.to_tensor, it return normalised value.

while using the non normalised value!

The same output came in JAVA 👍

Have any conclusion @Craigacp @pranavsharma

[Craigacp]

Good, I'm glad you've figured it out. I'll update the error message at some point in the next couple of weeks.

[Maaitrayo]

get This tensor is not representable in Java, it's too big, while inference fasterrcnn_resnet50_fpn model in onnx.

export_onnx.py (Python Source):

    model = torchvision.models.detection.fasterrcnn_resnet50_fpn(pretrained=True)
    x = [torch.rand(3, 1970, 1080)]
    torch.onnx.export(model, x, "faster_rcnn.onnx", opset_version = 11)

build.gradle (Gradle Source):

    dependencies {
          implementation group: 'com.microsoft.onnxruntime', name: 'onnxruntime', version: '1.7.0'
    }

Pridiction.java (Java Source):

47| try(OrtSession session = env.createSession(modelPath.toFile().getAbsolutePath(), 
48|                    new OrtSession.SessionOptions())) {
49|
50|     OnnxTensor t1 = OnnxTensor.createTensor(env, sourceData, dimensions);
51|     OrtSession.Result result = session.run(Collections.singletonMap("image.1", t1));
52|
53|     for (int i = 0; i < result.size(); i++) {
54|         System.out.println(result.get(i).getValue());
55|     }
56|
57|     result.close();
58|     t1.close();
59| }

Exception while inference:

ai.onnxruntime.OrtException: This tensor is not representable in Java, it's too big - shape = [0, 4]
	at ai.onnxruntime.TensorInfo.makeCarrier(TensorInfo.java:171)
	at ai.onnxruntime.OnnxTensor.getValue(OnnxTensor.java:99)
	at onnx.predict.Pridiction.main(Pridiction.java:54)

Refered sites:

       1. https://www.onnxruntime.ai/docs/reference/api/java-api.html
       2. https://pytorch.org/vision/stable/models.html#faster-r-cnn

making the shape [1,4] might work

[hiskuDN]

You ended 12 hours of search, thanks @anandvsr, setting a dynamic input fixes the issue