How to use this with phi3-vision

[jarroddavis68]

I was checking out this:
https://ai.azure.com/explore/models/Phi-3-vision-128k-instruct/version/2/registry/azureml?tid=53cbd9ee-7885-4d5b-9b24-adb57bcef189

and saw this:

{
  "input_data": {
    "input_string": [
      {
        "role": "user",
        "content": [
          {
            "type": "image_url",
            "image_url": {
              "url": "https://www.ilankelman.org/stopsigns/australia.jpg"
            }
          },
          {
            "type": "text",
            "text": "What is shown in this image? Be extremely detailed and specific."
          }
        ]
      }
    ],
    "parameters": { "temperature": 0.7, "max_new_tokens": 2048 }
  }
}

I tried to send this prompt, but I get this error:
Error: D:\a\_work\1\s\onnxruntime\core\providers\dml\DmlExecutionProvider\src\DmlCommittedResourceAllocator.cpp(22)\onnxruntime.dll!00000206807421E1: (caller: 000002068072423C) Exception(1) tid(1ee0) 8007000E Not enough memory resources are available to complete this operation.

Does is support this prompt format? It does work and have been working based on the phi3v.cpp C api example. Was wondering if I am also able to use the above. There error seems to be releated to not enough memory, yet `phi3v.cpp' way of doing works just fine on my machine. (RTX-3060 12GB VRAM, 32GB RAM).

[elephantpanda]

Yes I also get this error. I believe they said it was a memory bug for using DirectML and phi3 vision. Hopefully they will fix it one day. Seems to be a problem with long prompts or including images. Only seems to work without an image which is not much use for a vision model!