img decoding="async" with blazor

[wrharper-AASP]

When doing async with img src for a base64 string I get a deadlock. I have to use configureawait(false) on all tasks and according to sourceforge and sites like it, everyone is doing this to get around it.

I have recently learned about (decoding="async") inside of the img html object. Is it possible to use this in some way so that you can truly get images with async? this would be a huge performance increase for the project I am doing.

I use a custom REST API I have created to get the images and they come in as base64 strings.

[wrharper-AASP]

I am a bit confused about who supports what. This project says asp.net core does blazor, when going to asp.net core, it links to this. There appears to be a loop on who does what? If this is the wrong place, let me know

[javiercn]

@wrharper-AASP thanks for contacting us.

I am a bit confused about who supports what. This project says asp.net core does Blazor, when going to asp.net core, it links to this. There appears to be a loop on who does what? If this is the wrong place, let me know.

This repo is for Blazor, which is part of ASP.NET Core. Hope that helps clarify things.

When doing async with img src for a base64 string I get a deadlock. I have to use configureawait(false) on all tasks and according to sourceforge and sites like it, everyone is doing this to get around it.

I have recently learned about (decoding="async") inside of the img html object. Is it possible to use this in some way so that you can truly get images with async? this would be a huge performance increase for the project I am doing.

I use a custom REST API I have created to get the images and they come in as base64 strings.

I imagine you are doing something like <img src="{{base64Url}}" />?

First of all, we recommend against that. You are allocating very large strings that the system needs to hold on to, and that is going to perform badly.

If you already have data on an API, we suggest you avoid base64 URL encoding the image altogether, serve the bits raw from the endpoint and reference the endpoint in the link tag.

[wrharper-AASP]

@wrharper-AASP thanks for contacting us.

I am a bit confused about who supports what. This project says asp.net core does Blazor, when going to asp.net core, it links to this. There appears to be a loop on who does what? If this is the wrong place, let me know.

This repo is for Blazor, which is part of ASP.NET Core. Hope that helps clarify things.

When doing async with img src for a base64 string I get a deadlock. I have to use configureawait(false) on all tasks and according to sourceforge and sites like it, everyone is doing this to get around it.
I have recently learned about (decoding="async") inside of the img html object. Is it possible to use this in some way so that you can truly get images with async? this would be a huge performance increase for the project I am doing.
I use a custom REST API I have created to get the images and they come in as base64 strings.

I imagine you are doing something like <img src="{{base64Url}}" />?

First of all, we recommend against that. You are allocating very large strings that the system needs to hold on to, and that is going to perform badly.

If you already have data on an API, we suggest you avoid base64 URL encoding the image altogether, serve the bits raw from the endpoint and reference the endpoint in the link tag.

I actually can't in my case. I can't really explain why without exposing too much of my goals, but it comes down to the useragent. The images have to be a specific useragent to be viewed by the URL so that is why they are converted to base64.

[wrharper-AASP]

The other Idea I have had has been creating a List, Stack, or Queue cache, but that would increase memory usage tremendously if tons of people use the site. It can't be stored on the client for compliance reasons either.

[javiercn]

The other Idea I have had has been creating a List, Stack, or Queue cache, but that would increase memory usage tremendously if tons of people use the site. It can't be stored on the client for compliance reasons either.

The images are already "stored" in the client if they are being rendered in a browser.

The best you can do is something like this which is what we recommend doing.

I actually can't in my case. I can't really explain why without exposing too much of my goals, but it comes down to the useragent. The images have to be a specific useragent to be viewed by the URL so that is why they are converted to base64.

Not sure what the reason is for this, but I would say that relying on the useragent for any type of access restriction is not correct, as it's really easy for any app to change it/fake it. You'd be better of using some sort of authorization to prevent unauthorized access to the resource.

You can prevent the browser from caching anything by setting the appropriate response headers.

[wrharper-AASP]

The images are already "stored" in the client if they are being rendered in a browser.

The best you can do is something like this which is what we recommend doing.

Sort of, but it isn't the same as a directly stored .jpg. It's just a string in the code. It's not as easy for those common guys out there and I have found no other working alternative yet.

Not sure what the reason is for this, but I would say that relying on the useragent for any type of access restriction is not correct, as it's really easy for any app to change it/fake it. You'd be better of using some sort of authorization to prevent unauthorized access to the resource.

You can prevent the browser from caching anything by setting the appropriate response headers.

Oh, it does Auth as well. I don't know why they designed it that way and it isn't something I can change unfortunately since it's not on our end. I couldn't agree more though.

[javiercn]

Sort of, but it isn't the same as a directly stored .jpg. It's just a string in the code. It's not as easy for those common guys out there and I have found no other working alternative yet.

It's not clear what you mean by this. If it gets downloaded into "a browser" it's trivial for someone to save the image. If the image is displayed on a client machine (browser allegedly) the bits are already there.

As to "holding" on to the image in memory as a string, it's easy to do the math, and if each image you have is 5 mb, with 100 you are getting 500mb. If this is Blazor Server, then each user that does that, takes 500mb of your server memory.

It's not much better in webassembly either, other than the fact that the memory is on the client machine.

The general issue here is that allocating large blobs of string data is going to wreck the performance of your app, as the GC automatically puts the strings in the LOH heap, which means they wont be garbage collected unless the system is pretty much running out of memory.

Not to mention that if you use those strings directly within Blazor, we must hold a reference to them continuously up until you decide to stop rendering them.

If this is the approach you are taking, there's nothing we can do about it. This is going to perform badly pretty much in every framework, that's why blob urls exist for.

[wrharper-AASP]

ok, the last thing I can think of is adding the useragent somehow to all clients. Is that possible?

Also, yes, the user can always just use something like the snipping tool, but someone who has access to another person's drive would have no way to get the image?

[wrharper-AASP]

Based on your solution, it looks like it tries to use a button for the async but this needs to be done automatically in a for loop:

if (GetMedia(i).Result) <--- works
                            {
                                <img id=@SetId(i) />
                                @SetImageAsync(i).Result <--- won't work even with configure false on everything.
                            }

I set it the following way:

    async Task<Stream> GetImageStreamAsync(int i)
    {
        return await Http.GetStreamAsync(Info.MessageInfo[i].mediaid).ConfigureAwait(false);
    }

    async Task<string> SetImageAsync(int i)
    {
        var imageStream = await GetImageStreamAsync(i).ConfigureAwait(false);
        var dotnetImageStream = new DotNetStreamReference(imageStream);
        await JsRuntime.InvokeVoidAsync("setImage", SelectedI.ToString() + "_" + i.ToString(), dotnetImageStream).ConfigureAwait(false);
        return "";
    }

The return empty string is just a trick so it will process the info even though it is a asynced task.
I added the user agent into the program.cs so that it would be part of the standard injection:
builder.Services.AddHttpClient("WebClient", client => { client.DefaultRequestHeaders.UserAgent.ParseAdd("CUSTOMAGENT"); client.Timeout = TimeSpan.FromSeconds(600); });

[javiercn]

@wrharper-AASP thanks for the additional details.

Rendering in Blazor is synchronous, it does not work because the result from your task is being ignored (It's really hard to tell what you are exactly doing without the concrete snippets).

You need to load any data you need within OnInitializedAsync. It will render twice, the first time before any async work happens, and a second time after it has completed, and at that point you can use the data to render

foreach(var image in Images)
{
   <img src="" ... />
}

override OnInitializedAsync()
{
    Images = await LoadImages();
}

[wrharper-AASP]

right, that's what the configureawait(false) is for and using .Result. it basically makes it synchronous. There is a timer where new images are checked, and a refresh will need to happen if new images need to be added. It can't just be ran one time and that's it. It needs to be ran on every timer check when something new is detected. So OnInitializedAsync won't work and this is in a for loop.

[wrharper-AASP]

I have been using .Result to bypass these problems for many things and it works great, but for some reason it deadlocks with the solution you mentioned.

[javiercn]

@wrharper-AASP You can't do that within Blazor.

Blazor uses a synchronizationcontext and by doing .ConfigureAwait your callbacks won't run within it, which is sure to cause problems. ConfigureAwait is for libraries, not for app code.

[wrharper-AASP]

@wrharper-AASP You can't do that within Blazor.

Blazor uses a synchronizationcontext and by doing .ConfigureAwait your callbacks won't run within it, which is sure to cause problems. ConfigureAwait is for libraries, not for app code.

https://stackoverflow.com/questions/67516887/blazor-server-side-async-deadlock

[javiercn]

@wrharper-AASP that question/answer is wrong.

Not sure what you are trying to imply with that stackoverflow question/response, but the only comment from Stephen Cleary is correct. Do not block on async code.

The way you are trying to approach a solution is in direct conflict with how the framework works.

Rendering is synchronous, and if you want to do async work, you need to use a lifecycle method and do it there. Blocking the rendering threads will cause deadlocks at worst and thread starvation at best.

[wrharper-AASP]

@wrharper-AASP that question/answer is wrong.

Not sure what you are trying to imply with that stackoverflow question/response, but the only comment from Stephen Cleary is correct. Do not block on async code.

The way you are trying to approach a solution is in direct conflict with how the framework works.

Rendering is synchronous, and if you want to do async work, you need to use a lifecycle method and do it there. Blocking the rendering threads will cause deadlocks at worst and thread starvation at best.

There have been tons of stackoverflows that say the same thing, that was just one example. How would you do a lifecycle to make it possible then?
Keep in mind, this needs to be able to be updated during new changes being detected. So, it isn't a (one and done) thing.

[wrharper-AASP]

Ok I did the proper lifecycle, but it will not work due to the useragent issue. The blob just shows up as broken images.

I do OnParam set to set the values (before the render) for the HTML and then OnAfterRender without first render so it keeps happening for setting the ID (after the render).

[javiercn]

@wrharper-AASP there is a complete sample in the link I provided https://learn.microsoft.com/en-us/aspnet/core/blazor/images?view=aspnetcore-7.0#stream-image-data

The only thing you need to do is define the setImage function within your app, so that it's reachable via JS interop

[wrharper-AASP]

It isn't valid because you don't use an async button push. It should be happening on load and update on changes. No buttons.

[wrharper-AASP]

also, i have it working by doing what i mentioned, but the images are still broken:

[javiercn]

@wrharper-AASP change SetImageAsync and instead override OnInitializedAsync() to load the data and then do the JS interop call after the data has arrived.

You can use a boolean flag to know when the data is ready, and have a check on the render to avoid rendering anything until all the data is there

[javiercn]

That src attribute looks wrong. How did you create that

[wrharper-AASP]

With the solution you gave, it creates that.

[javiercn]

@wrharper-AASP what do the browser console logs say?

[wrharper-AASP]

The HTML is just <img id=@images[i] /> which is basically the same thing as the example setting an id. The src is generated from await JsRuntime.InvokeVoidAsync("setImage", SelectedI.ToString() + "_" + i.ToString(), dotnetImageStream); everything is like the example shows just with my values instead.

[wrharper-AASP]

@wrharper-AASP what do the browser console logs say?

nothing because there is actually no errors.

[wrharper-AASP]

I have tried this 2 ways and both results are the same thing.

1. converting base64 to a memory stream byte array
2. tried to mimic the custom user agent with the httpclient the example shows.

[wrharper-AASP]

I think this comes down to an auth issue. Because I cannot get that image without also doing Auth, I can't do Auth without exposing a key that the client side should never see. This is another reason why I used base64.

[wrharper-AASP]

Again, with OnInitialized, this cannot be used because it only runs once. These values and this component will change constantly, it cannot be used due to update changes.

[wrharper-AASP]

What really needs to exist is a proper before and after. That doesn't seem to be possible in blazor. What I used was OnParametersSet and then OnAfterRender

[javiercn]

@wrharper-AASP here is a working sample https://github.com/javiercn/ImageBlobSample

[wrharper-AASP]

You can't use the direct https:// without exposing the key. That was the whole point of base64 to begin with. Too much security is exposed. Your examples involves commonly exposed public images. That isn't the case...

[javiercn]

@wrharper-AASP The code is running on the server, where you get to do whatever you want.

You can replace the code that I used to get the image with your own code that does whatever it needs.

[wrharper-AASP]

No, not without the client seeing this:

private async Task<DotNetStreamReference> GetImage(int i)
    {
        var client = new HttpClient();
client.DefaultRequestHeaders.Add("Authorization", SHOULDNOTBESEEN);
        var stream = await client.GetStreamAsync($"https://dummyimage.com/450x2{i}0/f00/fff");
        return new DotNetStreamReference(stream);
    }

[javiercn]

@wrharper-AASP the c# code in Blazor Server runs on the server. Only the JS code runs on the browser.

[javiercn]

https://learn.microsoft.com/en-us/aspnet/core/blazor/hosting-models?view=aspnetcore-7.0

[wrharper-AASP]

it is more complicated than that. I can't explain. I cannot hide that key if that is on the server. We are working backwards further and further; this will not be a solution.

[javiercn]

@wrharper-AASP there doesn't seem to be a framework issue here.

It's not clear to us what problem you have, and we do not have enough details to offer a meaningful suggestion beyond the sample we provided.

Unless you want to provide a publicly available minimal repro project (not your codebase, something equivalent that replicates the problem) we can't be of further assistance.

While we try our best to help everyone that comes with questions, this tracker is dedicated to product issues and it seems you have an architectural/layering problem to solve. We recommend you post a question on Stackoverflow with the aspnetcore-blazor tag to see if other members on the community can step up to give you guidance.

[wrharper-AASP]

I have had to create a custom REST API for many reasons, and this is one of them. The images cannot be stored anywhere else due to compliance, and they cannot be seen without a specific key and that key would need to be dynamically distributed if it's on the server. It is more complicated than what is being shown. It is clear that it is just not possible at this point and base64 is the only option.

[wrharper-AASP]

I have found a way to make the reference in a way that works by converting the base64 and using your solution. Will this help performance?
DotNetStreamReference dotnetstream = new DotNetStreamReference(new MemoryStream(Convert.FromBase64String(base64string));

[wrharper-AASP]

Thank you, it does look like this is actually resolved with the resolution you provided now. It was just a matter of going about it a different way for the stream.

[likquietly]

here is a working sample https://github.com/javiercn/ImageBlobSample

@javiercn, great sample, but how to see the image arriving progressively?
Do you recommend stream image using JS interop instead of placing the image src at URL(img src="api/image/@imageNo") with api method like this at Blazor Server App:

[HttpGet("images/{imageNo:int}")]
public async Task<IActionResult> GetImageAsync(int imageNo)
{
var path = $"C:\\image_{imageNo}";
var data = await System.IO.File.ReadAllBytesAsync(path);

return File(data, "image/jpeg");
}

[wrharper-AASP]

I recommend doing ID instead of SRC like the original solution shows with a JS invoke:
https://learn.microsoft.com/en-us/aspnet/core/blazor/hosting-models?view=aspnetcore-7.0
His concept/idea is right though.