WEBSITE_RUN_FROM_PACKAGE conflicting with WEBSITE_LOCAL_CACHE_OPTION

[flagbug]

So this actually caused a production outage for us yesterday when we deployed for the first time with "Run From Package" instead of "Web Deploy".

According to the App Service documentation, it's recommended that one uses WEBSITE_LOCAL_CACHE_OPTION=Always for highly-available environments since it avoids the single-point-of-failure that is the shard App Service storage. We've been using this setting for years without any problems.

Immediately after our deployment yesterday, which was our first deployment with "Run From Package" instead if using "Web Deploy", we noticed that our application suddenly responding with 404 Not Found; The resource you are looking for has been removed, had its name changed, or is temporarily unavailable. on ALL endpoints. We tried restarting the application a few times and it was up for a ~2 minutes but then it started responding with the 404 Not Found responses again. These responses seem to have origined from the IIS proxy, since our application itself didn't receive ANY requests at this time.

Repro steps.

• Create an Azure App Service with the Run From Package option (WEBSITE_RUN_FROM_PACKAGE=1)
• You'll see that everything works fine and is responding normally
• Enable WEBSITE_LOCAL_CACHE_OPTION=Always
• Deploy again with Run From Package

At this point you'll see that the whole app has gone down and responds with 404 Not Found; The resource you are looking for has been removed, had its name changed, or is temporarily unavailable., except for the root endpoints, which responds with 403 Forbidden - You do not have permission to view this directory or page.

If these two options don't work together, it should be at least documented.

[suwatch]

Thanks for the feedback. We added to FAG of Azure/app-service-announcements#84.

[mattwoberts]

Hi

I understand that they aren't compatible - after experiencing this problem myself. However if you're deploying from azure pipelines, the deployment task uses zip deploy (which in turn seems to rely on, and indeed sets WEBSITE_RUN_FROM_PACKAGE as part of the deployment).

Since I've been advised multiple times to use WEBSITE_LOCAL_CACHE_OPTION, in order to improve reliability and reduce down time, I'm pretty confused what I am supposed to do now. Should I stop using local cache, has azure app service stability moved on to a point that this is no longer necessary? Or should I look for a way to deploy my artifact from azure pipelines that doesn't use zip deploy, if indeed that is possible?

Many thanks

Matt

[Mike-E-angelo]

Since I've been advised multiple times to use WEBSITE_LOCAL_CACHE_OPTION, in order to improve reliability and reduce down time, I'm pretty confused what I am supposed to do now

Should be the "ultimate guide on how to crash your server":
https://azure.github.io/AppService/2020/05/15/Robust-Apps-for-the-cloud.html#enable-local-cache

[rikbosch]

This broke our production instances as well, it might be good to clarify this in the documentation.

[thiagohdaqw]

It's 2024 and I just had this problem 😂