Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Race condition in ARM template #539

Closed
glahaye opened this issue Oct 23, 2023 · 5 comments
Closed

Race condition in ARM template #539

glahaye opened this issue Oct 23, 2023 · 5 comments
Assignees
Labels
bug Something isn't working deployment Issues related to deploying Chat-Copilot

Comments

@glahaye
Copy link
Collaborator

glahaye commented Oct 23, 2023

Describe the bug
When deploying the Azure resources from main.bicep / main.json, we hit errors like the following:

ERROR TYPE
Deployment was interrupted and the process running it was terminated unexpectedly, if an ARM template is used, the likely cause is a race condition. To solve the race condition, make all resources that could potentially restart the site (app settings or site config changes for example) have a dependency on MSDeploy resource. For more info please read the information here: http://go.microsoft.com/fwlink/?LinkId=808141 (Code: Failed)

To Reproduce
Deploy CC resources from scratch to Azure using either deploy-azure script or Deploy to Azure button.

Expected behavior
Error-free deployment of resources to Azure

Related: #488 #498

@glahaye glahaye self-assigned this Oct 23, 2023
@glahaye glahaye added bug Something isn't working deployment Issues related to deploying Chat-Copilot labels Oct 23, 2023
@glahaye
Copy link
Collaborator Author

glahaye commented Oct 23, 2023

Usually, this type of error is due to more than one resource being deployed on top of a "Microsoft.Web/sites" server.

Each resource deployed to the server (MSDeploy extension, Application Insights extension, site configuration) causes it to restart. If the server restarts while another of these resources is being deployed, this latter deployment will fail because it will be interupted by the restart.

The usual solution to this problem is to chain the resource deployments to the server serially using "dependsOn" clauses instead of letting them happening in parallel,

Despite having chained all resource deployments to the server in the CC bicep / ARM template, we still get this error.

The problem seems particularly acute (or even exclusive?) to the appServiceMemoryPipeline server.

@crickman crickman removed their assignment Oct 24, 2023
@crickman
Copy link
Contributor

@alliscode - Can you review where this is at a determine next steps?

@crickman
Copy link
Contributor

Apparently there is a response to Gil's stack-overflow query: https://stackoverflow.microsoft.com/questions/375089/375405

@jopapeador18
Copy link

I'm sorry, what would be the answer then ? Should we redeploy? I checked the template but it had the sentence dependon.

@glahaye
Copy link
Collaborator Author

glahaye commented Nov 2, 2023

@jopapeador18 You can always re-run a deployment right it fails (if it does).

At any rate, #564 we should be good and not encounter that problem anymore.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working deployment Issues related to deploying Chat-Copilot
Projects
No open projects
Development

No branches or pull requests

4 participants