Skip to content

chore: unskip mssql tests#36866

Merged
sneha122 merged 5 commits intoreleasefrom
chore/unskip-mssql-test
Oct 17, 2024
Merged

chore: unskip mssql tests#36866
sneha122 merged 5 commits intoreleasefrom
chore/unskip-mssql-test

Conversation

@sneha122
Copy link
Contributor

@sneha122 sneha122 commented Oct 14, 2024

Description

This PR fixes the flaky mssql Junit test case by replacing the test containers docker image to mssql server 2022 latest.

Root cause of flakiness

MSSQL Junit test cases like mssqlplugintest.java and mssqlGetDBSchemaTest.java recently started becoming flaky in the CI. Sometimes they would pass, sometimes they would throw the error org.testcontainers.containers.ContainerLaunchException: Container startup failed for image mcr.microsoft.com/azure-sql-edge:1.0.3. This was happening because MSSQL test cases create a docker container using Azure SQL Edge 1.0.3 image. Upon further researching found two relevant links as to why it has stopped working:

  1. https://forums.docker.com/t/sql-server-docker-container-fails-on-start-up-when-run-in-a-vm-with-ubuntu-24-04/142093
  2. 4.33.0 breaks containers like MSSQL docker/for-mac#7368

Although second link seems to be specific for Mac where as we use Ubuntu when running tests in CI, this comment and docker forum link above states that problem is there for Ubuntu as well. Two possible workaround suggested in above threads were:

  1. Downgrade docker desktop to 4.32
  2. Instead of Azure SQL Edge use MSSQL server 2022 latest image

This PR uses the second workaround and updates docker container image from Azure SQL Edge 1.0.3 to MSSQL server 2022 latest image.

I have triggered the server-unit-tests workflow 12 times and all times it passed, showing no flakiness, hence we should be good to go ahead with the fix.

Equivalent EE PR: https://github.com/appsmithorg/appsmith-ee/pull/5366

Fixes #36774
or
Fixes Issue URL

Warning

If no issue exists, please create an issue first, and check with the maintainers if the issue is valid.

Automation

/ok-to-test tags="@tag.Sanity"

🔍 Cypress test results

Tip

🟢 🟢 🟢 All cypress tests have passed! 🎉 🎉 🎉
Workflow run: https://github.com/appsmithorg/appsmith/actions/runs/11380762425
Commit: 4f95888
Cypress dashboard.
Tags: @tag.Sanity
Spec:


Thu, 17 Oct 2024 08:33:37 UTC

Communication

Should the DevRel and Marketing teams inform users about this change?

  • Yes
  • No

@github-actions github-actions bot added the skip-changelog Adding this label to a PR prevents it from being listed in the changelog label Oct 14, 2024
@sneha122
Copy link
Contributor Author

/ci-test-limit

@github-actions
Copy link

@github-actions
Copy link

Workflow run: https://github.com/appsmithorg/appsmith/actions/runs/11343697359.
Cypress dashboard url: Click here!
All cypress tests have passed 🎉🎉🎉

@github-actions
Copy link

Failed server tests

  • com.appsmith.server.services.ce.ActionServiceCE_Test#invalidCreateActionNullName

@github-actions
Copy link

Failed server tests

  • com.appsmith.server.git.ServerSchemaMigrationEnforcerTest#saveGitRepo_ImportAndThenExport_diffOccurs

@github-actions github-actions bot added Backend This marks the issue or pull request to reference server code Integrations Product Issues related to a specific integration MsSQL Issues related to MsSQL plugin Query & JS Pod Issues related to the query & JS Pod Redis Issues related to Redis Task A simple Todo labels Oct 16, 2024
@sneha122 sneha122 added the ok-to-test Required label for CI label Oct 16, 2024
@sneha122 sneha122 marked this pull request as ready for review October 17, 2024 06:13
@sneha122 sneha122 merged commit 765b1b3 into release Oct 17, 2024
@sneha122 sneha122 deleted the chore/unskip-mssql-test branch October 17, 2024 08:36
github-actions bot pushed a commit to Zeral-Zhang/appsmith that referenced this pull request Nov 20, 2024
## Description
This PR fixes the flaky mssql Junit test case by replacing the test
containers docker image to mssql server 2022 latest.

### Root cause of flakiness
MSSQL Junit test cases like `mssqlplugintest.java` and
`mssqlGetDBSchemaTest.java` recently started becoming flaky in the CI.
Sometimes they would pass, sometimes they would throw the error
`org.testcontainers.containers.ContainerLaunchException: Container
startup failed for image mcr.microsoft.com/azure-sql-edge:1.0.3`. This
was happening because MSSQL test cases create a docker container using
Azure SQL Edge 1.0.3 image. Upon further researching found two relevant
links as to why it has stopped working:
1.
https://forums.docker.com/t/sql-server-docker-container-fails-on-start-up-when-run-in-a-vm-with-ubuntu-24-04/142093
2. docker/for-mac#7368

Although second link seems to be specific for Mac where as we use Ubuntu
when running tests in CI, this
[comment](docker/for-mac#7368 (comment))
and docker forum link above states that problem is there for Ubuntu as
well. Two possible workaround suggested in above threads were:
1. Downgrade docker desktop to 4.32 
2. Instead of Azure SQL Edge use MSSQL server 2022 latest image

This PR uses the second workaround and updates docker container image
from Azure SQL Edge 1.0.3 to MSSQL server 2022 latest image.

I have triggered the server-unit-tests workflow 12 times and all times
it passed, showing no flakiness, hence we should be good to go ahead
with the fix.

Equivalent EE PR: https://github.com/appsmithorg/appsmith-ee/pull/5366

Fixes appsmithorg#36774 
_or_  
Fixes `Issue URL`
> [!WARNING]  
> _If no issue exists, please create an issue first, and check with the
maintainers if the issue is valid._

## Automation

/ok-to-test tags="@tag.Sanity"

### 🔍 Cypress test results
<!-- This is an auto-generated comment: Cypress test results  -->
> [!TIP]
> 🟢 🟢 🟢 All cypress tests have passed! 🎉 🎉 🎉
> Workflow run:
<https://github.com/appsmithorg/appsmith/actions/runs/11380762425>
> Commit: 4f95888
> <a
href="https://internal.appsmith.com/app/cypress-dashboard/rundetails-65890b3c81d7400d08fa9ee5?branch=master&workflowId=11380762425&attempt=1"
target="_blank">Cypress dashboard</a>.
> Tags: `@tag.Sanity`
> Spec:
> <hr>Thu, 17 Oct 2024 08:33:37 UTC
<!-- end of auto-generated comment: Cypress test results  -->


## Communication
Should the DevRel and Marketing teams inform users about this change?
- [ ] Yes
- [x] No

---------

Co-authored-by: “sneha122” <“sneha@appsmith.com”>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Backend This marks the issue or pull request to reference server code Integrations Product Issues related to a specific integration MsSQL Issues related to MsSQL plugin ok-to-test Required label for CI Query & JS Pod Issues related to the query & JS Pod Redis Issues related to Redis skip-changelog Adding this label to a PR prevents it from being listed in the changelog Task A simple Todo

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[Task]: Fix flaky server tests of MSSQL and Redis

2 participants