[Identity] Skipping device code tests on playback (live tests will help us release) #15047
Conversation
ghost
added
the
sadasant
changed the title
sadasant
force-pushed
the
identity/device-code-test
branch
from
8cd694c to
215bcb9
Compare
|
@sadasant Can you please update the PR description with a note on why the tests are being skipped? |
|
@ramya-rao-a updated! |
|
I believe I still saw a timeout issue even on this PR ... so I'll emphasize that I'm not certain this will fix the pipeline issue. |
|
Was the 20 time run successful after the tests are skipped? |
|
@HarshaNalluru I'll push that to see |
|
Running these tests 20 times caused an error I wasn't expecting, only in one environment: I'll investigate. |
|
That last error seems like a bad test. I'll push a slight improvement. We were comparing two dates and assuming they may be slightly different, but sometimes they may get different a bit above our considerations. |
|
We just had an issue installing dependencies in one of the pipelines. I don't know any other workaround than re-running the pipelines, so I'll wait until the pipeline finishes so I can re-run the item that failed. |
|
@HarshaNalluru I won't re-start the CI job because it timed out here: https://dev.azure.com/azure-sdk/public/_build/results?buildId=869291&view=logs&j=889ff074-62b8-55e4-5d7f-907488e92b21&t=0da6ba07-cc57-557e-3355-438c1b2e5790&l=428 |
|
@HarshaNalluru I'll make a separate PR skipping Identity tests entirely, and I'll try to reproduce. |
|
Since we were able to reproduce this without these tests, I'm testing some more aggressive skipping here: #15092 |
Remove files from package which are not part of management plane SDK - update readme.md (Azure#15047) * Update readme.md * Update readme.md
3 participants
This PR is now about skipping device code tests on playback. Live tests are staying for now, since we want to use them to release. I'll be working on a fix on MSAL's repo.
We're aware of a CI issue in which something gets stuck during tests, meaning that the Node process never closes, which timeouts the core pipeline. We are not 100% sure on what's the underlying issue, because we haven't been able to reproduce the problem locally.
However, we're aware of one logical case in which this may happen. There's an issue with MSAL in which the polling on the device code credential may end up triggering multiple polling requests and resolving them inappropriately. For more details, you can go here: AzureAD/microsoft-authentication-library-for-js#3476
In this PR, I made CI run these (playback) tests 20 times and the probability of failure seemed to increase.
Given that the MSAL bug is real, we should try skipping these tests to see if the core pipeline stops having the issue.
In the mean time, I've made a draft PR to MSAL with an approach that could solve the problem: AzureAD/microsoft-authentication-library-for-js#3553
There's one other small fix for a test. One ManagedIdentityCredential test was comparing times incorrectly.