-
Notifications
You must be signed in to change notification settings - Fork 1.3k
Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
Improve route status reconcile integration tests (#11456)
The route status reconcile integration tests are flaky and often spuriously fail in CI. The assertion which fails is that the `Accepted` status should be true after reconciliation but it is observed to be false. I have not been able to reproduce this failure locally so it is hard to determine what causes these failures. However, I noticed that these tests utilize a loop without any backoff and repeatedly request a route from the kubernetes API until the status modified timestamp is updated. This hot loop results in a very large number of kubernetes API requests very quickly. It's not clear if these excessive requests could cause the kubernetes API to become overloaded and result in the flaky failures, but it seems like something that's worth fixing regardless. We replace the hot loop with a usage of `await_condition` so that we take advantage of a watch on the route instead of repeatedly polling it. This results is far fewer API calls. We also add some logging of the status each time we retrieve it. This will be helpful in understanding why these tests fail, if they fail again in the future. Signed-off-by: Alex Leong <[email protected]>
- Loading branch information
Showing
3 changed files
with
113 additions
and
123 deletions.
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Oops, something went wrong.