Optimize e2e tests by parallelizing work in EnsureCleanState #20968
Assignees
Labels
Comments
andrii-korotkov-verkada
added
enhancement
andrii-korotkov-verkada
added a commit
to andrii-korotkov-verkada/argo-cd
that referenced
this issue
Nov 29, 2024
…rmation Closes argoproj#20968 Closes argoproj#20967 A number of cleanup and create steps can be done in parallel, reducing the total runtime. Gather timing information as well and log it. Signed-off-by: Andrii Korotkov <[email protected]>
14 tasks
adriananeci
pushed a commit
to adriananeci/argo-cd
that referenced
this issue
Dec 4, 2024
…rmation (argoproj#20998) * chore: Parallelize EnsureCleanState for e2e tests, adding timing information Closes argoproj#20968 Closes argoproj#20967 A number of cleanup and create steps can be done in parallel, reducing the total runtime. Gather timing information as well and log it. Signed-off-by: Andrii Korotkov <[email protected]> * Add underscores to function map keys Signed-off-by: Andrii Korotkov <[email protected]> * Have separate log statements for timing fields, since everything doesn't fit in Github logs and gets truncated Signed-off-by: Andrii Korotkov <[email protected]> * Re-run tests Signed-off-by: Andrii Korotkov <[email protected]> * Fix duplicate removal attempts, cleanup things in namespaces before removing namespaces Signed-off-by: Andrii Korotkov <[email protected]> * Combine config map cleanups, since they seem to have some kind of locking based on timings Signed-off-by: Andrii Korotkov <[email protected]> * Re-order some cleanups to not wait if something is already covered by no wait commands Signed-off-by: Andrii Korotkov <[email protected]> * Move rm temp dir to be together with re-creation, since otherwise repo server watcher seems to fail Signed-off-by: Andrii Korotkov <[email protected]> * Only update config maps and login as admin if there were any changes Signed-off-by: Andrii Korotkov <[email protected]> * Also check for user being logged in Signed-off-by: Andrii Korotkov <[email protected]> * Improve config maps equivalency check Signed-off-by: Andrii Korotkov <[email protected]> * Remove timing information Signed-off-by: Andrii Korotkov <[email protected]> * Improve loops with sleep Signed-off-by: Andrii Korotkov <[email protected]> * Properly propagate and handle errors for parallel functions Signed-off-by: Andrii Korotkov <[email protected]> * Add missing CheckError Signed-off-by: Andrii Korotkov <[email protected]> * Fix import Signed-off-by: Andrii Korotkov <[email protected]> * Fix imports linter error (try 2) Signed-off-by: Andrii Korotkov <[email protected]> * Parallelize EnsureCleanState for application sets Signed-off-by: Andrii Korotkov <[email protected]> * Fix errors.IsNotFound Signed-off-by: Andrii Korotkov <[email protected]> * Use errgroup to run functions in parallel Signed-off-by: Andrii Korotkov <[email protected]> * Remove name keys for functions as unused Signed-off-by: Andrii Korotkov <[email protected]> * Use apierr directly Signed-off-by: Andrii Korotkov <[email protected]> --------- Signed-off-by: Andrii Korotkov <[email protected]> Signed-off-by: Adrian Aneci <[email protected]>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Summary
There can be steps to be executed in parallel (a naive assumption could be that nearly all steps can be) to save time.
Motivation
EnsureCleanState is ~10-30% of each e2e test run time, so optimizing it can substantially reduce the overall e2e tests run time.
Proposal
Figure out which blocks are safe to execute in parallel and do so.
The text was updated successfully, but these errors were encountered: