Split nav2_system_tests into separate CI jobs#3554
Conversation
|
@SteveMacenski , would you happen see why this is listing no packages, rather than all excluding BUILD_PACKAGES=$(
colcon list \
--names-only \
--packages-select-regex * \
--packages-skip-regex nav2_system_tests \
--packages-above \
$BUILD_UNFINISHED \
$BUILD_FAILED \
$BUILD_INVALID \
| xargs) |
|
One caveat here is that we can either run the OR We can run the jobs linearly from This is represented with before and after the commit: 95ff181 . Perhaps we could add some more conditional glue for reviewers to override this fail fast behavior, but it might take a few hacks or more refactoring: |
|
We used to have a setup like Where these were separate pipelines. At that time, we did store the test results of both because I'd look at them both before merging PRs. Could we do something like? That would make it so that both sets of tests can run without if builds don't fail (which they never should). You mentioning caching problems, I don't know if things have changed since those older days that makes an issue now. I'm not concerned at all about caching the workspaces for system tests build since that always gets busted and runs no matter what (in case that makes things easier) - we only need the workspace cache for the source builds to speed things up. But I don't think that relates to the test results caching problem you mentioned Is it not possible to have 2 results caches (or in different directories to separate?), one for each job? We don't need the cache for the system-tests-related workspaces, just the output files to see what tests failed and how. I'm not sure if any of that is helpful to your question, maybe I'm just asking the same questions you asked yourself |
by skipping it in release build and test jobs
to linearly resume cache
in order to skip them from system_test
to skip unrelated tests
by requiring release_test for system_build to ensure linear cache store and restore access
to simplify catching of tests given release_test was never timing out before
to resolve package selection issues
|
I opted to keep the tests consolidated to one jobs and make it require the new |
* Refactor nav2_system_tests into Separate jobs by skipping it in release build and test jobs * Require release_test for system_build job to linearly resume cache * Keep cache test results from release_test in order to skip them from system_test * Alias matrix for system_build requires * Refactor to use package select as well to skip unrelated tests * Use yaml anchors to keep DRY * Rename regex parameters * Retain cache for build_and_test workflows by requiring release_test for system_build to ensure linear cache store and restore access * Bust cache to test CircleCI config * Resume testing all packages from release_test to simplify catching of tests given release_test was never timing out before * Remove parameter packages_select_regex to resolve package selection issues
* Refactor nav2_system_tests into Separate jobs by skipping it in release build and test jobs * Require release_test for system_build job to linearly resume cache * Keep cache test results from release_test in order to skip them from system_test * Alias matrix for system_build requires * Refactor to use package select as well to skip unrelated tests * Use yaml anchors to keep DRY * Rename regex parameters * Retain cache for build_and_test workflows by requiring release_test for system_build to ensure linear cache store and restore access * Bust cache to test CircleCI config * Resume testing all packages from release_test to simplify catching of tests given release_test was never timing out before * Remove parameter packages_select_regex to resolve package selection issues Signed-off-by: enricosutera <enricosutera@outlook.com>
from: ros-navigation#3554 Signed-off-by: enricosutera <enricosutera@outlook.com>
2 participants
to avoid CI timeout issues with CircleCI.
Related: