fix: propagate failed scans with --all-projects #1301
Conversation
lili2311
force-pushed
the
fix/throw-erorr-if-all-tests-failed
branch
from
87ef9d9 to
c8598d3
Compare
| analytics.add('pluginName', inspectResult.plugin.name); | ||
|
|
||
| const postingMonitorSpinnerLabel = |
There was a problem hiding this comment.
moved a little further down unchanged
lili2311
force-pushed
the
fix/throw-erorr-if-all-tests-failed
branch
3 times, most recently
from
93e7e73 to
b2954d6
Compare
| allProjects: true, | ||
| }); | ||
| let result; | ||
| try { |
There was a problem hiding this comment.
these were not previously propagated up if dependencies failed to be detected, but we do fail if one of the paths fails to be scanned so this aligns that behaviour.
This may however start failing some pipelines that previously were skipping the failures. How can this be communicated & is this the desired behaviour right? You want to know something you monitored failed and it did not pass.
lili2311
force-pushed
the
fix/throw-erorr-if-all-tests-failed
branch
4 times, most recently
from
af49e17 to
7e9c7ef
Compare
ghost
requested review from
lili2311
force-pushed
the
fix/throw-erorr-if-all-tests-failed
branch
from
7e9c7ef to
5cc6eee
Compare
- failed monitor paths generate an error but not failed attempts to get dependencies, these were skipped and shown only with -d output. Propagate full status back to the user. - this may affect the expected exit code for some users where previous failures were skipped in this manner - log failed snyk test scans with --all-projects
lili2311
force-pushed
the
fix/throw-erorr-if-all-tests-failed
branch
from
5cc6eee to
7ef59ed
Compare
|
Expected release notes (by @lili2311) fixes: others (will not be included in Semantic-Release notes):
|
|
🎉 This PR is included in version 1.369.3 🎉 The release is available on: Your semantic-release bot 📦🚀 |
What does this PR do?
--all-projectsand propagate it higher up.monitorthese are now propagated all the way and will appear in output + json output. This may affect the existing error code as previously we silently skipped failure like these but not failures to get vulnerabilities during monitor or when a path test failed. So it was inconsistent.-dWhere should the reviewer start?
https://github.com/snyk/snyk/compare/fix/throw-erorr-if-all-tests-failed?expand=1#diff-e4ec2ebbae5004d31be7a5b680580a46R42
How should this be manually tested?
You can use
cd snyk/test/acceptance/workspaces && snyk test --all-projectsandcd snyk/test/acceptance/workspaces && snyk monitor --all-projectsto see the output.Any background context you want to provide?
Sometimes users see a blank screen when 100% of
-all-projectsscans fail and we do not flag about other failures as it was too noisy unless you run with-dScreenshots