Skip to content

Relax error conditions for catalogers#1492

Merged
wagoodman merged 3 commits intomainfrom
binary-cataloger-suppress-errors
Jan 20, 2023
Merged

Relax error conditions for catalogers#1492
wagoodman merged 3 commits intomainfrom
binary-cataloger-suppress-errors

Conversation

@wagoodman
Copy link
Copy Markdown
Contributor

@wagoodman wagoodman commented Jan 19, 2023

This PR adjusts the binary cataloger to compartmentalize errors to individual classifiers and continue to the next classifier when an error occurs.

This prevents errors from being fatal, such as:

[0001] TRACE cataloging complete cataloger=portage-cataloger
[0001] TRACE cataloging started cataloger=sbom-cataloger
[0001] DEBUG discovered 0 packages cataloger=sbom-cataloger
[0001] TRACE cataloging complete cataloger=sbom-cataloger
[0001] TRACE cataloging started cataloger=binary-cataloger
[0001]  WARN error while cataloging cataloger=binary-cataloger
No packages discovered
2023/01/19 13:12:27 error during command execution: 1 error occurred:
	* failed to resolve files by glob (**/python*): cycle during symlink resolution

Signed-off-by: Alex Goodman <alex.goodman@anchore.com>
@wagoodman wagoodman added the bug Something isn't working label Jan 19, 2023
@github-actions
Copy link
Copy Markdown

github-actions Bot commented Jan 19, 2023

Benchmark Test Results

Benchmark results from the latest changes vs base branch
goos: linux
goarch: amd64
pkg: github.com/anchore/syft/test/integration
cpu: Intel(R) Xeon(R) Platinum 8171M CPU @ 2.60GHz
                                                          │ ./.tmp/benchmark-3f965a0.txt │
                                                          │            sec/op            │
ImagePackageCatalogers/alpmdb-cataloger-2                                   14.43m ±  1%
ImagePackageCatalogers/ruby-gemspec-cataloger-2                             1.595m ± 14%
ImagePackageCatalogers/python-package-cataloger-2                           4.053m ±  1%
ImagePackageCatalogers/php-composer-installed-cataloger-2                   1.325m ±  1%
ImagePackageCatalogers/javascript-package-cataloger-2                       938.8µ ±  1%
ImagePackageCatalogers/dpkgdb-cataloger-2                                   1.068m ±  1%
ImagePackageCatalogers/rpm-db-cataloger-2                                   1.578m ±  1%
ImagePackageCatalogers/java-cataloger-2                                     17.16m ±  1%
ImagePackageCatalogers/graalvm-native-image-cataloger-2                     8.196µ ±  1%
ImagePackageCatalogers/apkdb-cataloger-2                                    1.078m ±  0%
ImagePackageCatalogers/go-module-binary-cataloger-2                         17.21µ ±  1%
ImagePackageCatalogers/dotnet-deps-cataloger-2                              1.644m ±  3%
ImagePackageCatalogers/portage-cataloger-2                                  913.3µ ±  5%
ImagePackageCatalogers/sbom-cataloger-2                                     5.594m ±  1%
ImagePackageCatalogers/binary-cataloger-2                                   7.889m ±  1%
geomean                                                                     1.265m

                                                          │ ./.tmp/benchmark-3f965a0.txt │
                                                          │             B/op             │
ImagePackageCatalogers/alpmdb-cataloger-2                                   5.041Mi ± 0%
ImagePackageCatalogers/ruby-gemspec-cataloger-2                             204.5Ki ± 0%
ImagePackageCatalogers/python-package-cataloger-2                           953.6Ki ± 0%
ImagePackageCatalogers/php-composer-installed-cataloger-2                   217.2Ki ± 0%
ImagePackageCatalogers/javascript-package-cataloger-2                       158.7Ki ± 0%
ImagePackageCatalogers/dpkgdb-cataloger-2                                   199.9Ki ± 0%
ImagePackageCatalogers/rpm-db-cataloger-2                                   300.4Ki ± 0%
ImagePackageCatalogers/java-cataloger-2                                     3.362Mi ± 0%
ImagePackageCatalogers/graalvm-native-image-cataloger-2                     1.375Ki ± 0%
ImagePackageCatalogers/apkdb-cataloger-2                                    181.4Ki ± 0%
ImagePackageCatalogers/go-module-binary-cataloger-2                         1.809Ki ± 0%
ImagePackageCatalogers/dotnet-deps-cataloger-2                              375.8Ki ± 0%
ImagePackageCatalogers/portage-cataloger-2                                  138.1Ki ± 0%
ImagePackageCatalogers/sbom-cataloger-2                                     724.7Ki ± 0%
ImagePackageCatalogers/binary-cataloger-2                                   1.028Mi ± 0%
geomean                                                                     219.0Ki

                                                          │ ./.tmp/benchmark-3f965a0.txt │
                                                          │          allocs/op           │
ImagePackageCatalogers/alpmdb-cataloger-2                                    85.84k ± 0%
ImagePackageCatalogers/ruby-gemspec-cataloger-2                              4.350k ± 0%
ImagePackageCatalogers/python-package-cataloger-2                            16.82k ± 0%
ImagePackageCatalogers/php-composer-installed-cataloger-2                    5.607k ± 0%
ImagePackageCatalogers/javascript-package-cataloger-2                        3.431k ± 0%
ImagePackageCatalogers/dpkgdb-cataloger-2                                    4.578k ± 0%
ImagePackageCatalogers/rpm-db-cataloger-2                                    8.314k ± 0%
ImagePackageCatalogers/java-cataloger-2                                      58.33k ± 0%
ImagePackageCatalogers/graalvm-native-image-cataloger-2                       32.00 ± 0%
ImagePackageCatalogers/apkdb-cataloger-2                                     5.332k ± 0%
ImagePackageCatalogers/go-module-binary-cataloger-2                           55.00 ± 0%
ImagePackageCatalogers/dotnet-deps-cataloger-2                               7.227k ± 0%
ImagePackageCatalogers/portage-cataloger-2                                   3.682k ± 0%
ImagePackageCatalogers/sbom-cataloger-2                                      25.31k ± 0%
ImagePackageCatalogers/binary-cataloger-2                                    36.81k ± 0%
geomean                                                                      5.244k

Signed-off-by: Christopher Phillips <christopher.phillips@anchore.com>
@wagoodman wagoodman enabled auto-merge (squash) January 19, 2023 23:25
Signed-off-by: Christopher Phillips <christopher.phillips@anchore.com>
@wagoodman wagoodman merged commit 0f75f97 into main Jan 20, 2023
@wagoodman wagoodman deleted the binary-cataloger-suppress-errors branch January 20, 2023 00:28
GijsCalis pushed a commit to GijsCalis/syft that referenced this pull request Feb 19, 2024
* binary cataloger should continue on errors

Signed-off-by: Alex Goodman <alex.goodman@anchore.com>

* test: add redirect for cmd stderr stdout

Signed-off-by: Christopher Phillips <christopher.phillips@anchore.com>

* test: image update for test failure

Signed-off-by: Christopher Phillips <christopher.phillips@anchore.com>

Signed-off-by: Alex Goodman <alex.goodman@anchore.com>
Signed-off-by: Christopher Phillips <christopher.phillips@anchore.com>
Co-authored-by: Christopher Phillips <christopher.phillips@anchore.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

bug Something isn't working

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants