Skip to content

Clean package names found in python catalogers#1417

Merged
wagoodman merged 1 commit intomainfrom
clean-python-package-parsers
Dec 21, 2022
Merged

Clean package names found in python catalogers#1417
wagoodman merged 1 commit intomainfrom
clean-python-package-parsers

Conversation

@wagoodman
Copy link
Copy Markdown
Contributor

This PR fixes a couple of bugs:

  • When there are formatting directives in setup.py package names (for dynamically crafted require lines, such as %s or {}) we should skip these lines
  • When there are empty name or version fields for requirements.txt files (such as ==1.2.3) we should skip these lines

@wagoodman wagoodman added the bug Something isn't working label Dec 20, 2022
@wagoodman wagoodman self-assigned this Dec 20, 2022
@wagoodman wagoodman requested a review from a team December 20, 2022 15:46
Comment thread syft/pkg/cataloger/python/parse_requirements.go Outdated
@github-actions
Copy link
Copy Markdown

github-actions Bot commented Dec 20, 2022

Benchmark Test Results

Benchmark results from the latest changes vs base branch
name                                                       old time/op    new time/op    delta
ImagePackageCatalogers/alpmdb-cataloger-2                    13.3ms ± 1%    13.6ms ± 2%    ~     (p=0.114 n=4+4)
ImagePackageCatalogers/ruby-gemspec-cataloger-2              1.55ms ± 6%    1.54ms ± 2%    ~     (p=0.841 n=5+5)
ImagePackageCatalogers/python-package-cataloger-2            3.89ms ± 1%    3.86ms ± 0%    ~     (p=0.222 n=5+5)
ImagePackageCatalogers/php-composer-installed-cataloger-2    1.28ms ± 1%    1.28ms ± 1%    ~     (p=0.690 n=5+5)
ImagePackageCatalogers/javascript-package-cataloger-2         888µs ± 1%     890µs ± 1%    ~     (p=0.421 n=5+5)
ImagePackageCatalogers/dpkgdb-cataloger-2                    1.01ms ± 1%    1.03ms ± 1%  +2.04%  (p=0.008 n=5+5)
ImagePackageCatalogers/rpm-db-cataloger-2                    1.50ms ± 1%    1.54ms ± 1%  +2.23%  (p=0.008 n=5+5)
ImagePackageCatalogers/java-cataloger-2                      17.0ms ± 2%    17.4ms ± 2%    ~     (p=0.056 n=5+5)
ImagePackageCatalogers/apkdb-cataloger-2                     1.03ms ± 1%    1.04ms ± 2%    ~     (p=0.151 n=5+5)
ImagePackageCatalogers/go-module-binary-cataloger-2          7.44µs ± 2%    7.49µs ± 1%    ~     (p=0.421 n=5+5)
ImagePackageCatalogers/dotnet-deps-cataloger-2               1.60ms ± 1%    1.63ms ± 3%    ~     (p=0.421 n=5+5)
ImagePackageCatalogers/portage-cataloger-2                    848µs ± 1%     857µs ± 1%    ~     (p=0.095 n=5+5)
ImagePackageCatalogers/sbom-cataloger-2                      5.35ms ± 1%    5.43ms ± 1%  +1.47%  (p=0.016 n=5+5)
ImagePackageCatalogers/binary-cataloger-2                    4.74ms ± 1%    4.76ms ± 1%    ~     (p=0.151 n=5+5)

name                                                       old alloc/op   new alloc/op   delta
ImagePackageCatalogers/alpmdb-cataloger-2                    5.26MB ± 0%    5.26MB ± 0%    ~     (p=0.841 n=5+5)
ImagePackageCatalogers/ruby-gemspec-cataloger-2               205kB ± 0%     205kB ± 0%    ~     (p=0.548 n=5+5)
ImagePackageCatalogers/python-package-cataloger-2             962kB ± 0%     962kB ± 0%    ~     (p=0.222 n=5+5)
ImagePackageCatalogers/php-composer-installed-cataloger-2     217kB ± 0%     218kB ± 0%    ~     (p=0.421 n=5+5)
ImagePackageCatalogers/javascript-package-cataloger-2         159kB ± 0%     159kB ± 0%    ~     (p=0.056 n=5+5)
ImagePackageCatalogers/dpkgdb-cataloger-2                     199kB ± 0%     200kB ± 0%  +0.14%  (p=0.016 n=5+5)
ImagePackageCatalogers/rpm-db-cataloger-2                     303kB ± 0%     303kB ± 0%    ~     (p=0.063 n=5+5)
ImagePackageCatalogers/java-cataloger-2                      3.49MB ± 0%    3.49MB ± 0%    ~     (p=0.841 n=5+5)
ImagePackageCatalogers/apkdb-cataloger-2                      182kB ± 0%     182kB ± 0%    ~     (p=0.222 n=5+5)
ImagePackageCatalogers/go-module-binary-cataloger-2          1.12kB ± 0%    1.12kB ± 0%    ~     (all equal)
ImagePackageCatalogers/dotnet-deps-cataloger-2                375kB ± 0%     375kB ± 0%    ~     (p=0.841 n=5+5)
ImagePackageCatalogers/portage-cataloger-2                    139kB ± 0%     139kB ± 0%    ~     (p=1.000 n=5+5)
ImagePackageCatalogers/sbom-cataloger-2                       722kB ± 0%     722kB ± 0%  +0.01%  (p=0.008 n=5+5)
ImagePackageCatalogers/binary-cataloger-2                     656kB ± 0%     656kB ± 0%  +0.01%  (p=0.008 n=5+5)

name                                                       old allocs/op  new allocs/op  delta
ImagePackageCatalogers/alpmdb-cataloger-2                     85.7k ± 0%     85.7k ± 0%    ~     (p=1.000 n=5+5)
ImagePackageCatalogers/ruby-gemspec-cataloger-2               4.25k ± 0%     4.25k ± 0%    ~     (all equal)
ImagePackageCatalogers/python-package-cataloger-2             16.5k ± 0%     16.5k ± 0%    ~     (p=0.087 n=5+5)
ImagePackageCatalogers/php-composer-installed-cataloger-2     5.50k ± 0%     5.50k ± 0%    ~     (p=1.000 n=5+5)
ImagePackageCatalogers/javascript-package-cataloger-2         3.33k ± 0%     3.33k ± 0%    ~     (all equal)
ImagePackageCatalogers/dpkgdb-cataloger-2                     4.47k ± 0%     4.47k ± 0%    ~     (all equal)
ImagePackageCatalogers/rpm-db-cataloger-2                     8.12k ± 0%     8.12k ± 0%    ~     (all equal)
ImagePackageCatalogers/java-cataloger-2                       57.5k ± 0%     57.5k ± 0%    ~     (p=0.325 n=5+5)
ImagePackageCatalogers/apkdb-cataloger-2                      5.23k ± 0%     5.23k ± 0%    ~     (p=0.095 n=5+4)
ImagePackageCatalogers/go-module-binary-cataloger-2            38.0 ± 0%      38.0 ± 0%    ~     (all equal)
ImagePackageCatalogers/dotnet-deps-cataloger-2                7.12k ± 0%     7.12k ± 0%    ~     (p=1.000 n=5+5)
ImagePackageCatalogers/portage-cataloger-2                    3.58k ± 0%     3.58k ± 0%    ~     (all equal)
ImagePackageCatalogers/sbom-cataloger-2                       24.4k ± 0%     24.4k ± 0%    ~     (all equal)
ImagePackageCatalogers/binary-cataloger-2                     22.2k ± 0%     22.2k ± 0%    ~     (all equal)

Signed-off-by: Alex Goodman <alex.goodman@anchore.com>
@wagoodman wagoodman force-pushed the clean-python-package-parsers branch from f06afa1 to 8a36d2c Compare December 21, 2022 18:10
@wagoodman wagoodman enabled auto-merge (squash) December 21, 2022 18:11
@wagoodman wagoodman merged commit 5dd726f into main Dec 21, 2022
@wagoodman wagoodman deleted the clean-python-package-parsers branch December 21, 2022 18:31
GijsCalis pushed a commit to GijsCalis/syft that referenced this pull request Feb 19, 2024
Signed-off-by: Alex Goodman <alex.goodman@anchore.com>

Signed-off-by: Alex Goodman <alex.goodman@anchore.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

bug Something isn't working

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants