Skip to content

Commit

Permalink
Merge branch 'develop' into rename_match_spdx_expression_3838
Browse files Browse the repository at this point in the history 
Signed-off-by: Ayan Sinha Mahapatra <[email protected]>
  • Loading branch information
@AyanSinhaMahapatra
AyanSinhaMahapatra committed Oct 19, 2024
2 parents 8a0a030 + 9b3cf12 commit d26142c
Show file tree
Hide file tree
Showing 2,796 changed files with 145,061 additions and 64,742 deletions.
27 changes: 24 additions & 3 deletions CHANGELOG.rst
View file
Original file line number Diff line number Diff line change
Expand Up @@ -9,10 +9,13 @@ v33.0.0 (next next, roadmap)
- OpenWRT packages.
- Yocto/BitBake .bb recipes.


- Fallback packages for non-native dependencies of SCTK.
- Dependencies for
- Support for copyright detection objects.

- We can now collect packages from a Go binary using go-inspector (Linux-only)

- A new field in packages with the license category for the
detected license expression and also an API function to
compute license categories from license expressions.
Expand All @@ -34,6 +37,22 @@ v33.0.0 (next next, roadmap)
of these in other summary plugins.
See https://github.com/nexB/scancode-toolkit/issues/1745

- Update link references of ownership from nexB to aboutcode-org
See https://github.com/aboutcode-org/scancode-toolkit/issues/3885

- New and updated licenses, including support for newly released
SPDX license list versions:
- SPDX License List 3.25.0:
This release of the SPDX license list had 9 new licenses
and exceptions, and out of them 5 were present as licenses
and 2 were present as rules already. There were 2 new
license/exception texts added, and also 1 license was deprecated.
For more details see https://github.com/aboutcode-org/scancode-toolkit/pull/3897

- New and improved copyright detection with many false positive removed
and refined detection added.

- Fix Python ``SyntaxWarning`` in textcode module.

v32.2.1 - 2024-07-02
---------------------
Expand Down Expand Up @@ -155,7 +174,7 @@ Changes in Output Data Structure:
file-level ``package_data``
* ``license_detections`` and ``other_license_detections`` in
codebase level ``packages``

- On using the CLI option ``--license-text-diagnostics`` there is
now a new license match attribute ``matched_text_diagnostics``
with the matched text and highlighted diagnostics, instead of
Expand All @@ -164,7 +183,7 @@ Changes in Output Data Structure:
- A new ``reference_matches`` attribute is added to codebase-level
``license_detections`` which is same as the ``matches`` attribute
in other license detections.

- We now have SPDX license expressions everywhere we have
ScanCode license expressions for ease of use and adopting
SPDX everywhere. A new attribute ``license_expression_spdx``
Expand Down Expand Up @@ -212,7 +231,7 @@ Changes in Output Data Structure:
and https://github.com/nexB/scancode-toolkit/issues/3443
Also improve debian manifests parsing and purl parsing from
filenames. Support for https://github.com/nexB/purldb/issues/245
Bumps debian-inspector to v31.1.0
Bumps debian-inspector to v31.1.0

- Bump commoncode to v31.0.3

Expand Down Expand Up @@ -870,6 +889,8 @@ Package detection:

- For Pypi packages, python_requires is treated as a package dependency.

- Update JavaScript package handlers to handle aliases in npm and yarn manifests.


License Clarity Scoring Update:
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Expand Down
142 changes: 141 additions & 1 deletion README.rst
View file
Original file line number Diff line number Diff line change
Expand Up @@ -89,7 +89,7 @@ Why use ScanCode?
InstallShield installers, iOS apps, ISO images, Apache IVY, JBoss Sar,
R CRAN, Apache Maven, Meteor, Mozilla extensions, MSI installers,
JavaScript npm packages, package-lock.json, yarn.lock, NSIS Installers,
NugGet, OPam, Cocoapods, Python PyPI setup.py, setup.cfg, and
NuGet, OPam, Cocoapods, Python PyPI setup.py, setup.cfg, and
several related lockfile formats, semi structured README
files such as README.android, README.chromium, README.facebook, README.google,
README.thirdparty, RPMs, Shell Archives, Squashfs images, Java WAR, Windows
Expand Down Expand Up @@ -246,6 +246,7 @@ See the NOTICE file and the .ABOUT files that document the origin and license of
the third-party code used in ScanCode for more details.



.. |azure| image:: https://dev.azure.com/nexB/scancode-toolkit/_apis/build/status/nexB.scancode-toolkit?branchName=develop
:target: https://dev.azure.com/nexB/scancode-toolkit/_build/latest?definitionId=1&branchName=develop
:alt: Azure tests status (Linux, macOS, Windows)
Expand All @@ -261,3 +262,142 @@ the third-party code used in ScanCode for more details.
.. |release-github-actions| image:: https://github.com/nexB/scancode-toolkit/actions/workflows/scancode-release.yml/badge.svg?event=push
:target: https://github.com/nexB/scancode-toolkit/actions/workflows/scancode-release.yml
:alt: Release tests


Acknowledgements, Funding, Support and Sponsoring
--------------------------------------------------------

This project is funded, supported and sponsored by:

- Generous support and contributions from users like you!
- the European Commission NGI programme
- the NLnet Foundation
- the Swiss State Secretariat for Education, Research and Innovation (SERI)
- Google, including the Google Summer of Code and the Google Seasons of Doc programmes
- Mercedes-Benz Group
- Microsoft and Microsoft Azure
- AboutCode ASBL
- nexB Inc.



|europa| |dgconnect|

|ngi| |nlnet|

|aboutcode| |nexb|


This project was funded through the NGI0 Discovery Fund, a fund established by NLnet with financial
support from the European Commission's Next Generation Internet programme, under the aegis of DG
Communications Networks, Content and Technology under grant agreement No 825322.

|ngidiscovery| https://nlnet.nl/project/vulnerabilitydatabase/


This project was funded through the NGI0 Entrust Fund, a fund established by NLnet with financial
support from the European Commission's Next Generation Internet programme, under the aegis of DG
Communications Networks, Content and Technology under grant agreement No 101069594.

|ngizeroentrust| https://nlnet.nl/project/Back2source/


This project was funded through the NGI0 Core Fund, a fund established by NLnet with financial
support from the European Commission's Next Generation Internet programme, under the aegis of DG
Communications Networks, Content and Technology under grant agreement No 101092990.

|ngizerocore| https://nlnet.nl/project/Back2source-next/


This project was funded through the NGI0 Core Fund, a fund established by NLnet with financial
support from the European Commission's Next Generation Internet programme, under the aegis of DG
Communications Networks, Content and Technology under grant agreement No 101092990.

|ngizerocore| https://nlnet.nl/project/FastScan/


This project was funded through the NGI0 Commons Fund, a fund established by NLnet with financial
support from the European Commission's Next Generation Internet programme, under the aegis of DG
Communications Networks, Content and Technology under grant agreement No 101135429. Additional
funding is made available by the Swiss State Secretariat for Education, Research and Innovation
(SERI).

|ngizerocommons| |swiss| https://nlnet.nl/project/MassiveFOSSscan/

This project was funded through the NGI0 Entrust Fund, a fund established by NLnet with financial
support from the European Commission's Next Generation Internet programme, under the aegis of DG
Communications Networks, Content and Technology under grant agreement No 101069594.

|ngizeroentrust| https://nlnet.nl/project/purl2sym/


.. |nlnet| image:: https://nlnet.nl/logo/banner.png
:target: https://nlnet.nl
:height: 50
:alt: NLnet foundation logo

.. |ngi| image:: https://ngi.eu/wp-content/uploads/thegem-logos/logo_8269bc6efcf731d34b6385775d76511d_1x.png
:target: https://ngi.eu35
:height: 50
:alt: NGI logo

.. |nexb| image:: https://nexb.com/wp-content/uploads/2022/04/nexB.svg
:target: https://nexb.com
:height: 30
:alt: nexB logo

.. |europa| image:: https://ngi.eu/wp-content/uploads/sites/77/2017/10/bandiera_stelle.png
:target: http://ec.europa.eu/index_en.htm
:height: 40
:alt: Europa logo

.. |aboutcode| image:: https://aboutcode.org/wp-content/uploads/2023/10/AboutCode.svg
:target: https://aboutcode.org/
:height: 30
:alt: AboutCode logo

.. |swiss| image:: https://www.sbfi.admin.ch/sbfi/en/_jcr_content/logo/image.imagespooler.png/1493119032540/logo.png
:target: https://www.sbfi.admin.ch/sbfi/en/home/seri/seri.html
:height: 40
:alt: Swiss logo

.. |dgconnect| image:: https://commission.europa.eu/themes/contrib/oe_theme/dist/ec/images/logo/positive/logo-ec--en.svg
:target: https://commission.europa.eu/about-european-commission/departments-and-executive-agencies/communications-networks-content-and-technology_en
:height: 40
:alt: EC DG Connect logo

.. |ngizerocore| image:: https://nlnet.nl/image/logos/NGI0_tag.svg
:target: https://nlnet.nl/core
:height: 40
:alt: NGI Zero Core Logo

.. |ngizerocommons| image:: https://nlnet.nl/image/logos/NGI0_tag.svg
:target: https://nlnet.nl/commonsfund/
:height: 40
:alt: NGI Zero Commons Logo

.. |ngizeropet| image:: https://nlnet.nl/image/logos/NGI0PET_tag.svg
:target: https://nlnet.nl/PET
:height: 40
:alt: NGI Zero PET logo

.. |ngizeroentrust| image:: https://nlnet.nl/image/logos/NGI0Entrust_tag.svg
:target: https://nlnet.nl/entrust
:height: 38
:alt: NGI Zero Entrust logo

.. |ngiassure| image:: https://nlnet.nl/image/logos/NGIAssure_tag.svg
:target: https://nlnet.nl/image/logos/NGIAssure_tag.svg
:height: 32
:alt: NGI Assure logo

.. |ngidiscovery| image:: https://nlnet.nl/image/logos/NGI0Discovery_tag.svg
:target: https://nlnet.nl/discovery/
:height: 40
:alt: NGI Discovery logo






27 changes: 27 additions & 0 deletions azure-pipelines.yml
View file
Original file line number Diff line number Diff line change
Expand Up @@ -75,6 +75,33 @@ jobs:
tests/licensedcode/test_detection_validate.py \
-k TestValidateLicenseExtended5
license_validate_ignorables_1: |
venv/bin/pytest -n 3 -vvs --test-suite=validate \
tests/licensedcode/test_detection_validate.py \
-k TestValidateLicenseIgnorableClues1
license_validate_ignorables_2: |
venv/bin/pytest -n 3 -vvs --test-suite=validate \
tests/licensedcode/test_detection_validate.py \
-k TestValidateLicenseIgnorableClues2
license_validate_ignorables_3: |
venv/bin/pytest -n 3 -vvs --test-suite=validate \
tests/licensedcode/test_detection_validate.py \
-k TestValidateLicenseIgnorableClues3
license_validate_ignorables_4: |
venv/bin/pytest -n 3 -vvs --test-suite=validate \
tests/licensedcode/test_detection_validate.py \
-k TestValidateLicenseIgnorableClues4
license_validate_ignorables_5: |
venv/bin/pytest -n 3 -vvs --test-suite=validate \
tests/licensedcode/test_detection_validate.py \
-k TestValidateLicenseIgnorableClues5
license_cache: |
venv/bin/pytest -n 3 -vvs --test-suite=all \
tests/licensedcode/test_zzzz_cache.py --reruns 2
Expand Down
18 changes: 9 additions & 9 deletions docs/source/cli-reference/basic-options.rst
View file
Original file line number Diff line number Diff line change
Expand Up @@ -108,7 +108,7 @@
"license_expression": "apache-2.0",
"rule_identifier": "apache-2.0_65.RULE",
"rule_relevance": 100,
"rule_url": "https://github.com/nexB/scancode-toolkit/tree/develop/src/licensedcode/data/rules/apache-2.0_65.RULE",
"rule_url": "https://github.com/aboutcode-org/scancode-toolkit/tree/develop/src/licensedcode/data/rules/apache-2.0_65.RULE",
"matched_text": "License: Apache-2.0"
}
],
Expand Down Expand Up @@ -587,19 +587,19 @@

A scan example using the ``--license-url-template TEXT`` option ::

scancode -clpieu --json-pp output.json samples --license-url-template https://github.com/nexB/scancode-toolkit/tree/develop/src/licensedcode/data/licenses/{}.LICENSE
scancode -clpieu --json-pp output.json samples --license-url-template https://github.com/aboutcode-org/scancode-toolkit/tree/develop/src/licensedcode/data/licenses/{}.LICENSE

In a normal scan, reference url for "ZLIB License" is as follows::

"reference_url": "https://scancode-licensedb.aboutcode.org/zlib",

After using the option in the following manner::

``--license-url-template https://github.com/nexB/scancode-toolkit/tree/develop/src/licensedcode/data/licenses/{}.LICENSE``
``--license-url-template https://github.com/aboutcode-org/scancode-toolkit/tree/develop/src/licensedcode/data/licenses/{}.LICENSE``

the reference URL changes to this `zlib.LICENSE file <https://github.com/nexB/scancode-toolkit/blob/develop/src/licensedcode/data/licenses/zlib.LICENSE>`_::
the reference URL changes to this `zlib.LICENSE file <https://github.com/aboutcode-org/scancode-toolkit/blob/develop/src/licensedcode/data/licenses/zlib.LICENSE>`_::

"reference_url": "https://github.com/nexB/scancode-toolkit/blob/develop/src/licensedcode/data/licenses/zlib.LICENSE",
"reference_url": "https://github.com/aboutcode-org/scancode-toolkit/blob/develop/src/licensedcode/data/licenses/zlib.LICENSE",

The reference URL changes for all detected licenses in the scan, across the scan result file.

Expand Down Expand Up @@ -691,7 +691,7 @@
"license_expression": "unknown-license-reference",
"rule_identifier": "lead-in_unknown_30.RULE",
"rule_relevance": 100,
"rule_url": "https://github.com/nexB/scancode-toolkit/tree/develop/src/licensedcode/data/rules/lead-in_unknown_30.RULE",
"rule_url": "https://github.com/aboutcode-org/scancode-toolkit/tree/develop/src/licensedcode/data/rules/lead-in_unknown_30.RULE",
"matched_text": "dual-licensed under [`
},
{
Expand All @@ -704,7 +704,7 @@
"license_expression": "wtfpl-2.0",
"rule_identifier": "spdx_license_id_wtfpl_for_wtfpl-2.0.RULE",
"rule_relevance": 50,
"rule_url": "https://github.com/nexB/scancode-toolkit/tree/develop/src/licensedcode/data/rules/spdx_license_id_wtfpl_for_wtfpl-2.0.RULE",
"rule_url": "https://github.com/aboutcode-org/scancode-toolkit/tree/develop/src/licensedcode/data/rules/spdx_license_id_wtfpl_for_wtfpl-2.0.RULE",
"matched_text": "WTFPL"
},
{
Expand All @@ -717,7 +717,7 @@
"license_expression": "wtfpl-2.0",
"rule_identifier": "wtfpl-2.0_27.RULE",
"rule_relevance": 100,
"rule_url": "https://github.com/nexB/scancode-toolkit/tree/develop/src/licensedcode/data/rules/wtfpl-2.0_27.RULE",
"rule_url": "https://github.com/aboutcode-org/scancode-toolkit/tree/develop/src/licensedcode/data/rules/wtfpl-2.0_27.RULE",
"matched_text": "www.wtfpl.net/"
},
{
Expand All @@ -730,7 +730,7 @@
"license_expression": "mit",
"rule_identifier": "mit_64.RULE",
"rule_relevance": 100,
"rule_url": "https://github.com/nexB/scancode-toolkit/tree/develop/src/licensedcode/data/rules/mit_64.RULE",
"rule_url": "https://github.com/aboutcode-org/scancode-toolkit/tree/develop/src/licensedcode/data/rules/mit_64.RULE",
"matched_text": "MIT`](https://opensource.org/licenses/MIT)."
}
],
Expand Down
4 changes: 2 additions & 2 deletions docs/source/cli-reference/help-text-options.rst
View file
Original file line number Diff line number Diff line change
Expand Up @@ -79,7 +79,7 @@ The Following Help Text is displayed, i.e. This is the help text for Scancode Ve
--csv FILE [DEPRECATED] Write scan output as CSV to FILE. The
--csv option is deprecated and will be replaced by new
CSV and tabular output formats in the next ScanCode
release. Visit https://github.com/nexB/scancode-
release. Visit https://github.com/aboutcode-org/scancode-
toolkit/issues/3043 to provide inputs and feedback.
--html FILE Write scan output as HTML to FILE.
--custom-output FILE Write scan output to FILE formatted with the custom
Expand Down Expand Up @@ -321,7 +321,7 @@ The Following Text is displayed, i.e. This is the available plugins for Scancode
required_plugins:
options:
help_group: output formats, name: csv: --csv
help: [DEPRECATED] Write scan output as CSV to FILE. The --csv option is deprecated and will be replaced by new CSV and tabular output formats in the next ScanCode release. Visit https://github.com/nexB/scancode-toolkit/issues/3043 to provide inputs and feedback.
help: [DEPRECATED] Write scan output as CSV to FILE. The --csv option is deprecated and will be replaced by new CSV and tabular output formats in the next ScanCode release. Visit https://github.com/aboutcode-org/scancode-toolkit/issues/3043 to provide inputs and feedback.
doc: None

--------------------------------------------
Expand Down
Loading

0 comments on commit d26142c

Please sign in to comment.