Replace pipdeptree with python-inspector

[pombredanne]

This PR replaces pipdeptree with python-inspector to resolve Python packages dependencies found in requirement files.

python-inspector can resolve dependencies for any target Python version and OS (and not only the one running the tool).
In this integration in ORT, it replaces pipdeptree pretty much in place as python-inspector implements a similar output data structure by design to ease the integration.

Reference: https://github.com/nexB/python-inspector
Reference: #4637
Reference: #3671
Signed-off-by: Tushar Goel [email protected]
Signed-off-by: Philippe Ombredanne [email protected]

[mnonnenmacher]

Commit message: Please add the details from the PR description to the commit message as well, to make sure it is recorded in the Git log. Also, please prefix the title with "Pip: Replace ...".

[pombredanne]

Done

[mnonnenmacher]

Please also remove the ticket ids from the commit message title, it is enough to mention them in the body. And we usually use Relates-to: #4637 for ticket references.

[codecov]

Codecov Report

Merging #5645 (0ce6b0f) into main (65c1565) will increase coverage by 0.08%.
The diff coverage is 71.42%.

@@             Coverage Diff              @@
##               main    #5645      +/-   ##
============================================
+ Coverage     65.46%   65.54%   +0.08%     
+ Complexity     2219     2212       -7     
============================================
  Files           271      271              
  Lines         16594    16600       +6     
  Branches       3445     3473      +28     
============================================
+ Hits          10863    10881      +18     
+ Misses         4588     4575      -13     
- Partials       1143     1144       +1     

Flag	Coverage Δ
funTest-analyzer-docker	74.58% <83.33%> (+0.09%)	⬆️
funTest-non-analyzer	46.12% <ø> (ø)
test	32.01% <0.00%> (-0.05%)	⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.

Impacted Files	Coverage Δ
analyzer/src/main/kotlin/managers/Pip.kt	49.70% <71.42%> (+2.96%)	⬆️

Help us with your feedback. Take ten seconds to tell us how you rate us. Have a feature suggestion? Share it here.

[oheger-bosch]

There are two (trivial) issues in the static code analysis: Multiple blank lines at lines 64 and 197.

[oheger-bosch]

The changes look good to me, however, the commits probably need to be reworked. As stated in the contributor guide, requested changes should not be added in new commits, but the existing commits should be amended. Could you please do this?

[pombredanne]

The changes look good to me, however, the commits probably need to be reworked. As stated in the contributor guide, requested changes should not be added in new commits, but the existing commits should be amended. Could you please do this?

@oheger-bosch Thanks! My mistake there.

[mnonnenmacher]

@pombredanne This assumption is not correct anymore with the python-inspector. I tested it locally and the python-inspector output is:

[
  {
    "key": "ply",
    "package_name": "ply",
    "installed_version": "3.11",
    "dependencies": []
  },
  {
    "key": "rdflib",
    "package_name": "rdflib",
    "installed_version": "6.2.0",
    "dependencies": [
      {
        "key": "isodate",
        "package_name": "isodate",
        "installed_version": "0.6.1",
        "dependencies": [
          {
            "key": "six",
            "package_name": "six",
            "installed_version": "1.16.0",
            "dependencies": []
          }
        ]
      },
      {
        "key": "pyparsing",
        "package_name": "pyparsing",
        "installed_version": "3.0.9",
        "dependencies": []
      },
      {
        "key": "setuptools",
        "package_name": "setuptools",
        "installed_version": "65.2.0",
        "dependencies": []
      }
    ]
  }
]

So it does not contain a root node with the project name. Therefore the function returns EMPTY_JSON_NODE and the spdx-tools-python test is failing. The solution is to simplify lines 367-379 to:

val projectDependencies = fullDependencyTree.filterNot {
    isPhonyDependency(it["package_name"].textValue(), it["installed_version"].textValueOrEmpty())
}

(and to remove the projectName parameter of the function because it is unused now)

The test will still fail because "38" is always passed to python-inspector as Python version and therefore different versions of the dependencies are found. I would suggest to ignore this issue for now and just update the expected results file, and I will address this topic in a separate PR as discussed in the meeting today.