DLPX-84163 Update all the greater than equal to (>=) dependencies in setup.py and requirements.txt files of dvp packages to equal to(==)

[SumoSourabh]

Background

There are around 40 PRs opened by dependentBot to bump up the dependencies version. Also, some dependencies in our setup.py uses >= dependencies for example flake8 >= 3.6 which can cause some unknown errors in case dependencies releases a new version.

Problem

Recently, flake8 released 6.0.0 version which started causing dvp build to fail with 'Manager' object has no attribute 'checkers', as flake8 6.0.0 has made changes to manager package.

Evaluation

The exact cause of the issue is plugin_validator.py in tools package (line no. 213) where we try to access checkers object @style_guide._application.file_checker_manager.checkers.

As flake8 6.0.0 has removed checkers and started using result object, the issue occurred.

Solution

1. Change all >= to >=, < or == in setup.py files of all packages.
2. Use flake8 == 6.0.0 and update plugin_validator.py to sync with flake8 changes.
3. Updated all the dependencies to latest version as required in requirements.txt files.
   • bump2version 0.5.11 -> 1.0.1 for common, dvp, libs, platform, tools
   • packaging 19.2 -> 22.0 for common, dvp, libs, platform, tools
   • pluggy 0.13.0 -> 1.0.0 for common, dvp, libs, platform, tools
   • pyparsing 2.4.5 -> 3.0.9 for common, dvp, libs, platform, tools
   • pytest 4.6.11 -> 7.2.0 for common, dvp, libs, platform, tools
   • six 1.13.0 -> 1.16.0 for common, dvp, libs, platform, tools
   • zipp 0.6.0 -> 3.11.0 for common, dvp, libs, platform, tools
   • mock 3.0.5 -> 4.0.3 for libs, platform, tools
   • coverage 5.0.2 -> 6.5.0 for tools
   • entrypoints 0.3 -> 0.4 for tools
   • flake8 3.7.9 -> 6.0.0 for tools
   • httpretty 0.9.7 -> 1.0.5 for tools
   • isort 4.3.21 -> 5.11.1 for tools
   • mccabe 0.6.1 -> 0.7.0 for tools
   • more-itertools 5.0.0 -> 9.0.0 for tools
   • pycodestyle 2.5.0 -> 2.10.0 for tools
   • pyflakes 2.1.1 -> 3.0.1 for tools
   • pytest-cov 2.8.1 -> 4.0.0 for tools
   • yapf 0.28 -> 0.32 for tools
4. Removed older and unused dependencies.
   • All dependencies with python_version less than 3.8 have been removed.
   • enum34 and py have been removed as they were not being used anywhere in the code.

Testing Done

appdata_python_samples

• http://selfservice.jenkins.delphix.com/job/blackbox-self-service/62929

appdata_basic

• Appdata Python nix_direct_python - CentOS 8.0
• Appdata Python nix_staged_python - RHEL 7.9
• Appdata Python win_direct_python - Win 2016 - Facing some issues with machines for this run. Tested the same without the changes and got the same errors.
• Appdata Python win_staged_python - Win 2012

virtualization_sdk

• Nix direct appdata python plugin
• Nix staged appdata python plugin
• Windows direct appdata python plugin
• Windows staged appdata python plugin
• Windows staged default plugin built on Windows 10 Pro SDK host

Bonus

• PRs that will be closed
  • Bump pytest-cov 388
  • Bump six 384, 383, 381, 379, 378
  • Bump flake8 380
  • Bump pyflakes 376
  • Bump yafp 375
  • Bump pycodestyle 374
  • Bump py 372, 368, 367, 366, 365, 364, 300, 296, 295, 294
  • Bump enum34 354, 352, 162
  • Bump coverage 353
  • Bump packaging 329, 328, 327, 324
  • Bump pytest 189
  • Bump zipp 161, 142, 139, 131, 130
  • Bump pluggy 155
  • Bump pyparsing 151, 145, 138, 137, 133
• dependentBot Security alerts will be resolved once changes are merged.

Dependency Tree for dvp 4.0.5 install

Dependency Tree for local changes install

Diff of Pip Freeze