Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Performance: Stop using pkg_resources/declare_namespace #2951

Merged
merged 20 commits into from
Apr 25, 2017

Conversation

johanste
Copy link
Member

Loading pkg_resources and calling declare_namespace caused startup performance to suffer. This change adopts the solution from the azure sdk, where pkg_resources are only used for dev/editable installs and are stripped out when building wheels.

In order to be able to reuse the azure_bdist_wheel.py wheel extension implementation from the azure sdk, we also introduced an additional namespace package (the name of the namespace package is used to determine which init.py files to exclude when building the wheel)

This checklist is used to make sure that common guidelines for a pull request are followed.

General Guidelines

  • [N/A] The PR has modified HISTORY.rst with an appropriate description of the change (see Modifying change log).

Command Guidelines

  • [N/A] Each command and parameter has a meaningful description.
  • [N/A] Each new command has a test.

(see Authoring Command Modules)

…ing pylint on azure-cli-nspkg namespace package
…same azure_bdist_wheel.py file that the azure sdk is using
… file as the rest of the azure sdk. This required adding an azure-cli-command_modules-nspkg package since the sdk version uses the name of the package as the path in which to look for/exclude __init__.py files.
…i-command-module-nspkg since that makes setuptools happier.
Copy link
Member

@lmazuel lmazuel left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Some comments I think are important

@@ -47,7 +47,7 @@ def build_package(path_to_package, dist_dir):
print_heading('Building {}'.format(path_to_package))
path_to_setup = os.path.join(path_to_package, 'setup.py')
set_version(path_to_setup)
cmd_success = exec_command('python setup.py sdist -d {0} bdist_wheel -d {0}'.format(dist_dir), cwd=path_to_package)
cmd_success = exec_command('python setup.py bdist_wheel -d {0}'.format(dist_dir), cwd=path_to_package)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You can still do both if you need

zip_safe=False,
classifiers=CLASSIFIERS,
install_requires=[
'azure-cli-nspkg>=1.0.0'
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

should be 3.0.0

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yep. Done.

'azure.cli.core',
'azure.cli.core.commands',
'azure.cli.core.extensions',
'azure.cli.core.sdk',
'azure.cli.core.profiles',
],
install_requires=DEPENDENCIES
install_requires=DEPENDENCIES,
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I can't comment upper, but if you want setuptools to work out of the box, you should remove "azure-cli-nspkg" as a direct dependencies of the package. This way the "setuptools" install does not install it, and azure_bdist_wheel will add it automatically to the wheel.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yep - thanks, missed that. Fixed!

- set correct version requirement for azure-cli-nspkg in the command_modules nspkg setup.py
- Remove hard-coded dependency on azure cli nspkg from azure cli core and let the bdist_wheel extension added it required.
@codecov-io
Copy link

Codecov Report

Merging #2951 into master will increase coverage by 0.06%.
The diff coverage is n/a.

Impacted file tree graph

@@            Coverage Diff             @@
##           master    #2951      +/-   ##
==========================================
+ Coverage   63.33%   63.39%   +0.06%     
==========================================
  Files         466      463       -3     
  Lines       26539    26579      +40     
  Branches     4069     4081      +12     
==========================================
+ Hits        16809    16851      +42     
+ Misses       8617     8612       -5     
- Partials     1113     1116       +3
Impacted Files Coverage Δ
src/azure-cli-core/azure/cli/core/__init__.py 100% <ø> (ø) ⬆️
...li-redis/azure/cli/command_modules/redis/custom.py 32.43% <0%> (-7.57%) ⬇️
...c/azure-cli-testsdk/azure/cli/testsdk/preparers.py 68.93% <0%> (-0.16%) ⬇️
...rc/azure-cli-testsdk/azure/cli/testsdk/__init__.py 100% <0%> (ø) ⬆️
...-redis/azure/cli/command_modules/redis/commands.py 100% <0%> (ø) ⬆️
...ault/azure/cli/command_modules/keyvault/_params.py 86.15% <0%> (+0.21%) ⬆️
...vault/azure/cli/command_modules/keyvault/custom.py 67.73% <0%> (+0.35%) ⬆️
...-cli-role/azure/cli/command_modules/role/custom.py 33.1% <0%> (+1.14%) ⬆️
...li-role/azure/cli/command_modules/role/commands.py 86.66% <0%> (+2.22%) ⬆️
src/azure-cli-testsdk/azure/cli/testsdk/base.py 68.42% <0%> (+2.51%) ⬆️
... and 3 more

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 86c5414...4f215ff. Read the comment docs.

@lmazuel
Copy link
Member

lmazuel commented Apr 25, 2017

@johanste LGTM but didn't clone the branch. If you want I can clone it tomorrow and build to check everything is fine

@johanste
Copy link
Member Author

@lmazuel, we would very much appreciate you cloning & building. We do intend to merge today.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

6 participants