Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Bump hass-nabucasa to avoid the performance penalty loading ecdsa #38056

Merged

Conversation

bdraco
Copy link
Member

@bdraco bdraco commented Jul 22, 2020

Proposed change

The ecdsa module which is used by jose via cloud does it calculations
in pure python. The latest python-jose release no longer loads ecdsa
if cryptography is available.

This shaved another second off of startup time.

On a basic install:

Home Assistant initialized in 7.93s -> Home Assistant initialized in 6.82s

Type of change

  • Dependency upgrade
  • Bugfix (non-breaking change which fixes an issue)
  • New integration (thank you!)
  • New feature (which adds functionality to an existing integration)
  • Breaking change (fix/feature causing existing functionality to break)
  • Code quality improvements to existing code or addition of tests

Example entry for configuration.yaml:

# Example configuration.yaml

Additional information

  • This PR fixes or closes issue: fixes #
  • This PR is related to issue:
  • Link to documentation pull request:

Checklist

  • The code change is tested and works locally.
  • Local tests pass. Your PR cannot be merged unless tests pass
  • There is no commented out code in this PR.
  • I have followed the development checklist
  • The code has been formatted using Black (black --fast homeassistant tests)
  • Tests have been added to verify that the new code works.

If user exposed functionality or configuration variables are added/changed:

If the code communicates with devices, web services, or third-party tools:

  • The manifest file has all fields filled out correctly.
    Updated and included derived files by running: python3 -m script.hassfest.
  • New or updated dependencies have been added to requirements_all.txt.
    Updated by running python3 -m script.gen_requirements_all.
  • Untested files have been added to .coveragerc.

The integration reached or maintains the following Integration Quality Scale:

  • No score or internal
  • 🥈 Silver
  • 🥇 Gold
  • 🏆 Platinum

@probot-home-assistant probot-home-assistant bot added core small-pr PRs with less than 30 lines. labels Jul 22, 2020
@bdraco
Copy link
Member Author

bdraco commented Jul 22, 2020

apk add gmp-dev mpfr-dev mpc1-dev gcc musl-dev

I assume we need pre-built wheels to make this work?

@balloob balloob requested a review from pvizeli July 22, 2020 06:48
@pvizeli
Copy link
Member

pvizeli commented Jul 22, 2020

We do not use this libs because we use cryptography. The question is, why they there...
https://pypi.org/project/python-jose/

@bdraco
Copy link
Member Author

bdraco commented Jul 22, 2020

It looks like the cryptography backend still uses ecdsa via jose/backends/cryptography_backend.py

try:
    from ecdsa import SigningKey as EcdsaSigningKey, VerifyingKey as EcdsaVerifyingKey
except ImportError:
    EcdsaSigningKey = EcdsaVerifyingKey = None
  File "/usr/src/homeassistant/homeassistant/components/cloud/__init__.py", line 4, in <module>
    from hass_nabucasa import Cloud
  File "/usr/local/lib/python3.8/site-packages/hass_nabucasa/__init__.py", line 12, in <module>
    from jose import jwt
  File "/usr/local/lib/python3.8/site-packages/jose/jwt.py", line 13, in <module>
    from jose import jws
  File "/usr/local/lib/python3.8/site-packages/jose/jws.py", line 11, in <module>
    from jose import jwk
  File "/usr/local/lib/python3.8/site-packages/jose/jwk.py", line 10, in <module>
    from jose.backends.base import Key
  File "/usr/local/lib/python3.8/site-packages/jose/backends/__init__.py", line 3, in <module>
    from jose.backends.cryptography_backend import CryptographyRSAKey as RSAKey  # noqa: F401
  File "/usr/local/lib/python3.8/site-packages/jose/backends/cryptography_backend.py", line 8, in <module>
    from ecdsa import SigningKey as EcdsaSigningKey, VerifyingKey as EcdsaVerifyingKey
  File "/usr/local/lib/python3.8/site-packages/ecdsa/__init__.py", line 1, in <module>
    from .keys import SigningKey, VerifyingKey, BadSignatureError, BadDigestError,\
  File "/usr/local/lib/python3.8/site-packages/ecdsa/keys.py", line 72, in <module>
    from . import ecdsa
  File "/usr/local/lib/python3.8/site-packages/ecdsa/ecdsa.py", line 61, in <module>

@bdraco
Copy link
Member Author

bdraco commented Jul 22, 2020

Seems better to fix this upstream.

mpdavis/python-jose#178

@bdraco bdraco marked this pull request as draft July 22, 2020 17:53
@bdraco bdraco added the waiting-for-upstream We're waiting for a change upstream label Jul 22, 2020
@pvizeli
Copy link
Member

pvizeli commented Jul 22, 2020

Yeah, cryptography is allready high optimize and running in C with openssl with is better optimize as ecdsa with the gmp helper.

@bdraco bdraco changed the title Improve home assistant restart time by calculating ecdsa curves in c Bump python-jose to avoid the performance penalty loading ecdsa Jul 30, 2020
@bdraco bdraco changed the title Bump python-jose to avoid the performance penalty loading ecdsa Bump hass-nabucasa to avoid the performance penalty loading ecdsa (requires change in pycognito first) Jul 30, 2020
@bdraco
Copy link
Member Author

bdraco commented Jul 30, 2020

@bdraco
Copy link
Member Author

bdraco commented Jul 30, 2020

Just the hass-nabucasa change left, and this can move forward.

@bdraco bdraco changed the title Bump hass-nabucasa to avoid the performance penalty loading ecdsa (requires change in pycognito first) Bump hass-nabucasa to avoid the performance penalty loading ecdsa Jul 31, 2020
@bdraco bdraco added dependency-bump and removed waiting-for-upstream We're waiting for a change upstream labels Jul 31, 2020
@bdraco bdraco marked this pull request as ready for review July 31, 2020 18:34
@bdraco bdraco force-pushed the speed_upstartup_with_ecdsa_calcs_in_c_code branch from 9a60a42 to 62ffe70 Compare July 31, 2020 18:35
@balloob balloob merged commit 2f1d989 into home-assistant:dev Aug 3, 2020
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants