Cyptography pr 20566 rebase (ansible#25560)

Make pyca/cryptography the preferred backend for cryptographic needs (mainly vault) falling back to pycrypto

pyca/cryptography is already implicitly a dependency in many cases
through paramiko (2.0+) as well as the new openssl_publickey module,
which requires pyOpenSSL 16.0+. Additionally, pyca/cryptography is
an optional dep for better performance with vault already.

This commit leverages cryptography's padding, constant time comparisons,
and CBC/CTR modes to reduce the amount of code ansible needs to
maintain.

* Handle wrong password given for VaultAES format

* Do not display deprecation warning for cryptography on python-2.6

* Namespace all of the pycrypto imports and always import them

  Makes unittests better and the code less likely to get stupid mistakes
  (like using HMAC from cryptogrpahy when the one from pycrypto is needed)

* Add back in atfork since we need pycrypto to reinitialize its RNG just in case we're being used with old paramiko

* contrib/inventory/gce: Remove spurious require on pycrypto

(cherry picked from commit 9e16b9d)

* Add cryptography to ec2_win_password module requirements
  * Fix python3 bug which would pass text strings to a function which
    requires byte strings.

* Attempt to add pycrypto version to setup deps

* Change hacking README for dual pycrypto/cryptography

* update dependencies for various CI scripts

* additional CI dockerfile/script updates

* add paramiko to the windows and sanity requirement set

  This is needed because ansible lists it as a requirement. Previously
  the missing dep wasn't enforced, but cryptography imports pkg_resources
  so you can't ignore a requirement any more

* Add integration test cases for old vault and for wrong passwords

* helper script for manual testing of pycrypto/cryptography

* Skip the pycrypto tests so that users without it installed can still run the unittests

* Run unittests for vault with both cryptography and pycrypto backend

Author: abadger

CHANGELOG.md

@@ -68,6 +68,8 @@ Ansible Changes By Release
 * Experimentally added pmrun become method.
 * Enable the docker connection plugin to use su as a become method
 * Add an encoding parameter for the replace module so that it can operate on non-utf-8 files
+* By default, Ansible now uses the cryptography module to implement vault
+  instead of the older pycrypto module.
 
 #### New Callbacks:
 - profile_roles
@@ -92,6 +94,7 @@ Ansible Changes By Release
 
 - The docker_container module has gained a new option, working_dir which allows
   specifying the working directory for the command being run in the image.
+- The ec2_win_password module now requires the cryptography python module be installed to run
 
 ### New Modules
 

contrib/inventory/gce.py

@@ -74,7 +74,6 @@
 Version: 0.0.3
 '''
 
-__requires__ = ['pycrypto>=2.6']
 try:
     import pkg_resources
 except ImportError:

hacking/README.md

@@ -17,7 +17,7 @@ and do not wish to install them from your operating system package manager, you
 can install them from pip
 
     $ easy_install pip               # if pip is not already available
-    $ pip install pyyaml jinja2 nose pytest passlib pycrypto
+    $ pip install -r requirements.txt
 
 From there, follow ansible instructions on docs.ansible.com as normal.
 

lib/ansible/executor/process/worker.py

@@ -26,13 +26,15 @@
 
 from jinja2.exceptions import TemplateNotFound
 
-# TODO: not needed if we use the cryptography library with its default RNG
-# engine
-HAS_ATFORK = True
+HAS_PYCRYPTO_ATFORK = False
 try:
     from Crypto.Random import atfork
-except ImportError:
-    HAS_ATFORK = False
+    HAS_PYCRYPTO_ATFORK = True
+except:
+    # We only need to call atfork if pycrypto is used because it will need to
+    # reinitialize its RNG.  Since old paramiko could be using pycrypto, we
+    # need to take charge of calling it.
+    pass
 
 from ansible.errors import AnsibleConnectionFailure
 from ansible.executor.task_executor import TaskExecutor
@@ -99,7 +101,7 @@ def run(self):
         # pr = cProfile.Profile()
         # pr.enable()
 
-        if HAS_ATFORK:
+        if HAS_PYCRYPTO_ATFORK:
             atfork()
 
         try:

lib/ansible/modules/cloud/amazon/ec2_win_password.py

@@ -60,6 +60,13 @@
 extends_documentation_fragment:
     - aws
     - ec2
+
+requirements:
+    - cryptography
+
+notes:
+    - As of Ansible 2.4, this module requires the python cryptography module rather than the
+      older pycrypto module.
 '''
 
 EXAMPLES = '''
@@ -95,16 +102,21 @@
 '''
 
 from base64 import b64decode
-from Crypto.Cipher import PKCS1_v1_5
-from Crypto.PublicKey import RSA
+from os.path import expanduser
 import datetime
+from cryptography.hazmat.backends import default_backend
+from cryptography.hazmat.primitives.asymmetric.padding import PKCS1v15
+from cryptography.hazmat.primitives.serialization import load_pem_private_key
 
 try:
     import boto.ec2
     HAS_BOTO = True
 except ImportError:
     HAS_BOTO = False
 
+BACKEND = default_backend()
+
+
 def main():
     argument_spec = ec2_argument_spec()
     argument_spec.update(dict(
@@ -122,7 +134,7 @@ def main():
 
     instance_id = module.params.get('instance_id')
     key_file = module.params.get('key_file')
-    key_passphrase = module.params.get('key_passphrase')
+    b_key_passphrase = to_bytes(module.params.get('key_passphrase'), errors='surrogate_or_strict')
     wait = module.params.get('wait')
     wait_timeout = int(module.params.get('wait_timeout'))
 
@@ -147,21 +159,18 @@ def main():
         module.fail_json(msg = "wait for password timeout after %d seconds" % wait_timeout)
 
     try:
-        f = open(key_file, 'r')
+        f = open(key_file, 'rb')
     except IOError as e:
         module.fail_json(msg = "I/O error (%d) opening key file: %s" % (e.errno, e.strerror))
     else:
         try:
             with f:
-                key = RSA.importKey(f.read(), key_passphrase)
-        except (ValueError, IndexError, TypeError) as e:
+                key = load_pem_private_key(f.read(), b_key_passphrase, BACKEND)
+        except (ValueError, TypeError) as e:
             module.fail_json(msg = "unable to parse key file")
 
-    cipher = PKCS1_v1_5.new(key)
-    sentinel = 'password decryption failed!!!'
-
     try:
-        decrypted = cipher.decrypt(decoded, sentinel)
+        decrypted = key.decrypt(decoded, PKCS1v15())
     except ValueError as e:
         decrypted = None