crypto: refactor crypto subsystem and introduce WebCrypto API #35093
Conversation
|
Review requested:
|
nodejs-github-bot
added
the
lib / src
jasnell
changed the title
jasnell
added
crypto
jasnell
force-pushed
the
subtlecrypto
branch
14 times, most recently
from
409f56e
to
13f1691
Compare
|
For anyone curious about the performance difference that the Keep in mind, however, that this is tracking raw execution speed, and ignores factors such as event loop delay, event loop utilization and memory. The |
|
@jasnell thank you for this work! I assume the goal with the subtle interface is to 100% cover the specification, i also assume some Node.js specifics are going to be in place (e.g. passing Buffer instances, possibly also KeyObject instances to importKey?, dunno ...). I welcome the addition of built-in hkdf exposed outside of the subtle interfaces. Altho the interface doesn't enable anything new algorithm-wise it's a great way to support modules such as By far the biggest addition of this interface, from my point of view, is its implementation using the libuv threadpool. Being able to run both fast and slow algorithms using the same interface and not having the main thread blocked is finally here. Which brings me to obvious question - what about sign/verify/ecdh/etc operations using either key types or algorithms that are not supported by the webcrypto API specification? Is there going to be a |
|
Yes, the goal is 100% compatibility with the Web Crypto API. The CryptoKey object is a wrapper around the existing KeyObject. Allowing conversion between the two using importKey and exportKey is a good idea. Buffer will be supported but is treated as if it is any other TypedArray. Once the basic algorithms required by the WebCrypto spec are implemented, support for the broader range of algorithms we support in the legacy API will be added, starting with a few we already support such as scrypt, DH (non-ECDH), all of the ciphers, hashes, and curves reported in the getHashes, getCiphers, and getCurves APIs, and all of the keygen types supported by the existing generateKeyPair API. This will take some effort as they will be integrated into the extensible framework already provided by WebCrypto. Anything that does not fit within the WebCrypto API will be exposed directly off the crypto/promises module and not the SubtleCrypto class. After that, I have plans for a few new algorithms we do not currently support in either API. CMAC for instance. And possibly UUID and simple HOTP/TOTP based token generation. Once the changes here are completed, it will be far easier to incorporate such additions. |
|
Do you plan to integrate the WebCryptoAPI web platform tests? |
|
@jasnell amazing! Happy to hear about this roadmap. Let me know if you need assistance with testing. |
I would be very careful with extending WebCrypto with custom algorithms as the proprietary unregistered algs may clash and confuse developers thinking it's part of WebCrypto. I'd suggest to leave WebCrypto implementation inline with the current version of the spec and treat everything unregistered in WebCrypto like so 👇
Ad CMAC > there's this PR that could use wrapping up. Altho from my testing it's not working as intended just yet. |
Absolutely, but that will likely be in a follow on PR. |
Caution will definitely be waranted and any such extension will be in line with the extensibility guidelines of the spec (https://www.w3.org/TR/WebCryptoAPI/#extensibility). Specifically, any Node.js specific algorithms would be prefixed as such when used. For instance... const { subtle } = require('crypto/promises');
const ec = new TextEncoder();
subtle.digest({ name: 'node.shake256', length: 100 }, ec.encode('hello'));For well-known algorithms like Scrypt and CMAC, my intention would be to introduce those initially as node-prefixed algorithms e.g. One alternative that I'm considering is exposing a separate So, for instance, with this alternative, the following would fail: const { subtle } = require('crypto/promises');
const ec = new TextEncoder();
subtle.digest({ name: 'node.shake256', length: 100 }, ec.encode('hello'));But this would work: const { nodeSubtle } = require('crypto/promises');
const ec = new TextEncoder();
nodeSubtle.digest({ name: 'node.shake256', length: 100 }, ec.encode('hello'));In either case, developers will have to explicitly opt in to using the Node.js specific extensions. |
|
@jasnell sounds good |
jasnell
force-pushed
the
subtlecrypto
branch
from
13f1691
to
4e2952e
Compare
Before nodejs/node#35093 this include was explicitly there but it was remove in the refactor. Upstreamed at nodejs/node#38864.
Before nodejs/node#35093 this include was explicitly there but it was remove in the refactor. Upstreamed at nodejs/node#38864.
Before nodejs/node#35093 this include was explicitly there but it was remove in the refactor. Upstreamed at nodejs/node#38864.
Before nodejs/node#35093 this include was explicitly there but it was remove in the refactor. Upstreamed at nodejs/node#38864.
Before nodejs/node#35093 this include was explicitly there but it was remove in the refactor. Upstreamed at nodejs/node#38864.
Before nodejs/node#35093 this include was explicitly there but it was remove in the refactor. Upstreamed at nodejs/node#38864.
Before nodejs/node#35093 this include was explicitly there but it was remove in the refactor. Upstreamed at nodejs/node#38864.
Fixes: #39936 Refs: #35093 PR-URL: #39937 Reviewed-By: Colin Ihrig <[email protected]> Reviewed-By: Filip Skokan <[email protected]> Reviewed-By: James M Snell <[email protected]>
Fixes: #39936 Refs: #35093 PR-URL: #39937 Reviewed-By: Colin Ihrig <[email protected]> Reviewed-By: Filip Skokan <[email protected]> Reviewed-By: James M Snell <[email protected]>
Fixes: #39936 Refs: #35093 PR-URL: #39937 Reviewed-By: Colin Ihrig <[email protected]> Reviewed-By: Filip Skokan <[email protected]> Reviewed-By: James M Snell <[email protected]>
Commit dae283d from August 2020 introduced a call to EntropySource() in SecretKeyGenTraits::DoKeyGen() in src/crypto/crypto_keygen.cc. There are two problems with that: 1. It does not check the return value, it assumes EntropySource() always succeeds, but it can (and sometimes will) fail. 2. The random data returned byEntropySource() may not be cryptographically strong and therefore not suitable as keying material. An example is a freshly booted system or a system without /dev/random or getrandom(2). EntropySource() calls out to openssl's RAND_poll() and RAND_bytes() in a best-effort attempt to obtain random data. OpenSSL has a built-in CSPRNG but that can fail to initialize, in which case it's possible either: 1. No random data gets written to the output buffer, i.e., the output is unmodified, or 2. Weak random data is written. It's theoretically possible for the output to be fully predictable because the CSPRNG starts from a predictable state. Replace EntropySource() and CheckEntropy() with new function CSPRNG() that enforces checking of the return value. Abort on startup when the entropy pool fails to initialize because that makes it too easy to compromise the security of the process. Refs: https://hackerone.com/bugs?report_id=1690000 Refs: nodejs/node#35093 Backport-PR-URL: https://github.com/nodejs-private/node-private/pull/349 PR-URL: https://github.com/nodejs-private/node-private/pull/346 Reviewed-By: Vladimir de Turckheim <[email protected]> Reviewed-By: Tobias Nießen <[email protected]> CVE-ID: CVE-2022-35255
It is confusing to have both ERR_CRYPTO_SCRYPT_INVALID_PARAMETER and ERR_CRYPTO_INVALID_SCRYPT_PARAMS. The former was the original error code, added in 371103d, but parameter validation gradually changed and now produces ERR_CRYPTO_INVALID_SCRYPT_PARAMS for all parameter validation errors coming from OpenSSL, as well as different error codes for validation errors coming from JavaScript. The only remaining use of ERR_CRYPTO_SCRYPT_INVALID_PARAMETER is in the validation logic that ensures that no two synonymous options were passed. We already have an error code for that particular case, ERR_INCOMPATIBLE_OPTION_PAIR, so replace these last instances of ERR_CRYPTO_SCRYPT_INVALID_PARAMETER with that error code and remove ERR_CRYPTO_SCRYPT_INVALID_PARAMETER. If there ever is need again for such an error code, we can just use ERR_CRYPTO_INVALID_SCRYPT_PARAMS. Refs: nodejs#35093 Refs: nodejs#21525 Refs: nodejs#20816
It is confusing to have both ERR_CRYPTO_SCRYPT_INVALID_PARAMETER and ERR_CRYPTO_INVALID_SCRYPT_PARAMS. The former was the original error code, added in 371103d, but parameter validation gradually changed and now produces ERR_CRYPTO_INVALID_SCRYPT_PARAMS for all parameter validation errors coming from OpenSSL, as well as different error codes for validation errors coming from JavaScript. The only remaining use of ERR_CRYPTO_SCRYPT_INVALID_PARAMETER is in the validation logic that ensures that no two synonymous options were passed. We already have an error code for that particular case, ERR_INCOMPATIBLE_OPTION_PAIR, so replace these last instances of ERR_CRYPTO_SCRYPT_INVALID_PARAMETER with that error code and remove ERR_CRYPTO_SCRYPT_INVALID_PARAMETER. If there ever is need again for such an error code, we can just use ERR_CRYPTO_INVALID_SCRYPT_PARAMS. Refs: #35093 Refs: #21525 Refs: #20816 PR-URL: #53305 Reviewed-By: James M Snell <[email protected]> Reviewed-By: Filip Skokan <[email protected]> Reviewed-By: Luigi Pinca <[email protected]> Reviewed-By: Richard Lau <[email protected]> Reviewed-By: Benjamin Gruenbaum <[email protected]>
It is confusing to have both ERR_CRYPTO_SCRYPT_INVALID_PARAMETER and ERR_CRYPTO_INVALID_SCRYPT_PARAMS. The former was the original error code, added in 371103d, but parameter validation gradually changed and now produces ERR_CRYPTO_INVALID_SCRYPT_PARAMS for all parameter validation errors coming from OpenSSL, as well as different error codes for validation errors coming from JavaScript. The only remaining use of ERR_CRYPTO_SCRYPT_INVALID_PARAMETER is in the validation logic that ensures that no two synonymous options were passed. We already have an error code for that particular case, ERR_INCOMPATIBLE_OPTION_PAIR, so replace these last instances of ERR_CRYPTO_SCRYPT_INVALID_PARAMETER with that error code and remove ERR_CRYPTO_SCRYPT_INVALID_PARAMETER. If there ever is need again for such an error code, we can just use ERR_CRYPTO_INVALID_SCRYPT_PARAMS. Refs: nodejs#35093 Refs: nodejs#21525 Refs: nodejs#20816 PR-URL: nodejs#53305 Reviewed-By: James M Snell <[email protected]> Reviewed-By: Filip Skokan <[email protected]> Reviewed-By: Luigi Pinca <[email protected]> Reviewed-By: Richard Lau <[email protected]> Reviewed-By: Benjamin Gruenbaum <[email protected]>
@nodejs/tsc @nodejs/crypto ... This is a big one. It is still a work in progress. I'm opening the draft PR so that people can follow along with the work.
What does this do:
Three main things... lots of other little things
It refactors the Node.js src crypto internals so that they are more maintainable and organized. The existing
node_crypto.cchas grown into a massive disorganized and unmaintainable mess that very few brave touch. This breaks that functionality up across multiple files insrc/cryptothat are organized by purpose/algorithm.It makes a number of important fixes and improvements to the existing crypto internals. For example, previously,
CryptoJobhad no mechanism for tracking the memory associated with it.CryptoJobhas been refactored into anAsyncWrapderived object, allowing and improving memory tracking.It introduces an experimental Web Crypto API implementation as
require('crypto').webcrypto. The initial intent of this API is to be standards compliant.It introduces HKDF support for the legacy API. HKDF is required by the Web Crypto API implementation, so I decided to go ahead and add a variation for the legacy API also.
5. It introducesPulled this due to some outstanding technical concerns that will need to be looked at later.crypto.timingSafeEqual.bigint()for performing constant-time comparisons of bigint values.It introduces the ability to use
ArrayBufferfor the existing legacy crypto APIs. Previously, the legacy API was restricted toBuffer,TypedArray, andDataViewobjects. Because the Web Crypto API makes use ofArrayBufferalso, I decided to go ahead and extend that capability to the legacy API. (Note: this is still a work in progress that will be completed before this PR moves out of draft status)It introduces one extension to the Web Crypto API
exportKey()andimportKey()methods to allow converting back and forth between a Node.jsKeyObjectand a Web CryptoCryptoKeyobject. TheCryptoKeyis currently implemented as a wrapper aroundKeyObject.My next task on this PR is to begin filling out the tests. This PR will remain in draft status until I have those ready. However, please feel free to begin reviewing the changes made so far.
This is a big PR that will take some time to review. I'm happy to jump on a zoom call with anyone to walk through it.
Outside of changes to error messages and codes (which have been made more consistent), there should be no backwards compatibility breaking changes to the existing legacy API. Any backwards breaking changes should be considered bugs.
Checklist
make -j4 test(UNIX), orvcbuild test(Windows) passes