Skip to content

Fix AES-GCM output when OPENSSL_ia32cap masks AES-NI but not VAES - #32129

Open
robobun wants to merge 3 commits into
mainfrom
farm/29623eb3/fix-gcm-vaes-dispatch
Open

robobun wants to merge 3 commits into
mainfrom
farm/29623eb3/fix-gcm-vaes-dispatch

Conversation

@robobun

@robobun robobun commented Jun 11, 2026 •

Copy link
Copy Markdown
Collaborator

Fixes #32126

Repro

On any x86-64 CPU with VAES (the reporter's i5-13420H, or any recent Intel/AMD), with the issue's NIST-test-vector script:

OPENSSL_ia32cap='~0x200000000000000' bun repro.js

produces wrong AES-GCM output on every released build (reproduced with bun-linux-x64 1.3.14 and current main, same bytes as the Windows report):

encrypted: e2f9ddf26034baa3fe6dd790aa5fa47f   (expected 0388dace60b6a392f328c2b971b2fe78)
tag:       c22d854b1be1643fbf3294953f75a6b4   (expected ab6e47d42cec13bdf53a67b21257bddf)

decipher.final() does not throw: decrypting the authentic NIST ciphertext yields garbage whose tag still verifies.

Cause

~0x200000000000000 clears AES-NI (CPUID.1:ECX bit 25) from the first OPENSSL_ia32cap word but leaves VAES/VPCLMULQDQ (CPUID.7:ECX bits 9-10) set. In vendored BoringSSL, crypto/fipsmodule/aes/gcm.cc.inc CRYPTO_gcm128_init_aes_key:

  1. aes_ctr_set_key sees hwaes_capable() false and builds the key schedule with vpaes_set_encrypt_key, which stores round keys in a transformed basis (is_hwaes = 0).
  2. The impl selection then picks gcm_x86_vaes_avx2/avx512 based only on the ghash choice and CRYPTO_is_VAES_capable(), without checking is_hwaes.
  3. The stitched VAES bulk assembly expects the standard aes_hw_set_encrypt_key round-key layout, so it encrypts with a misinterpreted key schedule: deterministic garbage ciphertext. H and EK0 still come from the consistent vpaes block function, so tags verify over the garbage.

The aarch64 branches directly below gate on is_hwaes; the two x86-64 VAES branches are missing the same gate. Upstream BoringSSL has since fixed this the same way in 997bafbf, after the reporter filed it upstream. The commit Bun pins (upstream 606d3a344, via #32521) predates that fix, so the patch is still required. It only fires when capability bits are split artificially via OPENSSL_ia32cap, but that is exactly what users do to work around CPU errata (e.g. the VAES masking suggested in #32124), and silently wrong crypto is the worst possible failure mode. Both node:crypto cipheriv and WebCrypto AES-GCM funnel through this init.

Node is unaffected because OpenSSL keys its stitched paths off AES-NI.

Fix

patches/boringssl/gcm-vaes-requires-hwaes-key-schedule.patch, applied at dep fetch time via the existing patches mechanism: add is_hwaes to the two x86-64 VAES impl branches, mirroring the aarch64 branches and the upstream fix. Behavior on consistent capability sets is unchanged (real VAES hardware always has AES-NI). When a future BoringSSL bump includes the upstream fix, the patch stops applying and dep fetch fails loudly. That is the signal to delete the patch file.

Verification

New test in test/js/node/crypto/node-crypto.test.js spawns subprocesses with the masks set (the env var is read once at startup) and checks GCM spec test vectors for AES-128/192/256-GCM, the decrypt-of-authentic-ciphertext case, and that multi-block output is identical across masked/unmasked runs. On CPUs without VAES the masked runs dispatch identically and pass trivially, so it is safe on all CI lanes.

  • USE_SYSTEM_BUN=1 bun test ... -t "AES-GCM stays correct": fails with the exact bytes from the issue (all three key sizes corrupted)
  • bun bd test test/js/node/crypto/node-crypto.test.js: all pass (213 after the rebase onto the BoringSSL 606d3a344 bump, which still lacks the upstream fix; the patch applies cleanly to the new pin)
  • test/js/web/crypto/web-crypto.test.ts under the mask: 10 pass
  • The issue's repro script under bun bd with the mask now matches the NIST vector

Note for reviewers: the production change lives in patches/ + scripts/build/deps/boringssl.ts (vendored dep patch), not src/, so fail-before verification means running the test against a build without the patch (or any released bun).


[stamp-90s] gate passed · iteration 6 · 3 files touched

passes on PR (with fix)
Test-only change.

Debug/ASAN (expected pass):
$ bun bd test 'test/js/node/crypto/node-crypto.test.js'
$ BUN_DEBUG_QUIET_LOGS=1 bun scripts/build.ts --profile=debug --quiet test test/js/node/crypto/node-crypto.test.js
bun test v1.4.1 (65362b53b)

test/js/node/crypto/node-crypto.test.js:
(pass) crypto.randomBytes should return a Buffer [4.38ms]
(pass) crypto.randomInt should return a number [2.10ms]
(pass) crypto.randomInt with no arguments [3.61ms]
(pass) crypto.randomInt with one argument [3.12ms]
(pass) crypto.randomInt with a callback [36.89ms]
(pass) createHash > rsa-md5 - "Hello World" [8.91ms]
(pass) createHash > rsa-md5 - "Hello World" -> binary [4.39ms]
(pass) createHash > rsa-ripemd160 - "Hello World" [0.84ms]
(pass) createHash > rsa-ripemd160 - "Hello World" -> binary [0.82ms]
(pass) createHash > rsa-sha1 - "Hello World" [7.93ms]
(pass) createHash > rsa-sha1 - "Hello World" -> binary [1.29ms]
(pass) createHash > rsa-sha1-2 - "Hello World" [0.60ms]
(pass) createHash > rsa-sha1-2 - "Hello World" -> binary [0.65ms]
(pass) createHash > rsa-sha224 - "Hello World" [2.27ms]
(pass) createHash > rsa-sha224 - "Hello World" -> binary [0.83ms]
(pass) createHash > rsa-sha256 - "Hello World" [1.52ms]
(pass) createHash > rsa-sha256 - "Hello World" -> binary [0.59ms]
(pass) createHash > rsa-sha3-224 - "Hello World" [0.59ms]
(pass) createHash > rsa-sha3-224 - "Hello World" -> binary [0.54ms]
(pass) createHash > rsa-sha3-256 - "Hello World" [0.55ms]
(pass) createHash > rsa-sha3-256 - "Hello World" -> binary [1.29ms]
(pass) createHash > rsa-sha3-384 - "Hello World" [0.61ms]
(pass) createHash > rsa-sha3-384 - "Hello World" -> binary [0.78ms]
(pass) createHash > rsa-sha3-512 - "Hello World" [0.55ms]
(pass) createHash > rsa-sha3-512 - "Hello World" -> binary [0.58ms]
(pass) createHash > rsa-sha384 - "Hello World" [1.61ms]
(pass) createHash > rsa-sha384 - "Hello World" -> binary [0.82ms]
(pass) createHash > rsa-sha512 - "Hello World" [3.05ms]
(pass) createHash > rsa-sha512 - "Hello World" -> binary [0.94ms]
(pass) createHash > rsa-sha512/224 - "Hello World" [0.58m
... (truncated)
Exit: 0
diff hotspot
.../gcm-vaes-requires-hwaes-key-schedule.patch     | 41 ++++++++++++
 scripts/build/deps/boringssl.ts                    | 10 +++
 test/js/node/crypto/node-crypto.test.js            | 73 ++++++++++++++++++++++
 3 files changed, 124 insertions(+)

gate history · 2 passed · 0 rejected · iteration 6

evidence per changed file
file                                                      reads  edits  tests
…es/boringssl/gcm-vaes-requires-hwaes-key-schedule.patch      1      2     10
scripts/build/deps/boringssl.ts                               2      2     10
test/js/node/crypto/node-crypto.test.js                       1      1     10

root cause · written by the author bot

The bug was in the vendored BoringSSL's AES-GCM dispatch, where CRYPTO_gcm128_init_aes_key selected the VAES bulk implementation based solely on CRYPTO_is_VAES_capable(), without confirming that hardware AES was also available. When OPENSSL_ia32cap=~0x200000000000000 masked the AES-NI bit on a VAES-capable CPU, the key schedule was generated by the vpaes fallback, whose layout is incompatible with the VAES assembly, so encryption silently produced incorrect ciphertext and tags. The fix patches the dispatch to additionally require is_hwaes before taking the VAES path, ensuring the VA…

@robobun

robobun commented Jun 11, 2026 •

Copy link
Copy Markdown
Collaborator Author
Updated 1:22 AM PT - Aug 28th, 2026

❌ @robobun, your commit e173a17 has 1 failures in Build #107511 (All Failures):


🧪   To try this PR locally:

bunx bun-pr 32129

That installs a local version of the PR into your bun-32129 executable, so you can run:

bun-32129 --bun

@coderabbitai

coderabbitai Bot commented Jun 11, 2026 •

Copy link
Copy Markdown
Contributor

Review Change Stack

Walkthrough

This PR fixes an AES-GCM correctness bug in BoringSSL where VAES (AVX-512 Vector AES) implementations could silently produce incorrect ciphertext. The fix gates VAES selection on hardware AES support, a patch is wired into the build, and a regression test validates the fix across different CPU capability masks.

Changes

AES-GCM VAES capability-gating fix and regression test

Layer / File(s) Summary
BoringSSL VAES-GCM capability gating fix
patches/boringssl/gcm-vaes-requires-hwaes-key-schedule.patch
Patch fixes CRYPTO_gcm128_init_aes_key to require is_hwaes alongside CRYPTO_is_VAES_capable() before selecting VAES AES-GCM implementations, preventing VAES bulk code from running with incompatible vpaes-based key schedules.
Build configuration patch integration
scripts/build/deps/boringssl.ts
Boringssl dependency now includes the VAES-GCM patch in its patches array to apply the fix to vendored BoringSSL sources during build.
AES-GCM regression test across CPU capability masks
test/js/node/crypto/node-crypto.test.js
New test spawns child processes with varying OPENSSL_ia32cap masks (unmasked, AES-NI masked, fully masked), validates ciphertext/tag output against NIST vectors, and confirms output consistency across all three configurations.
🚥 Pre-merge checks | ✅ 4
✅ Passed checks (4 passed)
Check name Status Explanation
Linked Issues check ✅ Passed The PR directly addresses issue #32126 by fixing the AES-GCM dispatch bug through a BoringSSL patch that adds is_hwaes gate to x86-64 VAES branches, matching all requirements.
Out of Scope Changes check ✅ Passed All changes are directly scoped to fixing AES-GCM correctness: the BoringSSL patch, build dependency configuration, and regression tests—no out-of-scope modifications present.
Title check ✅ Passed The title clearly and concisely describes the main change: correcting AES-GCM output when OPENSSL_ia32cap masks AES-NI while leaving VAES enabled.
Description check ✅ Passed The description provides a detailed explanation of the bug, root cause, fix, reproduction steps, verification results, and affected files. It uses equivalent sections instead of the exact template hea…
Full details: Description check

Explanation

The description provides a detailed explanation of the bug, root cause, fix, reproduction steps, verification results, and affected files. It uses equivalent sections instead of the exact template headings, but it fully covers the required information.


Comment @coderabbitai help to get the list of available commands.

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 1

🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@test/js/node/crypto/node-crypto.test.js`:
- Around line 879-901: Test must assert the VAES/x64 precondition before
validating masked vs unmasked behavior: detect whether the runner is x64 and
supports VAES (or whether runWithCaps(undefined) reports VAES-capable behavior)
and if not, skip the regression check; modify the test around runWithCaps / the
unmasked/aesniMasked/bothMasked comparison to assert the precondition (e.g.,
require process.arch==='x64' and a VAES capability flag or a runtime check via
runWithCaps result) and call test.skip or return early when the precondition is
not met so the masked-case comparisons only run on hosts where VAES would change
the implementation.
🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: ASSERTIVE

Plan: Pro

Run ID: 7575454a-e6d0-4808-9ed6-667478db307a

📥 Commits

Reviewing files that changed from the base of the PR and between f8723b1 and 6258c88.

📒 Files selected for processing (3)
  • patches/boringssl/gcm-vaes-requires-hwaes-key-schedule.patch
  • scripts/build/deps/boringssl.ts
  • test/js/node/crypto/node-crypto.test.js

Comment thread test/js/node/crypto/node-crypto.test.js
@robobun

robobun commented Jun 11, 2026 •

Copy link
Copy Markdown
Collaborator Author

CI status for the rebased head (e173a17, build 107511): 180 of 181 jobs passed. The one red lane is macOS x64 failing test/js/web/url/url.test.ts ("special-scheme hosts use the Unicode 16 IDNA table", TypeError: Invalid URL). That test is unrelated to this change and fails on main's code path for ICU IDNA tables; it has been reported for main-break triage. The only other blip was a known-flaky inspector-protocol test that passed on retry.

The AES-GCM test and the node:crypto shards passed on every lane. The branch is rebased onto the BoringSSL 606d3a344 bump (#32521), which still predates the upstream fix, and the patch applies cleanly to the new pin. Ready for review.

@Jarred-Sumner

Copy link
Copy Markdown
Collaborator

@robobun what if we upgraded boringssl to the latest upstream version? does it contain a fix?

@robobun

robobun commented Jun 12, 2026

Copy link
Copy Markdown
Collaborator Author

Checked just now against upstream google/boringssl main at 0676117 (committed today, 2026-06-11): no, it does not contain a fix. CRYPTO_gcm128_init_aes_key still selects the VAES implementations on CRYPTO_is_VAES_capable() alone, without checking is_hwaes:

https://github.com/google/boringssl/blob/0676117daa6f86f4c71f25fbc508c1dc595da3e8/crypto/fipsmodule/aes/gcm.cc.inc#L279-L287

The aarch64 branches right below gate on is_hwaes, so this looks like an oversight in the x86-64 branches rather than a design choice. So an upgrade alone would not fix this, and the patch stays necessary either way.

Two things that make the patch cheap to carry across a bump:

  • I verified it applies cleanly to today's upstream HEAD file as well (the surrounding context is unchanged; the only nearby drift is the aarch64 gcm_sha3_capable -> gcm_eor3_capable rename, which is outside the hunk).
  • If upstream fixes it later, the next commit bump will fail the patch application loudly at dep-fetch time, which is the signal to delete the patch file.

Still worth reporting on the BoringSSL tracker so the patch can eventually go away.

@zshs000

zshs000 commented Aug 28, 2026

Copy link
Copy Markdown

@Jarred-Sumner I reported this upstream to BoringSSL/Google, and it has now been fixed:

https://issues.chromium.org/issues/552666770

BoringSSL fix:
https://boringssl-review.googlesource.com/c/boringssl/+/101787

Since Bun statically links its vendored BoringSSL, updating Bun's BoringSSL revision to include this fix (997bafbf73886ec19ac3eb70b31db64fe42832c3) and rebuilding should fix the AES-GCM failure reported here.

@robobun

robobun commented Aug 28, 2026

Copy link
Copy Markdown
Collaborator Author

Thanks for reporting it upstream. I verified the upstream fix (997bafbf, https://boringssl-review.googlesource.com/c/boringssl/+/101787): it is the same change this PR carries, an is_hwaes gate on the two x86-64 VAES branches in CRYPTO_gcm128_init_aes_key.

On taking it via a revision bump instead of this patch: Bun pins the oven-sh/boringssl fork, so picking up 997bafbf means a fork sync that pulls in months of unrelated upstream changes plus a regeneration of the build source lists. That is a much larger change than this bug needs, and it deserves its own PR and full CI run. This PR fixes the bug now with the same one-line-per-branch gate.

The patch header and the dep comment now cite 997bafbf (commit 5fc83f7). When a future BoringSSL bump includes that commit, the patch stops applying and dep-fetch fails loudly, which is the signal to delete the patch file. So the fix cannot silently duplicate or rot.

…VAES

BoringSSL's CRYPTO_gcm128_init_aes_key picked the stitched VAES AES-GCM
implementation (gcm_x86_vaes_avx2/avx512) from CPU capability bits alone,
without checking that the AES key schedule was produced by
aes_hw_set_encrypt_key. When OPENSSL_ia32cap masks AES-NI but leaves VAES
set, the key schedule comes from vpaes (transformed layout) and the VAES
bulk assembly silently produces wrong ciphertext, and decrypts garbage
whose tag still verifies.

Add the same is_hwaes gate the aarch64 branches already use, shipped as a
vendor patch applied at dep fetch time. Upstream BoringSSL main has the
same bug.

Fixes #32126
Upstream fixed the same bug in google/boringssl
997bafbf73886ec19ac3eb70b31db64fe42832c3. Point the patch header and the
dep comment at that commit so the patch is deleted when a future pin bump
includes it.

@claude claude Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I reviewed this PR and didn't find any bugs. Because it patches a vendored crypto path (BoringSSL AES-GCM dispatch), a human look is still worthwhile before merging.

What was reviewed:

  • The patch adds && is_hwaes to both x86-64 VAES branches in gcm.cc.inc, matching the aarch64 branches directly below and upstream BoringSSL 997bafbf7388 — no other behavior touched.
  • The patches: [...] entry in scripts/build/deps/boringssl.ts follows the same mechanism used by other deps; patch application will fail loudly once the pinned commit already contains the fix.
  • The regression test spawns subprocesses (OPENSSL_ia32cap is read once at startup), spreads bunEnv, drains pipes concurrently, and asserts against published NIST GCM test vectors 2/8/14 rather than relying on self-round-trip alone.
Extended reasoning...

Overview

This PR adds a small vendored patch to BoringSSL fixing AES-GCM implementation dispatch on x86-64. The bug: when OPENSSL_ia32cap masks AES-NI but leaves VAES enabled, CRYPTO_gcm128_init_aes_key selected the stitched VAES bulk assembly while the key schedule was built by vpaes (which uses a transformed round-key basis), producing silently wrong ciphertext whose auth tag still verified. The two-line patch adds && is_hwaes to the two x86-64 VAES selection branches, mirroring both the adjacent aarch64 branches and the identical upstream BoringSSL fix (997bafbf7388). Three files changed: the new patch under patches/boringssl/, a patches: [...] registration in scripts/build/deps/boringssl.ts, and a regression test appended to test/js/node/crypto/node-crypto.test.js.

Security risks

The change touches a cryptographic primitive, which is inherently security-sensitive. However, the risk profile is low: the patch is additive (a stricter gate that can only cause fallback to a slower-but-correct implementation), it mirrors upstream verbatim, and the failure mode being fixed is "silently wrong crypto" — the worst possible outcome — so leaving it unfixed is the greater risk. On real hardware with consistent capability bits (VAES implies AES-NI), the added is_hwaes condition is always true, so unmasked runs are unchanged. I did not identify any way this patch could weaken crypto on any input.

Level of scrutiny

High, per the approval guidelines for crypto paths. Even though the diff is small and mechanically matches an accepted upstream commit, changes to AES-GCM dispatch warrant a human maintainer's sign-off rather than automated approval. The patches: mechanism is well-established in this repo (used by 12 other deps), and the comment in boringssl.ts documents the upstream commit and self-obsoleting condition, satisfying the vendoring conventions in CLAUDE.md.

Other factors

The test is well-constructed against the review checklist: it appends to the existing node-crypto.test.js rather than creating a new file, carries the issue-URL comment, spawns fresh subprocesses because OPENSSL_ia32cap is read once at process init, spreads bunEnv, drains stdout/stderr/exited concurrently, and asserts exact known-answer vectors from the McGrew & Viega GCM spec (test cases 2/8/14) plus an independent decrypt of the authentic NIST ciphertext — addressing the "self-round-trip proves nothing for codecs" rule. The multi-block case is compared across all three capability masks. On non-x86-64 or non-VAES CI lanes the masked runs dispatch identically and pass trivially, so the test is safe everywhere. The PR description states the test fails against unpatched builds with the exact bytes from the issue report.

@robobun
robobun force-pushed the farm/29623eb3/fix-gcm-vaes-dispatch branch from 5fc83f7 to e173a17 Compare August 28, 2026 07:25

@claude claude Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code review found no issues

No high-confidence issues detected in this change.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

AES-GCM incorrect on bun-windows-x64 with OPENSSL_ia32cap=~0x200000000000000; baseline and Node are correct

3 participants