fix(deps): OTP 27.1.2 → 27.3.4.14 — ssl session-ticket VM crash (#966)#974
Merged
Conversation
Bump the release/CI toolchain to OTP 27.3.4.14 (ssl 11.2.12.10), whose release notes fix the exact prod crash class from the Jul 3 incident: tls_client_ticket_store crashing on TLS 1.3 session tickets in auto mode (OTP-20232, erlang/otp PR-11311), plus ticket/session store recovery resilience (OTP-20216) and the 2026-06/07 ssl CVE fixes. Also harden rel/env.sh.eex: retry when hostname -I is transiently empty during boot, then fail loudly instead of booting undistributed forever (the #966 secondary anomaly). Closes #966 Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Rasbandit
enabled auto-merge (squash)
July 8, 2026 23:54
Member
Author
|
e2e-clerk + e2e-crdt failures on the first run are known pre-existing flake classes, not the OTP bump: test_34 old-paths-cleaned is open flake #972 (3rd occurrence in 24h, commented there); crdt test_concurrent_edits_both_survive was a one-direction 30s delivery-latency timeout (delivery-flake family — 8/9 crdt + 88/89 clerk passed, stack booted clean on the new image, prebuild green). CI TLS is not in the data path so the ssl bump can't affect delivery timing. Rerunning failed jobs. |
Rasbandit
added a commit
that referenced
this pull request
Jul 9, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What
Bumps the release/CI toolchain from OTP 27.1.2 to OTP 27.3.4.14 (ssl 11.2.12.10) and hardens
rel/env.sh.eexagainst the undistributed-boot anomaly. Closes #966.Why: the Jul 3 incident
Prod (OTP 27.1.2) hit repeated
GenServer :tls_client_ticket_store terminating(TryClauseError)crashes on TLS 1.3 session tickets fromkms.us-east-1.amazonaws.com(bursts 05:02:50 / 05:37 / 06:30 UTC). The store is a singleton under thesslapp supervisor; restart-intensity exhaustion took the permanentsslapp down → VM exit → ~2h of ECS task churn (05:05-07:10 UTC), zero graceful-drain lines in Loki. Full evidence trail in #966.The fix is in what we ship
OTP 27.3.4.14 / ssl 11.2.12.10 release notes (SSL Release Notes, OTP 27):
That is exactly our crash class: client-side
session_ticketsauto-mode ticket store crashing under load. The same release also ships:so even a future store bug no longer cascades into an ssl-app takedown. Note the OTP-20232 fix landed only in 27.3.4.14 — no earlier 27.x patch carries it — and the bump also picks up the 2026-06/07 ssl CVE fixes (CVE-2026-54887, CVE-2026-54891, CVE-2026-55950, CVE-2026-55952).
Verified the shipped toolchain directly:
Pins changed (Elixir stays 1.17.3)
Dockerfile—OTP_VERSION27.1.2 → 27.3.4.14,DEBIAN_VERSIONbookworm-20241202-slim → bookworm-20260623-slim (the newer erlang tag only exists on the newer snapshot;debian:bookworm-20260623-slimrunner tag verified active on Docker Hub).github/workflows/verify.yml—MIX_BUILDER_IMAGEci/compose.yml,ci/compose.database.yml—BUILDER_IMAGE.github/setup-runner.sh—ELIXIR_IMAGEdocs/context/ci-mix-compile-cache-runner-path.md— present-tense pin referenceSecondary:
rel/env.sh.eexundistributed-boot anomaly (#966)Three tasks in the churn window booted undistributed (
hostname -Iplausibly returned nothing very early under churn pressure → empty_eni_ipv4→ bogusRELEASE_NODE, node falls back to undistributed). Now: retry up to 5×1s, then fail loudly (exit 1) instead of booting blind to clustering forever — a failed boot gets replaced by ECS; an undistributed boot is invisible.Notes for reviewers
mix compile --warnings-as-errors,mix format --check-formatted,mix credo --strict(0 issues),mix sobelow, fullmix test.🤖 Generated with Claude Code