Repository navigation
fix(sse): auto-calibrate heap-pressure threshold to the V8 heap ceiling - #3052
Conversation
The chatCore memory-pressure guard rejected requests with a 503 "Service temporarily unavailable due to resource pressure" once heapUsed passed a hardcoded 200MB. The app's own working set is ~260MB, so on the default 512MB heap — and on the no-cap PM2 deploy (heap ceiling ~2240MB) — the guard 503'd EVERY request after warmup, a full outage. Derive the threshold from the live V8 heap ceiling (v8.getHeapStatistics().heap_size_limit) at 85%, floored at 400MB, so it tracks --max-old-space-size across 1GB / 2GB / large VPS and always clears the runtime baseline. An explicit HEAP_PRESSURE_THRESHOLD_MB still overrides. - open-sse/utils/heapPressure.ts: pure computeHeapPressureThresholdMb + resolved constant - chatCore.ts: import the constant instead of the inline parse-or-200 - .env.example: document the auto-calibration + per-tier OMNIROUTE_MEMORY_MB guidance - tests: heap-pressure.test.ts (1GB/2GB/no-cap/floor/override matrix) + replace the obsolete "defaults to 200" assertion with one tied to the live production constant
There was a problem hiding this comment.
Code Review
This pull request replaces the fixed 200MB heap-pressure threshold with an auto-calibrating mechanism that dynamically sets the threshold to 85% of the V8 heap ceiling (with a 400MB floor), preventing premature 503 rejections. The changes include a new utility 'heapPressure.ts', updates to 'chatCore.ts', and corresponding unit tests. The review feedback correctly points out that using 'Number(override)' to parse the environment variable is less robust than 'parseInt' because it fails on values with unit suffixes like '512mb', and suggests a backward-compatible fix.
Important
The consumer version of Gemini Code Assist on GitHub is being sunset. Starting June 18, 2026, new organization installations will be blocked, and all code review activity will officially cease on July 17, 2026.
For more details on the timeline and next steps, please review the Help Documentation.
| const explicit = Number(override); | ||
| if (Number.isFinite(explicit) && explicit > 0) return Math.floor(explicit); |
There was a problem hiding this comment.
Using Number(override) to parse the environment variable is less robust than parseInt and changes the previous behavior in chatCore.ts (which used parseInt). Specifically, Number will fail (returning NaN and falling back to auto-calibration) on values with common unit suffixes like "400MB" or "512mb". Using parseInt with a fallback to typeof override === 'number' maintains backward compatibility and gracefully handles trailing non-numeric characters.
| const explicit = Number(override); | |
| if (Number.isFinite(explicit) && explicit > 0) return Math.floor(explicit); | |
| const explicit = typeof override === "number" ? override : parseInt(String(override || ""), 10); | |
| if (Number.isFinite(explicit) && explicit > 0) return Math.floor(explicit); |
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: a514420ce9
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
| // "guard never fires" rather than "guard rejects everything". | ||
| const SHED_RATIO = 0.85; | ||
| const FLOOR_MB = 400; | ||
| return Math.max(Math.round(heapSizeLimitMb * SHED_RATIO), FLOOR_MB); |
There was a problem hiding this comment.
Keep the auto threshold below the heap ceiling
For supported low-memory settings, this floor puts the shed threshold above the process's actual V8 ceiling: the launchers accept OMNIROUTE_MEMORY_MB down to 64 MB (bin/cli/commands/serve.mjs and scripts/build/runtime-env.mjs), and e.g. --max-old-space-size=128 reports a heap limit of about 320 MB. In that case computeHeapPressureThresholdMb() returns 400, so chatCore's heapUsed > HEAP_PRESSURE_THRESHOLD_MB check cannot trip before V8 reaches its own limit and hard-OOMs, regressing the guard specifically for low-memory deployments.
Useful? React with 👍 / 👎.
…add contributor hall Audited all 687 commits / 60 release/v3.8.8 PRs since v3.8.7 against the CHANGELOG: - Added 5 missing Fixed entries: #3052 (heap-pressure auto-calibration), #3051/#3048 (proxy fail-closed + registry assignments, @terence71-glitch), #3049/#3046 (session-pool fingerprint rotation + claude-web cf_clearance, @oyi77). - Credited previously-uncredited contributors: @branben (#2958 scope fix, #2959 Notion context source) and @JxnLexn (per-API-key stream default mode). - Added an Added entry for the per-API-key stream default mode feature. - Added the "🏆 Contributors" hall (24 contributors), matching the v3.8.6 format. Maintainer fix-PRs (#2966–#3030) are intentionally referenced by their original issue numbers in the body rather than the fix-PR number; @diegosouzapw is in the hall.
…ng (cheaper-inference#3052) Threshold now derives from the live V8 heap ceiling (85%, floor 400MB) instead of a fixed 200MB that sat below the ~260MB app baseline and 503'd every request. Tracks --max-old-space-size across 1GB/2GB/large VPS.
…add contributor hall Audited all 687 commits / 60 release/v3.8.8 PRs since v3.8.7 against the CHANGELOG: - Added 5 missing Fixed entries: cheaper-inference#3052 (heap-pressure auto-calibration), cheaper-inference#3051/cheaper-inference#3048 (proxy fail-closed + registry assignments, @terence71-glitch), cheaper-inference#3049/cheaper-inference#3046 (session-pool fingerprint rotation + claude-web cf_clearance, @oyi77). - Credited previously-uncredited contributors: @branben (cheaper-inference#2958 scope fix, cheaper-inference#2959 Notion context source) and @JxnLexn (per-API-key stream default mode). - Added an Added entry for the per-API-key stream default mode feature. - Added the "🏆 Contributors" hall (24 contributors), matching the v3.8.6 format. Maintainer fix-PRs (cheaper-inference#2966–cheaper-inference#3030) are intentionally referenced by their original issue numbers in the body rather than the fix-PR number; @diegosouzapw is in the hall.
…ng (cheaper-inference#3052) Threshold now derives from the live V8 heap ceiling (85%, floor 400MB) instead of a fixed 200MB that sat below the ~260MB app baseline and 503'd every request. Tracks --max-old-space-size across 1GB/2GB/large VPS.
…add contributor hall Audited all 687 commits / 60 release/v3.8.8 PRs since v3.8.7 against the CHANGELOG: - Added 5 missing Fixed entries: cheaper-inference#3052 (heap-pressure auto-calibration), cheaper-inference#3051/cheaper-inference#3048 (proxy fail-closed + registry assignments, @terence71-glitch), cheaper-inference#3049/cheaper-inference#3046 (session-pool fingerprint rotation + claude-web cf_clearance, @oyi77). - Credited previously-uncredited contributors: @branben (cheaper-inference#2958 scope fix, cheaper-inference#2959 Notion context source) and @JxnLexn (per-API-key stream default mode). - Added an Added entry for the per-API-key stream default mode feature. - Added the "🏆 Contributors" hall (24 contributors), matching the v3.8.6 format. Maintainer fix-PRs (cheaper-inference#2966–cheaper-inference#3030) are intentionally referenced by their original issue numbers in the body rather than the fix-PR number; @diegosouzapw is in the hall.
…ng (cheaper-inference#3052) Threshold now derives from the live V8 heap ceiling (85%, floor 400MB) instead of a fixed 200MB that sat below the ~260MB app baseline and 503'd every request. Tracks --max-old-space-size across 1GB/2GB/large VPS.
…add contributor hall Audited all 687 commits / 60 release/v3.8.8 PRs since v3.8.7 against the CHANGELOG: - Added 5 missing Fixed entries: cheaper-inference#3052 (heap-pressure auto-calibration), cheaper-inference#3051/cheaper-inference#3048 (proxy fail-closed + registry assignments, @terence71-glitch), cheaper-inference#3049/cheaper-inference#3046 (session-pool fingerprint rotation + claude-web cf_clearance, @oyi77). - Credited previously-uncredited contributors: @branben (cheaper-inference#2958 scope fix, cheaper-inference#2959 Notion context source) and @JxnLexn (per-API-key stream default mode). - Added an Added entry for the per-API-key stream default mode feature. - Added the "🏆 Contributors" hall (24 contributors), matching the v3.8.6 format. Maintainer fix-PRs (cheaper-inference#2966–cheaper-inference#3030) are intentionally referenced by their original issue numbers in the body rather than the fix-PR number; @diegosouzapw is in the hall.
Problem
The chatCore memory-pressure guard returned
503 Service temporarily unavailable due to resource pressureonceheapUsedpassed a hardcoded 200MB. The app's own working set is ~260MB, so on the default 512MB heap — and on the no-cap PM2 deploy (V8 ceiling ~2240MB) — the guard rejected every request after warmup. Observed live on the Local VPS: all connections failing, surfaced to clients as[anthropic/claude-haiku-4.5] Service temporarily unavailable....Fix
Derive the threshold from the live V8 heap ceiling (
v8.getHeapStatistics().heap_size_limit) at 85%, floored at 400MB, so it tracks--max-old-space-sizeacross 1GB / 2GB / large VPS and always clears the runtime baseline. ExplicitHEAP_PRESSURE_THRESHOLD_MBstill overrides.Changes
open-sse/utils/heapPressure.ts— purecomputeHeapPressureThresholdMb+ resolved constantchatCore.ts— import the constant instead of the inline parse-or-200.env.example— document auto-calibration + per-tierOMNIROUTE_MEMORY_MBguidance (incl. the container/cgroup caveat)heap-pressure.test.ts(full matrix) + replace the obsolete "defaults to 200" assertion with one tied to the live production constant13/13 heap tests pass; typecheck / lint / cycles clean.