Skip to content

fix(admission): derive conservative process-global heavyweight headroom - #11541

Closed
KaspaPulse wants to merge 2 commits into
cheaper-inference:release/v3.8.51from
KaspaPulse:feat/admission-runtime-capacity-headroom-20260825
Closed

KaspaPulse wants to merge 2 commits into
cheaper-inference:release/v3.8.51from
KaspaPulse:feat/admission-runtime-capacity-headroom-20260825

Conversation

@KaspaPulse

@KaspaPulse KaspaPulse commented Aug 25, 2026 •

Copy link
Copy Markdown
Contributor

Summary

  • derive portable process-global heavyweight headroom from the tightest valid V8, RSS/external-memory, Node process, host, cgroup v1, cgroup v2 memory.max, or cgroup v2 memory.high budget
  • reject invalid, contradictory, unlimited, and sentinel telemetry without manufacturing capacity; Windows, macOS, and bare-metal hosts fall back to portable Node/V8/host signals
  • retain no universal session cap: eligible provider supply is only an upper bound and cannot create memory permits
  • preserve feat(resilience): add hierarchical concurrency admission #11493 as the sole global/provider/account generation scheduler
  • charge adaptive admission by weighted lifecycle phase: full preparation cost is reduced to the configured streaming-generation cost when the handler yields its response
  • release the structural heavyweight preparation reservation at that response handoff instead of retaining full request-side cost throughout SSE/model generation
  • make byte-heavy structural preparation consume the same primary-plus-runtime-headroom budget already used by structure-only admission
  • emit event-time, privacy-safe rejection telemetry for primary/headroom activity, calculated capacity, available slots, limiting budget, headroom reason, telemetry availability, queue state, and the opaque HMAC fairness lane
  • preserve immediate pressure contraction, hysteretic time-based recovery, bounded queues, cancellation, exact-once release, hard limits, pressure fuse, Retry-After behavior, and explicit override precedence
  • expose only management-protected, privacy-safe aggregate capacity, budget, telemetry, phase-cost, queue, and limiter fields

Production rejection diagnosis

Primary evidence:

  • request ID: 0b68c753-2fbc-47ee-909c-34f334e012b0
  • structural rejection: queue_timeout
  • rejection log: activeHeavy=1, waiting=0, queuedBytes=0, lane=key_3185d5107cab13ba
  • post-event protected monitoring: telemetry available, calculated total capacity 11, effective healthy headroom 10, available slots 11, shedTotal=1

This was not adaptive weighted saturation. The queue_timeout and activeHeavy fields come from the earlier byte/structure gate, before adaptive weighted provider admission. Source inspection found that structure-only admission tried the primary reservation and then healthy headroom, while the byte-heavy reservation path called acquireHeavyWithin() directly and therefore contended only for the single primary slot. The post-event available slots=11 includes release of the active request after the event; the old rejection log did not carry event-time headroom, so it could not distinguish pressure from this structural omission by itself.

A focused TDD reproduction fixed that ambiguity: with the production-shaped 1 + 10 = 11 structural budget, two independent byte-heavy requests were launched concurrently. Before the correction, the second request shed while only activeHeavy=1; after the correction, it acquires one healthy-headroom reservation, reports activeHeavyTotal=2, leaves nine slots available, and releases exactly once. The byte-heavy path now checks live heap pressure, consumes bounded healthy headroom only while healthy, and otherwise retains the existing bounded queue/shed path.

Every genuine structural rejection now records the full event-time scalar context: activeHeavy, activeHealthyHeadroom, activeHeavyTotal, configured/effective headroom, calculated and memory-safe capacity, available slots, limiting budget, headroom reason, telemetry availability, waiting count, queued bytes, and the existing opaque HMAC lane. No credential, tenant key, request body, model, account identity, cgroup path, or queue item is emitted.

Scheduler and lifecycle reconciliation

The branch is based on release/v3.8.51 at
e0ea3f92e1403351686cb783764ba3da6bbddd75.

The structural gate owns heavyweight request preparation through bounded ingestion,
parsing, policy, translation, and upstream-response preparation. It releases when the
handler yields a response. Adaptive admission then retains only its configured
streaming-generation charge. #11493's atomic global/provider/account semaphore remains
held through stream drain or cancellation and is unchanged by this PR.

Provider eligibility can lower an admission upper bound but cannot mint memory capacity
or duplicate scheduler authority.

Preserved safety invariants

  • one process-global heavyweight memory budget shared across sessions
  • exact explicit headroom override precedence, including zero
  • bounded request ingestion and structural inspection
  • immediate contraction without revoking active leases
  • recovery only after the configured real stable interval
  • bounded queue count, queued cost, and wait time
  • round-robin fairness across opaque HMAC lanes on the shared primary queue
  • cancellation and exact-once release across success, error, timeout, and stream teardown
  • phase-aware weighted preparation/generation accounting
  • privacy-safe monitoring and rejection logs
  • no production deployment, active-container mutation, database mutation, provider call, or model call in this correction run

Regression coverage

Coverage includes weighted preparation/generation accounting, queued-work promotion after
phase handoff, structural reservation release at handler response, concurrent independent
byte-heavy admission through healthy headroom, true saturation, event-time rejection
telemetry, HMAC-lane privacy, FIFO/round-robin fairness, queued-byte limits, cancellation,
exact-once release, SSE cancellation, cgroup v1/v2 discovery, memory.high pressure above
the soft ceiling, unlimited sentinels, V8/RSS/external/ArrayBuffer accounting, bare-metal
Windows/macOS fallback, invalid and contradictory telemetry, startup heap calibration,
monitoring projection, pressure contraction, stable recovery, override precedence, and
hard request limits.

Qualification

New correction:

  • concurrent headroom TDD: 0/1 before the correction; 1/1 after
  • rejection-telemetry TDD: failed before the telemetry fields existed; passed after
  • final affected admission/memory/lifecycle/visibility sweep: 100/100 passed
  • touched-file Prettier and git diff --check: passed
  • touched-file ESLint: 0 errors, 0 warnings; only the repository's stale-suppression notice
  • npm run typecheck:core: passed
  • no additional production build was required; the already completed production build below is reused because this correction changes only the typed admission path and focused tests

Previously completed and reused:

  • phase-aware TDD proof: 0/3 red before implementation; 3/3 passed after implementation
  • relevant regression sweep: 90/90 passed
  • focused admission/memory/lifecycle/monitoring sweep: 30/30 passed
  • memory telemetry file after the memory.high correction: 14/14 passed
  • npm run test:vitest: passed
  • npm run typecheck:noimplicit:core: only
    open-sse/translator/response/openai-responses/pureHelpers.ts; the exact base reports
    the same errors and the file has the same blob
    45dd8dea4f27ef77ae00c852f817a42ad6d39ac8
  • full npm run lint: 223 errors, 5 warnings, exactly reproduced on the base; no
    branch-only full-lint error
  • npm run check:docs-all: stops on the inherited missing
    OMNIROUTE_CORPUS_CACHE_SIZE entry in .env.example, exactly reproduced on the
    base; docs sync/counts, links, and fabricated-doc validation passed separately
  • production npm run build: passed (591/591 static pages; postbuild and standalone
    assembly completed)
  • official Compose base build
    (docker compose --profile base build omniroute-base): passed
  • official Compose web/browser build
    (docker compose --profile web build omniroute-web chatgpt-web-codex-browser):
    passed

Draft status

This PR remains Draft. This correction did not request review, merge, or deployment.

A historical pre-correction repo-wide unit-test attempt unexpectedly reached an external
OpenAI execution endpoint with its mock key and received 401; it was stopped and no
provider/model response was produced. This correction made no provider/model calls and no
production mutation.

@KaspaPulse
KaspaPulse force-pushed the feat/admission-runtime-capacity-headroom-20260825 branch 2 times, most recently from 1f322f4 to 335e522 Compare August 25, 2026 18:33
@KaspaPulse KaspaPulse changed the title feat(admission): derive runtime heavyweight headroom fix(admission): derive conservative process-global heavyweight headroom Aug 25, 2026
@KaspaPulse
KaspaPulse force-pushed the feat/admission-runtime-capacity-headroom-20260825 branch from 335e522 to 1d329dd Compare August 26, 2026 05:27
@KaspaPulse
KaspaPulse marked this pull request as ready for review August 26, 2026 06:21
@KaspaPulse
KaspaPulse marked this pull request as draft August 26, 2026 06:25
@KaspaPulse
KaspaPulse force-pushed the feat/admission-runtime-capacity-headroom-20260825 branch from 1d329dd to 8cd5b3b Compare August 26, 2026 08:30
@KaspaPulse
KaspaPulse marked this pull request as ready for review August 26, 2026 11:13
@diegosouzapw

Copy link
Copy Markdown
Collaborator

Diffstat muito grande/complexo (ou divergido de um ponto antigo do tip, misturando valor real com drift massivo) para validação segura no fluxo de lote atual — mesmo padrão já aplicado a #11390/#11461/#11392/#11566/#11564/#11562/#11539/#11513. Adiando para sessão dedicada no ciclo v3.8.52.

@diegosouzapw diegosouzapw added the deferred-v3.8.52 Grande demais / suspeito para o lote atual; precisa de sessão dedicada no ciclo v3.8.52 label Aug 26, 2026
@KaspaPulse
KaspaPulse marked this pull request as draft August 26, 2026 12:01
@KaspaPulse

Copy link
Copy Markdown
Contributor Author

Thanks for the clarification. Understood — I have moved the PR back to Draft and will defer its rebase and scope reduction to a dedicated v3.8.52 session. I will not push further changes during the current cycle.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

deferred-v3.8.52 Grande demais / suspeito para o lote atual; precisa de sessão dedicada no ciclo v3.8.52

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants