Skip to content

fix(monitoring): serve cached credentialHealth so /healthz is not starved - #12533

Merged
diegosouzapw merged 1 commit into
diegosouzapw:release/v3.8.51from
RaviTharuma:cursor/monitoring-health-nonblocking-55a1
Sep 4, 2026
Merged

diegosouzapw merged 1 commit into
diegosouzapw:release/v3.8.51from
RaviTharuma:cursor/monitoring-health-nonblocking-55a1

Conversation

@RaviTharuma

Copy link
Copy Markdown
Contributor

Summary

  • GET /api/monitoring/health (management view) now serves the last cached payload immediately, including a stale credentialHealth summary. Credential probes stay on the background scheduler and yield the event loop between batches so GET /healthz is not starved under scrape load.
  • Official diegosouzapw/omniroute:3.8.50 still rebuilt the ~8KB monitoring payload on the request path (1s TTL miss), which blocked in-pod /healthz (~6s stall) and caused a 10s json-exporter scrapeTimeout (context deadline exceeded). No env var on 3.8.50 made that handler non-blocking.
  • This change does not bump the published image tag and does not add cluster-specific timeouts or replica workarounds.

Related Issues

Validation

  • Change type: other (monitoring / resilience)
  • Focused tests: tests/unit/monitoring-health-cached-credential.test.ts, tests/integration/monitoring-health-cache.test.ts
  • Reconciled with parent default release/v3.8.51 (1 commit ahead, 0 behind)
  • Production-code changes include a new or updated automated test in this PR

Tests Added Or Updated

  • tests/unit/monitoring-health-cached-credential.test.ts — stale/cached credentialHealth is returned immediately; the health route does not import live probe entry points.
  • tests/integration/monitoring-health-cache.test.ts — expired payload is served immediately (stale-while-revalidate); DELETE still forces a rebuild.

Reviewer Notes

  • Request path never calls live credential probes. Probes remain in src/lib/credentialHealth/scheduler.ts and yield between batches.
  • Cold cache (first GET or post-DELETE) still aggregates in-process snapshots and DB reads, but does not run live credential I/O on that path.

GET /api/monitoring/health no longer rebuilds on every cache miss or
runs live credential probes while answering scrapes. Serve the last
payload immediately (stale-while-revalidate), read credentialHealth
from the in-memory cache including stale rows, and yield the event
loop between background probe batches so /healthz stays responsive.

Closes #12532

Co-authored-by: Ravi Tharuma <RaviTharuma@users.noreply.github.com>
Copilot AI lite review requested due to automatic review settings September 3, 2026 00:09

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot was unable to review this pull request because the user who requested the review has reached their quota limit.

@diegosouzapw
diegosouzapw merged commit 5f9c358 into diegosouzapw:release/v3.8.51 Sep 4, 2026
16 checks passed
@RaviTharuma
RaviTharuma deleted the cursor/monitoring-health-nonblocking-55a1 branch September 23, 2026 19:36
muhamadgalihsaputra pushed a commit to niyatna/NiyatnaRoute that referenced this pull request Sep 27, 2026
…iegosouzapw#12533)

Validado em lote numa worktree combinada com os 10 PRs desta leva sobre o tip de `release/v3.8.51`: `typecheck:core` limpo, `check-file-size` OK e **241/242** nos 29 arquivos de teste que os PRs tocam.

A única "falha" não é falha: `tests/unit/autoCombo/strict-zero-cost-filter.test.ts` é um teste em estilo Vitest que eu incluí por engano na invocação do runner nativo do Node — ele quebra no import (`@vitest/runner`), não numa asserção. Ao investigar, descobri que esse arquivo não roda em nenhum dos dois runners hoje (o glob do `test:unit` não lista `autoCombo` e o `include` do Vitest só pega `.tsx` nessa pasta); é um problema pré-existente do repositório, sem relação com esta leva, e vou registrá-lo separadamente.

O diegosouzapw#12636 conflitava apenas na lista de testes do `@omniroute/opencode-plugin/package.json`, de forma aditiva: o tip já tinha `models-fetcher.test.ts` (do diegosouzapw#12607, irmão desta mesma leva) e o diegosouzapw#12636 acrescenta `telemetry.test.ts`. Fiz a união dos dois lados (25 arquivos contra 24 de cada) em vez de escolher um, o que teria removido um arquivo da suíte do plugin em silêncio.

Obrigado, @RaviTharuma.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[BUG] GET /api/monitoring/health live probes starve /healthz on official 3.8.50

3 participants