feat(resilience): operator-configurable global credential health check interval - #12043
Merged
diegosouzapw merged 30 commits intoAug 30, 2026
Conversation
…k interval Expose the background credential health check cadence in Settings > Resilience and extend the per-connection Health Check field to every provider (not just OAuth), so operators can see and control what was previously a hardcoded 5-minute sweep against all connections. - Resilience settings gain a credentialHealthCheck section (intervalMinutes, 0-1440; 0 disables the sweep entirely; per-connection overrides always win). Settings precedence: DB > env (CREDENTIAL_HEALTH_CHECK_INTERVAL) > built-in 5 min default. - GET/PATCH /api/resilience round-trip the new section; schema added to updateResilienceSchema (credentialHealthCheckSettingsSchema). - The sweep re-reads the operator cadence each cycle and re-arms its timer with it; an explicit 0 pauses the sweep for every connection without a per-connection override. - EditConnectionModal: Health Check (min) is now rendered for API-key connections too. An empty value means "follow the global default" (clears the stored override via PATCH null semantics); 0 = explicit opt-out; max 1440. - /api/providers/[id] PATCH accepts healthCheckInterval: null to clear the override (schema widened to null | 0-1440). - Resilience tab renders a Credential Health Check card when the server reports the section (graceful no-op against older servers). - Tests: new credential-health-sweep-interval.test.ts (10 cases covering defaults, env fallback, DB precedence, explicit disable, clamping); updated the DEFAULT_RESILIENCE_SETTINGS section-set guard test. Co-authored-by: Cursor <cursoragent@cursor.com>
Co-authored-by: Cursor <cursoragent@cursor.com>
Co-authored-by: Cursor <cursoragent@cursor.com>
adivekar-utexas
force-pushed
the
feat/credential-health-check-global-interval
branch
from
August 30, 2026 14:01
89b72f6 to
6f953f1
Compare
…w#11814) Mantém tokens de cache-write no formato de usage do OpenAI, com teste próprio (`cache-write-openai-shape.test.ts`) e atualização do teste existente de tokens detalhados. Validado no worktree combinado (typecheck limpo, 351/351 testes focados). Obrigado!
… catalog (diegosouzapw#11971) Feature flag para desabilitar variantes de nível de thinking no catálogo, com testes de gate e de settings. Validado no worktree combinado. Obrigado!
…thropic discovery (diegosouzapw#11998) Usa a lista de publishers v1beta1 do Model Garden para descoberta de modelos Vertex Anthropic, com teste próprio. Validado no worktree combinado. Obrigado!
…y, but the diegosouzapw#2539 constraint is model-scoped — this blocks image input to Cloudflare vision models (diegosouzapw#12002) Corrige o achatamento incondicional de conteúdo de mensagem no cloudflare-ai — a restrição diegosouzapw#2539 é model-scoped, não global, e estava bloqueando entrada de imagem em modelos de visão da Cloudflare. Teste próprio atualizado. Validado no worktree combinado. Obrigado!
…nfig (diegosouzapw#12028) Poda árvores não-produção nos excludes do NFT trace e no tsconfig, com teste próprio atualizado. Validado no worktree combinado. Obrigado!
…iegosouzapw#11861) (diegosouzapw#12044) Injeta a tag de usuário obrigatória nas requisições de inferência do provider Nous, com teste próprio ampliado. Validado no worktree combinado. Obrigado!
…effective target pool (diegosouzapw#12046) Deriva as modalidades do auto-combo a partir do pool de targets efetivo, com teste próprio robusto (174 linhas). Validado no worktree combinado. Obrigado!
…souzapw#12054) Corrige ícones de provider colapsando para tamanho zero, com teste próprio. Validado no worktree combinado. Obrigado!
Alinha o body do combo e o acesso legado por chave, com testes atualizados (CLI api-generator + row parsers). Validado no worktree combinado. Obrigado!
Corrige a expectativa de auto-fetch inativo num teste de UI existente. Validado no worktree combinado (vitest 114/114). Obrigado!
Honra a chave de API dedicada de ambiente do CLIProxyAPI, com teste próprio. Validado no worktree combinado. Obrigado!
Adiciona o provider Perplexity Agent API, com dois arquivos de teste próprios (provider + sanitização de chatCore). Validado no worktree combinado. Obrigado!
…iegosouzapw#12106) Vincula o refresh OAuth do Google ao client que emitiu o token, com teste próprio (`google-oauth-client-binding.test.ts`). Validado no worktree combinado. Obrigado!
…ool support (diegosouzapw#12110) Corrige ERR_BN_LIMIT do DuckDuckGo sem retry cego, com suporte a pool de proxy e teste próprio (199 linhas). Validado no worktree combinado. Obrigado!
…ection PII never escapes (diegosouzapw#12115) Corrige vazamento de colunas de conexão (email/nome/etc.) através do cast em `getExclusiveConnectionLeaseStatus` — a projeção agora fica restrita às colunas de lease, evitando que um futuro consumidor sirva PII sem querer via o tipo `ExclusiveConnectionLease`. Documentado como Finding 8 do seu próprio bug-audit (diegosouzapw#12113). Teste próprio (103 linhas). Validado no worktree combinado. Obrigado!
…ook times out (diegosouzapw#12116) Corrige o kill do processo inteiro do plugin quando um handler fire-and-forget de `onStreamComplete` demora >10s — hook documentado como fire-and-forget não deveria derrubar o processo a cada stream completo. Finding 5 do diegosouzapw#12113. Teste próprio (214 linhas). Validado no worktree combinado. Obrigado!
…outes (diegosouzapw#12117) Restaura o log do injection-guard nas 13 rotas não-chat (embeddings, images, audio, moderations, etc.) — a correção de log duplicado anterior (diegosouzapw#11936) silenciou completamente o único emissor de log dessas rotas, deixando tentativas de injeção sem rastro nenhum em modo warn, e sem log mesmo quando bloqueadas em modo block. Achado de segurança real (Finding 2 do diegosouzapw#12113). Teste próprio (141 linhas). Validado no worktree combinado. Obrigado!
…ok fields reach existing installs (diegosouzapw#12120) Refresca o manifest do plugin a partir do disco ao ativar, para que instalações pré-existentes ganhem hooks novos adicionados por schema updates (ex.: `onStreamComplete` do diegosouzapw#11825/diegosouzapw#11934 nunca chegava a plugins já instalados antes do upgrade, pois o manifest persistido no DB era stripado pelo schema antigo). Finding 3 do diegosouzapw#12113. Teste próprio (259 linhas). Validado no worktree combinado. Obrigado!
… a matching connection (diegosouzapw#12122) Corrige o bulk-import do Codex apagando `providerSpecificData`/`tokenExpiresAt`/duplicando `priority` de conexões existentes ao fazer upsert num match — mescla o payload importado sobre o estado existente em vez de substituir tudo, igual ao caminho de import single-file já fazia. Findings 4, 6 e 7 do diegosouzapw#12113. Teste próprio (224 linhas). Validado no worktree combinado. Obrigado!
Migra o quota fetcher do OpenCode Go para a API oficial de uso, com refactor substancial que remove ~1850 linhas de código legado e atualiza a suíte de testes existente inteira para o novo contrato. Validado no worktree combinado (typecheck limpo, testes focados verdes). Obrigado!
…wait budget; preserve errors in oversized call-log artifacts (diegosouzapw#12027) Desacopla a expiração de execução do rate-limit do orçamento de espera na fila, e preserva erros em artefatos de call-log oversized. Testes próprios (`call-log-cap.test.ts` + atualizações em `rate-limit-execution-timeout-message-4165.test.ts`/`ratelimit-admission-control-6593.test.ts`). Validado no worktree combinado. Obrigado!
…101 (diegosouzapw#12036) Adiciona compatibilidade de renomeação de migração para 056/073/077/101. Teste próprio atualizado (7/7 verde no worktree combinado + isolado). Fiz cherry-pick só dos 2 commits reais da PR (o fix + o ajuste do teste) direto na tip atual: a branch original carregava 3 commits antigos de drift do ciclo (release-workflow/electron, já mergeados de outras formas) mais um commit de auto-resolução de merge seu, que juntos geravam conflito redundante contra `.github/workflows/electron-release.yml`. Nenhum conteúdo seu foi perdido — força-pushed a branch limpa (autoria preservada). Obrigado!
…ck (diegosouzapw#12031) Emite `web_search_call` nativo para o fallback de web_search da Responses API, com boa cobertura (integração + unitário). Validado no worktree combinado. Corrigi 3 problemas no próprio `tests/integration/skills-pipeline.test.ts` desta PR antes de mergear: faltava `encodeSkillToolName` no import (usado em 3 lugares, causava `ReferenceError` que se propagava como 502 no teste "matching tool calls execute the registered skill") e 2 asserções comparavam nomes decodificados (`decodeSkillToolName`) contra valores re-codificados (`encodeSkillToolName`) — copy-paste do helper usado para montar o mock. 30/30 testes focados verdes após a correção.
…the bodies (diegosouzapw#12026) (diegosouzapw#12095) Mantém o erro do call-log quando o limite de tamanho corta os bodies, com `preserveErrorForSizeLimit` (UTF-8-safe, preserva o valor original quando cabe, trata erro circular/não-serializável) — implementação mais robusta que a alternativa que já estava na tip (via diegosouzapw#12027, que resolvi combinando: mantive a camada extra "errorOnly" do diegosouzapw#12027 usando o helper mais seguro deste). Testes próprios + os de diegosouzapw#12027 todos verdes (30/30) no worktree combinado. Obrigado!
Adiciona suporte ao GLM-5.3-Flash Coding Plan (endpoint OpenAI-compatible, tiers de esforço low/high/max via reasoning_effort). Boa cobertura de testes. Validado no worktree combinado. Dois problemas resolvidos antes de mergear: 1. **Duplicata silenciosa de "glm-5.3-flash"** em `src/shared/constants/modelSpecs.ts` e `open-sse/config/glmProvider.ts` (diegosouzapw#11830, já mergeado nesta sessão, e sua PR inserem a mesma entrada em pontos diferentes do arquivo — git não detecta como conflito textual). Removida a duplicata, preservando a ordem que o teste pré-existente `open-sse/mcp-server/__tests__/glmCodingProviderConfig.test.ts` espera (glm-5.3-flash primeiro no array `GLM_SHARED_MODELS`). 2. Conflito real em `zai/index.ts`, `default.ts`, `pricing/shared-tiers.ts` e no teste de catálogo — todos aditivos, resolvidos mantendo ambos os lados. 27/27 + 10/10 (vitest) testes focados verdes. Obrigado!
diegosouzapw
merged commit Aug 30, 2026
26bfda3
into
diegosouzapw:release/v3.8.51
3 of 7 checks passed
diegosouzapw
added a commit
that referenced
this pull request
Sep 15, 2026
…bo catalog event-loop pin, unweighted quota-share, resilience key set, pack-gate stamp, synced-catalog env doc (#13678) * fix(combo): rotate unweighted quota-share targets instead of pinning the first The combo resolver turns an unset step weight into 0 (comboStructure.ts), and #10881 made normalizeWeight treat 0 as disabled plus return definition order when the total weight is 0. A quota-share combo without explicit weights therefore had no DRR quanta and dispatched every request to its first target — the combo-matrix/quota-share integration suite saw openai six times out of six. An all-zero set is now an unweighted combo and shares evenly; an explicit 0 still disables a target when its siblings are weighted. Refs #12732 * fix(models): resolve auto-combo target metadata once per catalog build #12046 derives vision/modalities for the built-in auto/* combos by resolving catalog metadata for every target of every combo. The ~40 auto combos share one candidate pool and the loop neither memoized nor yielded, so the #9147 fixture (60 connections, 720 synced models) went from a ~4s build with a 167ms longest event-loop gap to ~11s and a 860-1070ms gap on an idle box — past the 800ms contract and past #12628's 8s cold-build bound, which is why the test came back 500 catalog_build_timeout on every release-green run. Metadata depends only on the target's provider/model/connection scope within a build, so memoize it per build and yield between misses. Same fixture: 2.3-3.1s build, 56-72ms longest gap. The 9147 test is unchanged. Refs #12732 * test(resilience): list credentialHealthCheck in the configuration-only key set #12043 added credentialHealthCheck.intervalMinutes to DEFAULT_RESILIENCE_SETTINGS and to the /api/resilience GET projection. It is operator configuration (the background sweep cadence), not runtime breaker state, but the exact key-set assertion was never updated, so resilience-http-e2e failed on the release tip. The providerBreakers/runtime absence checks stay as they were. Refs #12732 * fix(ci): stamp BUILD_SHA before the release-green pack gate validates check:pack-artifact assembles dist/ through build:cli, which never writes dist/BUILD_SHA (only build:release does). #12959 pointed the provenance ref at HEAD, but the #10427 guard still stops at 'dist/BUILD_SHA is missing' before it ever reaches the ancestry check — reproduced on tip + #13635 + #13436, the first tree whose Turbopack build compiles. ci.yml sequences build -> stamp -> validate; the validator now does the same in both entry points, keeping PACK_GATE_ENV for the validate step. The guard is unchanged: an unstamped dist/ or one built from another commit still fails. On that tree the stamped gate passes: 'BUILD_SHA 5cb3ae5d9 is on the release line'. Refs #12732 * docs(env): document OMNIROUTE_SYNCED_CATALOG_STALE_AFTER_MS #13248 (#12849) added the override for when a connection's synced model list stops being authoritative, but neither .env.example nor ENVIRONMENT.md lists it, so the env/docs contract gate reports it as code-only. The other five vars that gate reports are already added by #13635 and #13361; this touches a different region of .env.example so it does not collide with either. Refs #12732
muhamadgalihsaputra
pushed a commit
to niyatna/NiyatnaRoute
that referenced
this pull request
Sep 27, 2026
…k interval (diegosouzapw#12043) Intervalo de checagem de saúde de credencial configurável pelo operador, com boa cobertura de testes. Validado no worktree combinado (20/20). Corrigi o import de `getCachedSettings` em `src/app/api/resilience/route.ts` e `src/lib/credentialHealth/scheduler.ts`, que apontava para `@/lib/db/settings` (path antigo antes do split para `@/lib/db/readCache`, já na tip). Resolvido também um conflito de tradução vi.json entre chaves duplicadas de outra feature (exclusive lease), sem relação com esta PR — mantida a versão já mergeada. Obrigado!
muhamadgalihsaputra
pushed a commit
to niyatna/NiyatnaRoute
that referenced
this pull request
Sep 27, 2026
…bo catalog event-loop pin, unweighted quota-share, resilience key set, pack-gate stamp, synced-catalog env doc (diegosouzapw#13678) * fix(combo): rotate unweighted quota-share targets instead of pinning the first The combo resolver turns an unset step weight into 0 (comboStructure.ts), and diegosouzapw#10881 made normalizeWeight treat 0 as disabled plus return definition order when the total weight is 0. A quota-share combo without explicit weights therefore had no DRR quanta and dispatched every request to its first target — the combo-matrix/quota-share integration suite saw openai six times out of six. An all-zero set is now an unweighted combo and shares evenly; an explicit 0 still disables a target when its siblings are weighted. Refs diegosouzapw#12732 * fix(models): resolve auto-combo target metadata once per catalog build diegosouzapw#12046 derives vision/modalities for the built-in auto/* combos by resolving catalog metadata for every target of every combo. The ~40 auto combos share one candidate pool and the loop neither memoized nor yielded, so the diegosouzapw#9147 fixture (60 connections, 720 synced models) went from a ~4s build with a 167ms longest event-loop gap to ~11s and a 860-1070ms gap on an idle box — past the 800ms contract and past diegosouzapw#12628's 8s cold-build bound, which is why the test came back 500 catalog_build_timeout on every release-green run. Metadata depends only on the target's provider/model/connection scope within a build, so memoize it per build and yield between misses. Same fixture: 2.3-3.1s build, 56-72ms longest gap. The 9147 test is unchanged. Refs diegosouzapw#12732 * test(resilience): list credentialHealthCheck in the configuration-only key set diegosouzapw#12043 added credentialHealthCheck.intervalMinutes to DEFAULT_RESILIENCE_SETTINGS and to the /api/resilience GET projection. It is operator configuration (the background sweep cadence), not runtime breaker state, but the exact key-set assertion was never updated, so resilience-http-e2e failed on the release tip. The providerBreakers/runtime absence checks stay as they were. Refs diegosouzapw#12732 * fix(ci): stamp BUILD_SHA before the release-green pack gate validates check:pack-artifact assembles dist/ through build:cli, which never writes dist/BUILD_SHA (only build:release does). diegosouzapw#12959 pointed the provenance ref at HEAD, but the diegosouzapw#10427 guard still stops at 'dist/BUILD_SHA is missing' before it ever reaches the ancestry check — reproduced on tip + diegosouzapw#13635 + diegosouzapw#13436, the first tree whose Turbopack build compiles. ci.yml sequences build -> stamp -> validate; the validator now does the same in both entry points, keeping PACK_GATE_ENV for the validate step. The guard is unchanged: an unstamped dist/ or one built from another commit still fails. On that tree the stamped gate passes: 'BUILD_SHA 5cb3ae5d9 is on the release line'. Refs diegosouzapw#12732 * docs(env): document OMNIROUTE_SYNCED_CATALOG_STALE_AFTER_MS diegosouzapw#13248 (diegosouzapw#12849) added the override for when a connection's synced model list stops being authoritative, but neither .env.example nor ENVIRONMENT.md lists it, so the env/docs contract gate reports it as code-only. The other five vars that gate reports are already added by diegosouzapw#13635 and diegosouzapw#13361; this touches a different region of .env.example so it does not collide with either. Refs diegosouzapw#12732
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
20 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
The background credential health check sweeps every active API-key and OAuth connection every 5 minutes by default — hardcoded via
CREDENTIAL_HEALTH_CHECK_INTERVAL, invisible in the UI, and with a per-connection "Health Check (min)" override that only rendered for OAuth connections. This PR makes the cadence operator-configurable in the dashboard and surfaces the per-connection control for all providers.What changed
credentialHealthCheck.intervalMinutes(bounded 0–1440):0disables the sweep entirely (no probes for any connection without an explicit per-connection override).1440= 24 hours.CREDENTIAL_HEALTH_CHECK_INTERVALenv > built-in 5 min default, so existing env-based deployments keep their cadence until an operator overrides it in the UI.EditConnectionModal: the Health Check (min) field now renders for API-key connections too, not just OAuth. Semantics:healthCheckInterval: null, which clears the stored override),0= never check this connection (explicit opt-out),1–1440= per-connection override in minutes.PATCH /api/providers/[id]acceptshealthCheckInterval: nullto clear the override (schema widened tonull | 0–1440);GET/PATCH /api/resilienceround-trip the new section via a newcredentialHealthCheckSettingsSchema.sweep()resolves the operator cadence each cycle (viagetCachedSettings→resolveCredentialHealthSweepInterval), paces connections without an override at that cadence, and re-arms its timer with the operator interval rather than the built-in default. An explicit0skips the sweep for every connection that has no per-connection override (a per-connection0still opts just that connection out even when the global sweep runs).Why
The sweep fires a real upstream probe per connection per cycle. With many accounts that is meaningful upstream traffic on a fixed 5-minute drumbeat that could neither be inspected nor tuned from the UI, and the per-connection escape hatch was hidden for API-key providers — the majority of connections in a typical setup. The Home "Recent Requests" feed already filters
connection-testrows, but the probes were otherwise unexplained traffic for operators.Tests
tests/unit/credential-health-sweep-interval.test.ts(10 cases): defaults, env fallback, DB-beats-env precedence, explicit0disable beats env, clamp to 1440, non-numeric stored value falls back, merge preserves other sections.DEFAULT_RESILIENCE_SETTINGSsection-set guard test inresilience-settings-normalize-split.test.tsfor the new section.resilience-tab-response-fields,resilience-settings-quota-preflight,resilience-provider-cooldown-api-3556, and the full resilience-settings suite (65 tests).npm run typecheck:coreclean; ESLint clean on all changed files (remaining schema-file unused-import warnings are pre-existing on the base commit, verified viagit stash).