Skip to content

feat(providers): let operators add per-provider error rules via settings - #11104

Merged
diegosouzapw merged 1 commit into
diegosouzapw:release/v3.8.50from
maxmad64bis:fix/30-error-rules-operator
Aug 22, 2026
Merged

diegosouzapw merged 1 commit into
diegosouzapw:release/v3.8.50from
maxmad64bis:fix/30-error-rules-operator

Conversation

@maxmad64bis

@maxmad64bis maxmad64bis commented Aug 22, 2026 •

Copy link
Copy Markdown
Contributor

Summary

Providers expose error signals with different shapes (HTTP status, headers, body text). A small
built-in rule set (providerRuleRegistry in open-sse/config/providerErrorRules.ts) handles the
common cases, but operators hit provider-specific quirks the built-in set does not cover. This
adds an operator-configurable error-rule layer (per provider, via settings.providerErrorRules)
that is consulted before the built-in rules, so an operator can reclassify a specific provider's
error signal (scope + cooldown + reason) without a code change.

Related to P#3370 (feat(error-rules): provider-specific error classification with scope), which
created providerErrorRules.ts for this purpose.

Declaring an operator rule for a provider is itself the opt-in. checkFallbackError gates two
things behind allowlists that exist to protect the default behavior of built-in catalog
rules: FULL_TEXT_RULE_PROVIDERS (which providers get the raw error text instead of just
{code, type}) and HONORS_RULE_LOCK_SCOPE_PROVIDERS (which providers have their matched scope
actually consumed by the persistence layer instead of staying informational). Both are exclusive
allowlists (agentrouter-only today) by deliberate owner decision. An operator rule bypasses both
— resolveRuleMatchBody() and honorsRuleLockScope() check hasOperatorRuleForProvider() first
— because the operator already opted in by declaring the rule; gating it a second time behind
allowlists meant for built-in defaults would make the settings mechanism inert for every provider
except the ones already listed there.

No shipped example catalog rules: a hardcoded rule for one specific provider (e.g. nvidia,
huggingface) is exactly the anti-pattern this PR exists to avoid (see PR 5 of the prior upstream
gaps campaign, closed for trying to hardcode 79 provider-specific secrets). The changelog entry
below shows the equivalent operator-declared config instead.

Related Issues

  • Related to upstream provider error classification.

Validation

  • Change type: providers / routing
  • Focused tests: node --import tsx/esm --test tests/unit/provider-error-rules-operator.test.ts → 12/12 pass; npm run lint; npm run typecheck:core; npm run check:cycles
  • npm run lint
  • Reconciled with the current active release base (upstream/release/v3.8.50 3caa59107), focused checks rerun
  • Production-code changes include a new or updated automated test in this PR

Tests Added Or Updated

  • tests/unit/provider-error-rules-operator.test.ts — table-driven scope/cooldown matching,
    operator rule overrides the built-in registry, sub-string match only (never RegExp / ReDoS),
    and a dedicated describe block regression-testing the allowlist bypass:
    resolveRuleMatchBody() hands full text once an operator rule exists for the provider (and
    keeps the structured error when it doesn't), honorsRuleLockScope() flips to true once an
    operator rule exists, and an end-to-end match on raw body text for a provider outside both
    built-in allowlists.

Coverage Notes

  • No src//open-sse/ line coverage regression; the operator rule path, the allowlist-bypass
    branch, and the match precedence are all covered.

Reviewer Notes

  • Task 0 (receptacle): Option A — operator rules live in settings.providerErrorRules
    (zod-validated config), not a DB table. getProviderErrorRuleMatch reads the operator cache
    ahead of the built-in providerRuleRegistry, so operator rules win on first match.
  • Allowlist bypass is the core of this PR — see Summary. Documented in
    docs/architecture/RESILIENCE_GUIDE.md alongside the existing FULL_TEXT_RULE_PROVIDERS/
    HONORS_RULE_LOCK_SCOPE_PROVIDERS write-up.
  • Known, documented, currently-inert gap (not fixed here): checkFallbackError's
    BAD_REQUEST branch classifies HTTP 400 entirely through its own pattern arrays and never
    reaches providerRuleRegistry — a status: 400 rule (built-in or operator) can never match.
    No existing rule targets 400 today, so nothing in production is affected. Fixing it would
    reclassify 400 for every provider already relying on the pattern-array behavior — out of scope
    for this PR; documented in RESILIENCE_GUIDE.md so the next 400-targeting rule doesn't
    reproduce the mistake silently.
  • The 401 re-check at code level: a 401 is classified as authentication in
    open-sse/utils/streamHandler.ts:67 (auth-adjacent, excluded from the generic
    transient/permanent quota path), so the legacy "401 counted permanent vs transient" ambiguity
    only applies to provider-specific 401s. The operator layer lets an operator reclassify a
    specific provider's 401/404/400-outside-BAD_REQUEST-branch signal before the built-in rule
    applies. The generic 401-as-auth path is unchanged.
  • YAGNI: no per-provider rule dump; the operator layer is config, not code — an operator adds
    exactly the providers they run.

@maxmad64bis
maxmad64bis force-pushed the fix/30-error-rules-operator branch from 156dda5 to 06e67ea Compare August 22, 2026 08:41
@maxmad64bis
maxmad64bis marked this pull request as draft August 22, 2026 08:53
Providers expose error signals with different shapes (HTTP status, headers,
body text). A small built-in rule set handles the common cases, but operators
hit provider-specific quirks the built-in set does not cover. This adds an
operator-configurable error-rule layer (per provider, via settings) that is
consulted before the built-in rules, so an operator can reclassify a specific
provider's error signal (scope + cooldown) without a code change.

Declaring an operator rule for a provider is itself the opt-in: it bypasses
FULL_TEXT_RULE_PROVIDERS and HONORS_RULE_LOCK_SCOPE_PROVIDERS (the allowlists
that gate BUILT-IN catalog rules) so the raw error text reaches the matcher
and the declared scope is actually honored end to end, for any provider —
not just the ones already on those allowlists. Without this, an operator
rule for a provider outside the allowlists would silently receive only the
structured {code,type} error and have its scope dropped by the persistence
layer, defeating the point of the settings mechanism.

Drops the two shipped example catalog rules (nvidia 404, huggingface 400):
nvidia's case is already covered by the generic status===404 model-scoping
in getModelLockKey plus passthroughModels (diegosouzapw#6888), and huggingface's 400
never reached providerRuleRegistry in the first place (checkFallbackError's
BAD_REQUEST branch classifies 400 through its own pattern arrays and returns
before providerRuleRegistry is consulted — documented as a known gap in
RESILIENCE_GUIDE.md). Both are demonstrated as operator-declared examples
instead, in the changelog.

Related to P#3370 (feat(error-rules): provider-specific error classification
with scope), which created providerErrorRules.ts for this purpose.
@maxmad64bis
maxmad64bis force-pushed the fix/30-error-rules-operator branch from 06e67ea to 4ebb6b3 Compare August 22, 2026 09:09
@maxmad64bis
maxmad64bis marked this pull request as ready for review August 22, 2026 09:10
@diegosouzapw
diegosouzapw merged commit c89fc6b into diegosouzapw:release/v3.8.50 Aug 22, 2026
17 of 24 checks passed
@maxmad64bis
maxmad64bis deleted the fix/30-error-rules-operator branch September 24, 2026 21:12
muhamadgalihsaputra pushed a commit to niyatna/NiyatnaRoute that referenced this pull request Sep 27, 2026
…ngs (diegosouzapw#11104)

Validated on the combined batch board over release/v3.8.50 tip 0f43f0f: static gates clean, typecheck:core clean, focused tests green.

Operator-declared per-provider error rules via settings, consulted before the built-ins; the allowlist bypass is correct — declaring a rule is itself the opt-in, and no provider-specific rule is hardcoded. Thank you @maxmad64bis!
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants