feat: add group-based refusal fallback routing - #6171
Draft
Faye-labs wants to merge 2 commits into
Draft
Conversation
Contributor
WalkthroughAdds configurable refusal-fallback rules, validates and stores cooldown state, detects upstream refusals, routes requests through fallback groups, and exposes configuration in both settings interfaces with localized text. ChangesRefusal fallback routing
Sequence Diagram(s)sequenceDiagram
participant ClaudeRelay
participant Distributor
participant RefusalFallback
participant HybridCache
participant ChannelSelector
ClaudeRelay->>Distributor: Set upstream refusal context
Distributor->>RefusalFallback: GetRefusalFallbackGroup
RefusalFallback->>HybridCache: Check active cooldown
HybridCache-->>RefusalFallback: Fallback group state
RefusalFallback-->>Distributor: Return routing group
Distributor->>ChannelSelector: Select satisfied fallback channel
ChannelSelector-->>Distributor: Selected channel
Distributor->>RefusalFallback: ObserveRefusalFallback
RefusalFallback->>HybridCache: Set cooldown if absent
Possibly related PRs
Poem
🚥 Pre-merge checks | ✅ 5✅ Passed checks (5 passed)
✨ Finishing Touches🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
Contributor
There was a problem hiding this comment.
🧹 Nitpick comments (2)
middleware/distributor.go (1)
107-123: 📐 Maintainability & Code Quality | 🔵 Trivial | ⚡ Quick winSilently swallows the fallback-channel-selection error.
When
CacheGetRandomSatisfiedChannelfails for the fallback group, the originalerris discarded (err = nil) with no log before falling through to affinity/normal selection. If the fallback group is misconfigured or has no available channels, operators get no signal — the request just quietly proceeds through the other paths. Consider logging the discarded error (e.g., viacommon.SysLog) before clearing it, mirroring thecommon.SysErrorlogging already used elsewhere in this cohort (e.g., inservice/refusal_fallback.go's cache-error paths).🔍 Suggested diagnostic logging
} else { + if err != nil { + common.SysLog(fmt.Sprintf("refusal fallback channel selection failed: group=%s err=%v", fallbackGroup, err)) + } service.ClearCurrentRefusalFallback(c) channel = nil err = nil }🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@middleware/distributor.go` around lines 107 - 123, In the fallback-selection failure branch of the distributor flow, log the non-nil error returned by CacheGetRandomSatisfiedChannel before clearing it and continuing. Use the existing common.SysLog-style diagnostic pattern established in nearby refusal-fallback cache-error paths, while preserving the current cleanup and fallback behavior.service/refusal_fallback_test.go (1)
49-146: 📐 Maintainability & Code Quality | 🔵 Trivial | ⚡ Quick winAll three new test files use
requireexclusively, with noassertfor non-fatal value checks. As per path instructions for**/*_test.go, "New or substantially rewritten Go backend tests must userequirefor setup and fatal assertions andassertfor non-fatal value checks." Every assertion here aborts the test on first failure, so a single wrong value hides any other assertion failures in the same test run — reducing CI diagnostic signal uniformly across this cohort's test additions.
service/refusal_fallback_test.go#L49-L146: keeprequirefor context/setting setup and pointer/error preconditions; switch the outcome checks (e.g.require.False(t, active),require.Equal(t, "claude-fallback", fallbackGroup),require.True(t, ClearCurrentRefusalFallback(retry))) toassert.pkg/cachex/hybrid_cache_test.go#L11-L36: keeprequire.NoErrorfor setup/error preconditions; switchrequire.True(t, created),require.False(t, created),require.True(t, found),require.Equal(t, "first", value),require.False(t, found)toassert.relay/channel/claude/relay_claude_test.go#L21-L35: switchrequire.True(...),require.Equal(...),require.False(...)value checks toassert, keepingrequireonly if a subsequent assertion in the same test would be meaningless after a failure.🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@service/refusal_fallback_test.go` around lines 49 - 146, Replace non-fatal outcome checks with assert while retaining require for setup, error, and pointer preconditions. In service/refusal_fallback_test.go lines 49-146, update value checks around TestRefusalFallbackUsesStableTokenScopeAcrossAffinityIdentityChanges, TestRefusalFallbackDoesNotLeakAcrossScope, TestRefusalFallbackSkipsAutoSourceGroupEvenWhenRuleMatchesAllGroups, TestShouldActivateRefusalFallbackKeepsFixedCooldownWindow, and TestResolveRoutingGroupDoesNotChangeBillingGroup; make the same require-to-assert changes in pkg/cachex/hybrid_cache_test.go lines 11-36 and relay/channel/claude/relay_claude_test.go lines 21-35, keeping require only where later assertions depend on successful setup.Source: Path instructions
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Nitpick comments:
In `@middleware/distributor.go`:
- Around line 107-123: In the fallback-selection failure branch of the
distributor flow, log the non-nil error returned by
CacheGetRandomSatisfiedChannel before clearing it and continuing. Use the
existing common.SysLog-style diagnostic pattern established in nearby
refusal-fallback cache-error paths, while preserving the current cleanup and
fallback behavior.
In `@service/refusal_fallback_test.go`:
- Around line 49-146: Replace non-fatal outcome checks with assert while
retaining require for setup, error, and pointer preconditions. In
service/refusal_fallback_test.go lines 49-146, update value checks around
TestRefusalFallbackUsesStableTokenScopeAcrossAffinityIdentityChanges,
TestRefusalFallbackDoesNotLeakAcrossScope,
TestRefusalFallbackSkipsAutoSourceGroupEvenWhenRuleMatchesAllGroups,
TestShouldActivateRefusalFallbackKeepsFixedCooldownWindow, and
TestResolveRoutingGroupDoesNotChangeBillingGroup; make the same
require-to-assert changes in pkg/cachex/hybrid_cache_test.go lines 11-36 and
relay/channel/claude/relay_claude_test.go lines 21-35, keeping require only
where later assertions depend on successful setup.
ℹ️ Review info
⚙️ Run configuration
Configuration used: Organization UI
Review profile: CHILL
Plan: Pro
Run ID: f7697c96-f482-4dc7-a5a3-9453c54faa08
📒 Files selected for processing (36)
constant/context_key.gocontroller/option.gocontroller/relay.gomiddleware/distributor.gopkg/cachex/hybrid_cache.gopkg/cachex/hybrid_cache_test.gorelay/channel/claude/relay-claude.gorelay/channel/claude/relay_claude_test.goservice/channel_select.goservice/log_info_generate.goservice/refusal_fallback.goservice/refusal_fallback_test.gosetting/operation_setting/refusal_fallback_setting.gosetting/operation_setting/refusal_fallback_setting_test.goweb/classic/src/components/settings/ModelSetting.jsxweb/classic/src/i18n/locales/en.jsonweb/classic/src/i18n/locales/fr.jsonweb/classic/src/i18n/locales/ja.jsonweb/classic/src/i18n/locales/ru.jsonweb/classic/src/i18n/locales/vi.jsonweb/classic/src/i18n/locales/zh-CN.jsonweb/classic/src/i18n/locales/zh-TW.jsonweb/classic/src/pages/Setting/Operation/SettingsRefusalFallback.jsxweb/default/src/features/models/components/drawers/model-mutate-drawer.tsxweb/default/src/features/system-settings/models/index.tsxweb/default/src/features/system-settings/models/refusal-fallback-section.tsxweb/default/src/features/system-settings/models/section-registry.tsxweb/default/src/features/system-settings/types.tsweb/default/src/i18n/locales/_reports/_sync-report.jsonweb/default/src/i18n/locales/en.jsonweb/default/src/i18n/locales/fr.jsonweb/default/src/i18n/locales/ja.jsonweb/default/src/i18n/locales/ru.jsonweb/default/src/i18n/locales/vi.jsonweb/default/src/i18n/locales/zh-TW.jsonweb/default/src/i18n/locales/zh.json
Calcium-Ion
force-pushed
the
main
branch
2 times, most recently
from
August 30, 2026 15:03
51fdfc5 to
2b6f1df
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Important
📝 变更描述 / Description
新增可配置的 refusal fallback 路由，用于上游完整返回
stop_reason=refusal后，将后续重试临时切换到一个备用分组：ContextKeyUsingGroup保持原值，因此普通明确分组的计费语义不变，备用渠道也不会覆盖原有渠道亲和记录。auto来源分组当前明确跳过 fallback，避免在没有具体auto_group的情况下错误回落到 1 倍计费；fallback 分组本身也不能配置为auto。当前发生 refusal 的 HTTP 请求不会在 New API 内部自动重放；本功能作用于其后的客户端或上层服务重试请求。
🚀 变更类型 / Type of change
🔗 关联任务 / Related Issue
✅ 提交前检查项 / Checklist
📸 运行证明 / Proof of Work
Go：
Default Admin：
Classic Admin：
其他：
Summary by CodeRabbit