fix(channel-affinity): reselect stale cached channels

[ZHOUKAILIAN]

Summary

• Clear the matched channel affinity cache entry when it points to a missing, disabled, or no-longer-eligible channel.
• Fall back to the existing channel selection flow instead of returning 403 before a healthy channel can be selected.
• Add a middleware regression test covering a disabled cached affinity channel being cleared and replaced by an enabled fallback channel.

Root cause

A channel affinity hit could resolve to a channel that had since been disabled. For rules with SkipRetryOnFailure enabled, the distributor returned 403 affinity_channel_disabled immediately, leaving the stale affinity key intact and preventing normal fallback selection.

Test Plan

• go test ./middleware -run TestDistributeClearsDisabledAffinityAndSelectsAvailableChannel -count=1
• go test ./middleware ./service -count=1
• git diff --check
• go test ./... -count=1 currently fails on existing unrelated issues in this checkout:
  • root package setup: main.go:44:12: pattern web/classic/dist: no matching files found
  • controller: sqlite DB closed in TestListModelsTokenLimitIncludesTieredBillingModel
  • relay/channel/claude: existing file content conversion expectations
  • relay/helper: existing TestStreamScannerHandler_StreamStatus_PreInitialized counter mismatch

Summary by CodeRabbit

• Bug Fixes

  • Improved channel selection logic to properly handle disabled channels. When a preferred channel becomes unavailable, the system now clears the preference and selects an enabled fallback channel instead of failing.

• Tests

  • Added test coverage to verify channel fallback behavior when preferred channels are disabled.

[coderabbitai]

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info

⚙️ Run configuration

Configuration used: Organization UI

Review profile: CHILL

Plan: Pro

Run ID: 01e07766-6fdb-4fd1-9317-5913036d4fb6

📥 Commits

Reviewing files that changed from the base of the PR and between 7aaa533 and f3b1e3b.

📒 Files selected for processing (3)

• middleware/distributor.go
• middleware/distributor_channel_affinity_test.go
• service/channel_affinity.go

---

Walkthrough

The PR modifies channel affinity matching in the distributor middleware to clear the affinity cache when a preferred channel is unavailable or disabled, enabling fallback selection. It adds a service helper ClearMatchedChannelAffinity() and updates Distribute() to call it in failure paths. A comprehensive test verifies the behavior with an in-memory database.

Changes

Channel affinity clearing for unavailable preferred channels

Layer / File(s)	Summary
Service-level affinity clearing function service/channel_affinity.go	New exported ClearMatchedChannelAffinity() function reads the request's affinity cache key from gin context and deletes it from the channel-affinity cache, returning success status.
Distributor channel affinity clearing on preferred-channel failure middleware/distributor.go	The Distribute() middleware now clears matched affinity when preferred-channel lookup fails, returns nil, or the channel is disabled; for group-based selection, affinity is cleared when no compatible channel is found in auto groups or when the preferred channel is invalid for the group/model.
Test coverage for disabled affinity channel selection middleware/distributor_channel_affinity_test.go	Comprehensive test suite with in-memory SQLite database setup, helpers for channel/ability seeding and HTTP request construction, affinity cache helpers, and main test case verifying that Distribute() clears a disabled affinity-matched channel and selects an available fallback channel.

Sequence Diagram

sequenceDiagram
  participant Request
  participant Distributor
  participant AffinityCache
  participant ChannelService
  
  Request->>Distributor: POST /v1/responses with prompt cache key
  Distributor->>AffinityCache: GetPreferredChannelByAffinity
  AffinityCache-->>Distributor: preferred channel id (or nil)
  
  alt Preferred channel not found or disabled
    Distributor->>ChannelService: ClearMatchedChannelAffinity
    ChannelService->>AffinityCache: delete affinity cache key
    AffinityCache-->>ChannelService: deletion confirmed
    ChannelService-->>Distributor: cleared
  end
  
  Distributor->>ChannelService: select fallback available channel
  ChannelService-->>Distributor: available channel
  Distributor-->>Request: response with fallback channel

Loading

Estimated code review effort

🎯 3 (Moderate) | ⏱️ ~25 minutes

Possibly related PRs

• QuantumNous/new-api#3333: Both PRs modify middleware/distributor.go's handling of preferred channels from channel affinity when the preferred channel is disabled; retrieved PR adds skip-retry/abort behavior while this PR clears the matched affinity and selects fallback.
• QuantumNous/new-api#4453: Both PRs address the distributor "affinity-selected channel is disabled" code path in middleware/distributor.go—this PR changes affinity clearing behavior while the retrieved PR updates the i18n error message.

Poem

🐰 A channel once sworn by affinity's call,
Found itself disabled—no good after all.
So the middleware cleared its cache with grace,
Found another to serve in its place! ✨

🚥 Pre-merge checks | ✅ 4 | ❌ 1

❌ Failed checks (1 warning)

Check name	Status	Explanation	Resolution
Docstring Coverage	⚠️ Warning	Docstring coverage is 10.00% which is insufficient. The required threshold is 80.00%.	Write docstrings for the functions missing them to satisfy the coverage threshold.

✅ Passed checks (4 passed)

Check name	Status	Explanation
Description Check	✅ Passed	Check skipped - CodeRabbit’s high-level summary is enabled.
Title check	✅ Passed	The title clearly and specifically summarizes the main change: fixing channel affinity by reselecting stale cached channels when they become unavailable.
Linked Issues check	✅ Passed	Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check	✅ Passed	Check skipped because no linked issues were found for this pull request.

_{✏️ Tip: You can configure your own custom pre-merge checks in the settings.}

✨ Finishing Touches

🧪 Generate unit tests (beta)

• Create PR with unit tests

---

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

• X
• Mastodon
• Reddit
• LinkedIn

_{Comment @coderabbitai help to get the list of available commands and usage tips.}