Skip to content

fix(channel): recover auto-disabled multi-key channels - #7071

Open
qwe605 wants to merge 1 commit into
QuantumNous:mainfrom
qwe605:fix/multikey-channel-recovery
Open

fix(channel): recover auto-disabled multi-key channels#7071
qwe605 wants to merge 1 commit into
QuantumNous:mainfrom
qwe605:fix/multikey-channel-recovery

Conversation

@qwe605

@qwe605 qwe605 commented Aug 28, 2026

Copy link
Copy Markdown

Agent

  • Tool: Codex Desktop(代码由 AI 辅助生成,提交者已人工检查并完成本地验证)
  • Tool version: 当前桌面版未暴露构建号
  • Model (full id): gpt-5.5
  • Host (CLI / IDE / GitHub coding agent / other): IDE(Codex Desktop)
  • Date (UTC): 2026-08-28 13:20:48 UTC

Links

User request

根据 #7040 做尽可能小的修改,先理清项目框架,遵循现有语言与代码风格,并进行真实本地测试后提交 PR。

Out of scope — refuse

If the change matches any item below, tell the user this repository does not
accept it and do not open a PR.

  • Coding Plan

  • Reverse-engineered channels

  • Third-party API wrappers

  • Codex channel-type changes, or compatibility from exposing Codex as a general-purpose API

  • Codex API-specific protocol or behavior treated as standard OpenAI API behavior

  • Pass-through-only forwarding

  • Third-party hosting sites, relay services, or API services

  • Usage, configuration, or integration (answer from docs and code instead)

  • Matched: no

  • If yes, what was told to the user (stop here; do not open a PR): 不适用

Kind

  • Bug fix
  • New feature
  • Performance / refactor
  • Docs
  • Other:

Issue facts

Take these from the linked issue. If a needed item is empty, ask the user that question.

  • Actual behavior: 多密钥渠道的所有 Key 自动禁用后,健康检查仍走 GetNextEnabledKey(),在本地返回 no enabled keys,不会请求上游,因此无法自动恢复。
  • Impact: 同时启用多 Key、自动禁用、自动启用和定期渠道测试的渠道可能永久停留在 auto-disabled,必须人工恢复。
  • Frequency: 条件满足后必现,之后每轮定期测试都会重复失败。
  • Evidence that the problem is in new-api rather than the client or upstream: 错误发生于 SetupContextForSelectedChannelGetNextEnabledKey 的本地 Key 选择阶段,请求尚未发往上游。
  • Applicable types and their fields (relay / billing / frontend / deployment; write "not applicable" otherwise): model.Channel.ChannelInfoIsMultiKeyMultiKeyStatusList;不涉及 billing、frontend 或 deployment。

Change

  • 为渠道测试上下文增加显式 keyIndex 入口;普通请求仍只使用 GetNextEnabledKey()
  • auto-disabled 多 Key 渠道的健康检查仅逐个探测 auto-disabled Key,跳过 manually-disabled Key。
  • 成功 Key 恢复 enabled;失败 Key 保持 auto-disabled;至少一个成功时渠道恢复 enabled。
  • 调整多 Key 状态更新的同状态短路条件,使渠道已被第一个 Key 恢复后,其余成功 Key 仍能清除各自的禁用状态。
  • System Task 的 tested/succeeded/enabled 统计继续按渠道计数。

Research

Duplicate / prior art

Docs and code

Open them. Do not write "already checked" without sources.

  • https://docs.newapi.ai/ : 检查了官方文档入口;该问题不是配置或接入问题,而是健康检查内部状态选择错误。
  • https://deepwiki.com/QuantumNous/new-api : 检查了项目架构资料;修复保持 Router → Controller → Service → Model 的现有分层。
  • README / repo docs: 检查 README 与 AGENTS.md;README 没有定义该内部恢复行为,改动遵循后端测试、数据库兼容和代码质量规则。
  • Code paths and what they imply for this change: TestAllChannels → system task → runChannelTestTasktestChannelForHealthChecktestChannelmiddleware.SetupContextForSelectedChannel;状态落盘复用 service.EnableChannelmodel.UpdateChannelStatus

Alternatives considered

Files

Path Why
controller/channel-test.go auto-disabled 多 Key 健康检查逐 Key 恢复
controller/channel_test_internal_test.go 验证只选择 auto-disabled Key
middleware/distributor.go 提供健康检查专用的显式 Key 上下文
middleware/distributor_test.go 验证显式选择与普通选择边界
model/channel.go 允许渠道恢复后继续更新其余 Key 状态
model/channel_status_test.go 验证多个 Key 能依次恢复

Behavior

  • Before: 所有 Key auto-disabled 后,健康检查在本地返回 no enabled keys,上游零请求,渠道不能自愈。
  • After: 健康检查逐个探测 auto-disabled Key;成功 Key 恢复,失败 Key 保持禁用,手动禁用 Key 不参与测试;普通业务请求仍只选择 enabled Key。
  • Explicit non-goals / leftover work: 不增加手动逐 Key 测试 UI、管理 API、新路由或并行 Key 测试。

Verification

Only what was actually run.

  • Commands and results:
    • go test ./model ./middleware ./controller -count=1:通过。
    • go vet ./model ./middleware ./controller:通过。
    • go build -p 1 ./...:通过。
    • go test -p 1 ./... -count=1:本 PR 涉及包均通过;仓库现有 service 指标测试存在跨用例状态污染,失败用例单独运行通过。
  • Manual steps and observed result:
    • 实际启动本分支编译的 new-api.exe、独立 SQLite、System Task Worker 和本地 HTTP 假上游,通过真实登录及 /api/channel/test API 验证。
    • 所有 Key 禁用时,普通单渠道测试返回 no enabled keys,上游请求数为 0。
    • key-a 返回 401、key-b 返回 200:两者均被探测;最终渠道 enabled,key-a 保持 auto-disabled,key-b 恢复。
    • 两个 Key 都返回 200:两者均被探测,最终禁用状态表为空。
    • key-a manually-disabled、key-b auto-disabled:只探测 key-bkey-a 保持 manually-disabled。
  • UI: 无 UI 改动,不适用。
  • Tests added or updated, or why none: 新增/更新 controller、middleware、model 回归测试,覆盖选择边界和连续恢复。
  • Databases / providers / platforms exercised: Windows 11、SQLite、OpenAI Chat Completions 格式的本地假上游。
  • Not verified: MySQL、PostgreSQL 和真实第三方上游;本次没有新增 SQL 或数据库方言逻辑。

Risks

  • Failure modes: Key 数量较多时,同一渠道内恢复测试仍为顺序执行,单轮健康检查时间会增加。
  • Billing / quota / auth impact: 渠道测试沿用现有测试计费/认证路径;不修改用户请求、额度或鉴权逻辑。
  • Follow-ups: 如维护者需要,可另行讨论 Key 级并发或管理端逐 Key 测试,不纳入本 PR。

Scope check

  • Single focused change: yes
  • Secrets included: no
  • Out of scope (Coding Plan / reverse-engineered channel / third-party wrapper / Codex): no

Summary by CodeRabbit

  • Bug Fixes

    • Improved recovery for multi-key channels by checking auto-disabled keys individually.
    • Preserved manually disabled keys during automatic recovery.
    • Enabled explicit selection of an auto-disabled key when appropriate.
    • Correctly restores channel availability after all eligible keys recover.
    • Improved per-key status updates and validation when selecting channel keys.
  • Tests

    • Added coverage for multi-key key selection, recovery behavior, status restoration, and manual-disable preservation.

@coderabbitai

coderabbitai Bot commented Aug 28, 2026

Copy link
Copy Markdown
Contributor

Review Change Stack

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info
⚙️ Run configuration

Configuration used: Organization UI

Review profile: CHILL

Plan: Pro Plus

Run ID: e25d8f96-f90a-440d-bd85-0832ff43daec

📥 Commits

Reviewing files that changed from the base of the PR and between 98d50d5 and 0cbce65.

📒 Files selected for processing (6)
  • controller/channel-test.go
  • controller/channel_test_internal_test.go
  • middleware/distributor.go
  • middleware/distributor_test.go
  • model/channel.go
  • model/channel_status_test.go
🚧 Files skipped from review as they are similar to previous changes (6)
  • model/channel_status_test.go
  • model/channel.go
  • controller/channel_test_internal_test.go
  • middleware/distributor_test.go
  • controller/channel-test.go
  • middleware/distributor.go

Included review availability: Your plan provides up to 10 included reviews per hour; 9 remain after this review.


Walkthrough

Health checks now probe auto-disabled keys in multi-key channels individually. Middleware accepts explicit key indexes. Per-key status updates can restore successful keys while preserving manual disables. Tests cover selection, filtering, and channel recovery.

Changes

Multi-key channel recovery

Layer / File(s) Summary
Explicit key selection
middleware/distributor.go, middleware/distributor_test.go
The middleware supports explicit multi-key selection, including auto-disabled keys. Invalid selections return the no-available-key error.
Per-key status updates
model/channel.go, model/channel_status_test.go
Status updates for specified keys proceed when the channel status already matches. Recovery clears auto-disabled key entries after successful re-enablement.
Health-check recovery orchestration
controller/channel-test.go, controller/channel_test_internal_test.go
Health checks test auto-disabled keys individually, aggregate results, stop on cancellation, and skip manually disabled keys. Normal channel tests retain automatic key selection.

Estimated code review effort: 3 (Moderate) | ~20 minutes

Merge Risk: ⚪ Minimal · up to 0cbce

This change lets health checks recover auto-disabled multi-key channels without altering normal request key selection, while preserving manually disabled keys; no actionable merge-blocking risk remains beyond normal checks and review.

Poem

A rabbit checks each key in line
Auto-disabled keys now align
Manual disables stay in place
Successful keys regain their space
The channel returns to enabled state

🚥 Pre-merge checks | ✅ 4 | ❌ 1

❌ Failed checks (1 warning)

Check name Status Explanation Resolution
Docstring Coverage ⚠️ Warning Docstring coverage is 10.00% which is insufficient. The required threshold is 80.00%. Docstring coverage is scoped to functions touched by this diff. Analyzed 20 functions across 6 files. Write docstrings for the functions missing them to satisfy the coverage threshold.
✅ Passed checks (4 passed)
Check name Status Explanation
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check ✅ Passed The title clearly summarizes the primary change: recovery of auto-disabled multi-key channels.
Linked Issues check ✅ Passed The changes address issue #7040. Recovery tests explicitly select auto-disabled keys, skip manually disabled keys, preserve failed keys, restore successful keys, and re-enable the channel when a key s…
Out of Scope Changes check ✅ Passed All changes support the linked issue. The controller, middleware, model logic, and regression tests directly implement or verify multi-key channel recovery. No unrelated UI, management API, route, or …
Full details: Linked Issues check

Explanation

The changes address issue #7040. Recovery tests explicitly select auto-disabled keys, skip manually disabled keys, preserve failed keys, restore successful keys, and re-enable the channel when a key succeeds. Normal requests retain enabled-key selection.

Full details: Out of Scope Changes check

Explanation

All changes support the linked issue. The controller, middleware, model logic, and regression tests directly implement or verify multi-key channel recovery. No unrelated UI, management API, route, or parallel-testing changes are present.

  • Fix all pre-merge checks with AI
✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@qwe605
qwe605 force-pushed the fix/multikey-channel-recovery branch from bb7bb27 to 44e34aa Compare August 28, 2026 13:31
@qwe605
qwe605 force-pushed the fix/multikey-channel-recovery branch from 44e34aa to 0cbce65 Compare August 29, 2026 11:25
@coderabbitai

coderabbitai Bot commented Aug 29, 2026

Copy link
Copy Markdown
Contributor

Note

GitHub couldn't provide a complete incremental comparison for this pull request, so CodeRabbit is performing a full review instead. This review may take a little longer.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[Bug] 多密钥渠道所有 Key 自动禁用后,定期测试无法恢复渠道

1 participant