Skip to content

feat(relay): expose user and group context to parameter override - #6534

Merged
Calcium-Ion merged 1 commit into
QuantumNous:mainfrom
seefs001:feature/internal-param-user-id
Aug 10, 2026
Merged

feat(relay): expose user and group context to parameter override#6534
Calcium-Ion merged 1 commit into
QuantumNous:mainfrom
seefs001:feature/internal-param-user-id

Conversation

@seefs001

@seefs001 seefs001 commented Jul 30, 2026

Copy link
Copy Markdown
Collaborator

⚠️ 提交说明 / PR Notice

Important

  • 请提供人工撰写的简洁摘要,避免直接粘贴未经整理的 AI 输出。

📝 变更描述 / Description

(简述:做了什么?为什么这样改能生效?请基于你对代码逻辑的理解来写,避免粘贴未经整理的内容)

🚀 变更类型 / Type of change

  • 🐛 Bug 修复 (Bug fix) - 请关联对应 Issue,避免将设计取舍、理解偏差或预期不一致直接归类为 bug
  • ✨ 新功能 (New feature) - 重大特性建议先通过 Issue 沟通
  • ⚡ 性能优化 / 重构 (Refactor)
  • 📝 文档更新 (Documentation)

🔗 关联任务 / Related Issue

  • Closes # (如有)

✅ 提交前检查项 / Checklist

  • 人工确认: 我已亲自整理并撰写此描述,没有直接粘贴未经处理的 AI 输出。
  • 非重复提交: 我已搜索现有的 IssuesPRs,确认不是重复提交。
  • Bug fix 说明: 若此 PR 标记为 Bug fix,我已提交或关联对应 Issue,且不会将设计取舍、预期不一致或理解偏差直接归类为 bug。
  • 变更理解: 我已理解这些更改的工作原理及可能影响。
  • 范围聚焦: 本 PR 未包含任何与当前任务无关的代码改动。
  • 本地验证: 已在本地运行并通过测试或手动验证,维护者可以据此复核结果。
  • 安全合规: 代码中无敏感凭据,且符合项目代码规范。

📸 运行证明 / Proof of Work

(请在此粘贴截图、关键日志或测试报告,以证明变更生效)

Summary by CodeRabbit

  • New Features

    • Parameter overrides can now be conditionally applied based on user, user group, token group, and group usage context.
    • Conditions can also target the upstream model, enabling more precise configuration such as applying settings only to selected GPT models.
  • Tests

    • Added coverage for user, model, and group-based parameter override conditions.

@coderabbitai

coderabbitai Bot commented Jul 30, 2026

Copy link
Copy Markdown
Contributor

Review Change Stack

Walkthrough

BuildParamOverrideContext now includes user and grouping fields from RelayInfo. Tests cover conditional service_tier overrides based on user/model matches and combined group context conditions.

Changes

Param override context

Layer / File(s) Summary
Context fields and conditional override validation
relay/common/override.go, relay/common/override_test.go
BuildParamOverrideContext now exposes user_id, user_group, token_group, and using_group; tests validate user/model and group-based conditional overrides.

Estimated code review effort: 2 (Simple) | ~10 minutes

Suggested reviewers: calcium-ion

Poem

A rabbit hops through fields so neat,
User and groups now guide the treat.
GPT matches, tiers softly bloom,
Tests check each condition’s room.
Carrots cheer the override tune!

🚥 Pre-merge checks | ✅ 5
✅ Passed checks (5 passed)
Check name Status Explanation
Docstring Coverage ✅ Passed No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check.
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check ✅ Passed The title clearly and concisely describes exposing user and group context to relay parameter overrides.
✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 1

🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@relay/common/override.go`:
- Around line 2047-2049: Implement the documented token_group fallback in the
override publication logic around the TokenGroup and UserGroup fields: when
TokenGroup is empty, publish UserGroup instead; preserve an explicitly provided
TokenGroup unchanged. Add a regression test covering UserGroup set with an empty
TokenGroup and verify token_group-based overrides match.
🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: Organization UI

Review profile: CHILL

Plan: Pro Plus

Run ID: fc631ee9-b2df-4044-bae4-2af201977a3c

📥 Commits

Reviewing files that changed from the base of the PR and between 66ee6b8 and 17d79d4.

📒 Files selected for processing (2)
  • relay/common/override.go
  • relay/common/override_test.go

Comment thread relay/common/override.go
Comment on lines +2047 to +2049
// - user_group:用户所属分组。
// - token_group:令牌指定的分组;未指定时回退为用户分组。
// - using_group:当前实际使用的分组,自动跨分组重试时可能变化。

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🎯 Functional Correctness | 🟡 Minor | ⚡ Quick win

Implement the documented token_group fallback.

Line 2060 publishes an empty TokenGroup unchanged, contradicting Line 2048’s fallback-to-UserGroup contract. Overrides conditioned on token_group will not match for tokens without an explicit group.

Proposed fix
 ctx["user_id"] = info.UserId
 ctx["user_group"] = info.UserGroup
-ctx["token_group"] = info.TokenGroup
+tokenGroup := info.TokenGroup
+if tokenGroup == "" {
+	tokenGroup = info.UserGroup
+}
+ctx["token_group"] = tokenGroup
 ctx["using_group"] = info.UsingGroup

Add a regression case with UserGroup set and TokenGroup empty.

Also applies to: 2058-2060

🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@relay/common/override.go` around lines 2047 - 2049, Implement the documented
token_group fallback in the override publication logic around the TokenGroup and
UserGroup fields: when TokenGroup is empty, publish UserGroup instead; preserve
an explicitly provided TokenGroup unchanged. Add a regression test covering
UserGroup set with an empty TokenGroup and verify token_group-based overrides
match.

@seefs001
seefs001 marked this pull request as ready for review August 10, 2026 04:46
@Calcium-Ion
Calcium-Ion merged commit 85feb7a into QuantumNous:main Aug 10, 2026
4 checks passed
latioswang added a commit to trycortexai/new-api that referenced this pull request Aug 10, 2026
* fix(relay): set Request.GetBody so the HTTP/2 transport can transparently retry after an upstream stream reset (QuantumNous#6249)

* fix(relay): set Request.GetBody so the HTTP/2 transport can transparently retry after an upstream stream reset

The outbound request body is a type-erased io.Reader over BodyStorage, so
net/http cannot derive Request.GetBody (it only does so for *bytes.Reader,
*bytes.Buffer and *strings.Reader). With GetBody nil, the HTTP/2 transport
cannot transparently retry a request once the body has been written and the
upstream resets the stream with a retryable error (REFUSED_STREAM, or a
connection-level GOAWAY); the relay request then fails with:

    http2: Transport: cannot retry err [...] after Request.Body was written;
    define Request.GetBody to avoid this error

This affects every relay path that goes through DoApiRequest (chat, claude,
gemini, responses, embedding, image, rerank).

BodyStorage (memory and disk) already implements io.Seeker, so replay support
only needed wiring:

- NewOutboundJSONBody additionally returns a getBody that rewinds the storage
  and hands out a fresh non-closing reader. The transport only calls GetBody
  after the previous attempt's body has been abandoned, so the rewind cannot
  race an in-flight read.
- RelayInfo carries it in the new UpstreamRequestGetBody field, set alongside
  UpstreamRequestBodySize by the handlers that build storage-backed bodies.
- applyUpstreamGetBody (symmetric with applyUpstreamContentLength) wires it
  into DoApiRequest/DoFormRequest/DoTaskApiRequest, only when req.GetBody is
  still nil.

Also remove the hand-rolled GetBody override in DoTaskApiRequest: it returned
the same already-consumed reader, so any transport-level replay would have
silently sent an empty body, and it clobbered the correct snapshot-based
GetBody that net/http derives from the *bytes.Reader bodies the task adaptors
pass in. For non-replayable bodies GetBody now stays nil, so a retry fails
loudly instead of corrupting the request.

Covered by unit tests plus an end-to-end raw-frame HTTP/2 test that resets
the first stream with REFUSED_STREAM after the body is written and asserts
the transport transparently retries with the complete body.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* fix(relay): hand out independent readers from GetBody (address review)

Per the http.Request.GetBody contract ("returns a new copy of Body"),
each call must yield a reader with its own cursor. The previous
implementation rewound and reused the shared BodyStorage, so two
consecutive GetBody readers would interfere with each other, and a
replay could disturb the primary body's offset under extreme transport
timing (e.g. attempt N's body write not yet fully abandoned when the
transport builds attempt N+1).

Instead of snapshotting the payload (an extra copy), add
BodyStorage.NewReader, which returns an independent zero-copy reader:

- memory mode: a fresh bytes.Reader over the same immutable backing
  array;
- disk mode: a separate file descriptor over the cache file, so the
  transport closing a replayed body only closes that descriptor.

NewOutboundJSONBody's getBody now simply hands out storage.NewReader,
and once the handler releases the storage, GetBody fails with
ErrStorageClosed instead of replaying stale data.

Tests: interleaved reads across two replay readers and the primary
body each observe exactly their own byte stream, for both the memory
and the disk-backed storage; the existing GetBody and HTTP/2 retry
suites still pass (h2 e2e tests flake-free with -count=20).

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* fix(relay): bind replayable metadata on pass-through requests

* fix(relay): reset upstream body metadata between channels

* test(relay): cover replay across retries and channel attempts

* fix(relay): stop following upstream redirects

---------

Co-authored-by: Claude Fable 5 <noreply@anthropic.com>

* refactor(relay): move replay metadata onto request bodies

* Merge commit from fork

* feat(channels): refine fetched model categorization (QuantumNous#6632)

* feat(channels): refine fetched model categorization

* fix: channel category

* fix: hy3 category

* fix: test Claude/Gemini endpoints with native request format (QuantumNous#6698)

* feat(rate-limit): add user critical rate limit middleware for access token and aff transfer routes

* fix: 修复兑换码额度精度损失 (QuantumNous#6685)

* fix: 修复兑换码额度精度损失(QuantumNous#6680)

* fix(redemption): guard update data integrity

* CI: enhance release synchronization workflow with optional file syncing

* fix(ali): stop injecting top_p into requests that omit it (QuantumNous#6674)

* fix(channels): classify Qwen TTS models correctly (QuantumNous#6711)

* feat(channels): add auto-disable-only channel test mode (QuantumNous#6728)

* perf(web): debounce server and large-list searches (QuantumNous#6727)

* fix: record reasoning effort consistently in usage logs (QuantumNous#6641)

* feat(relay): expose user and group context to parameter overrides (QuantumNous#6534)

* fix(ollama): preserve reasoning and tool-call context (QuantumNous#6605)

* fix: backend length validation (QuantumNous#5548)

* feat(billing): highlight matched conditional multipliers in logs (QuantumNous#6561)

* feat(billing): highlight matched conditional multipliers in usage logs

* fix(billing): make request rule tracing stable and type-safe

* fix(web): require confirmation before rotating access token (QuantumNous#6749)

---------

Co-authored-by: Lucas <hepo.lucas@gmail.com>
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
Co-authored-by: CaIon <i@caion.me>
Co-authored-by: RedwindA <128586631+RedwindA@users.noreply.github.com>
Co-authored-by: Seefs <40468931+seefs001@users.noreply.github.com>
Co-authored-by: lihu-001 <lihu9048@gmail.com>
Co-authored-by: ENCHIGO <38551565+ENCHIGO@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants