feat(relay): expose user and group context to parameter override - #6534
Conversation
Walkthrough
ChangesParam override context
Estimated code review effort: 2 (Simple) | ~10 minutes Suggested reviewers: Poem
🚥 Pre-merge checks | ✅ 5✅ Passed checks (5 passed)
✨ Finishing Touches🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
There was a problem hiding this comment.
Actionable comments posted: 1
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In `@relay/common/override.go`:
- Around line 2047-2049: Implement the documented token_group fallback in the
override publication logic around the TokenGroup and UserGroup fields: when
TokenGroup is empty, publish UserGroup instead; preserve an explicitly provided
TokenGroup unchanged. Add a regression test covering UserGroup set with an empty
TokenGroup and verify token_group-based overrides match.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Organization UI
Review profile: CHILL
Plan: Pro Plus
Run ID: fc631ee9-b2df-4044-bae4-2af201977a3c
📒 Files selected for processing (2)
relay/common/override.gorelay/common/override_test.go
| // - user_group:用户所属分组。 | ||
| // - token_group:令牌指定的分组;未指定时回退为用户分组。 | ||
| // - using_group:当前实际使用的分组,自动跨分组重试时可能变化。 |
There was a problem hiding this comment.
🎯 Functional Correctness | 🟡 Minor | ⚡ Quick win
Implement the documented token_group fallback.
Line 2060 publishes an empty TokenGroup unchanged, contradicting Line 2048’s fallback-to-UserGroup contract. Overrides conditioned on token_group will not match for tokens without an explicit group.
Proposed fix
ctx["user_id"] = info.UserId
ctx["user_group"] = info.UserGroup
-ctx["token_group"] = info.TokenGroup
+tokenGroup := info.TokenGroup
+if tokenGroup == "" {
+ tokenGroup = info.UserGroup
+}
+ctx["token_group"] = tokenGroup
ctx["using_group"] = info.UsingGroupAdd a regression case with UserGroup set and TokenGroup empty.
Also applies to: 2058-2060
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In `@relay/common/override.go` around lines 2047 - 2049, Implement the documented
token_group fallback in the override publication logic around the TokenGroup and
UserGroup fields: when TokenGroup is empty, publish UserGroup instead; preserve
an explicitly provided TokenGroup unchanged. Add a regression test covering
UserGroup set with an empty TokenGroup and verify token_group-based overrides
match.
* fix(relay): set Request.GetBody so the HTTP/2 transport can transparently retry after an upstream stream reset (QuantumNous#6249) * fix(relay): set Request.GetBody so the HTTP/2 transport can transparently retry after an upstream stream reset The outbound request body is a type-erased io.Reader over BodyStorage, so net/http cannot derive Request.GetBody (it only does so for *bytes.Reader, *bytes.Buffer and *strings.Reader). With GetBody nil, the HTTP/2 transport cannot transparently retry a request once the body has been written and the upstream resets the stream with a retryable error (REFUSED_STREAM, or a connection-level GOAWAY); the relay request then fails with: http2: Transport: cannot retry err [...] after Request.Body was written; define Request.GetBody to avoid this error This affects every relay path that goes through DoApiRequest (chat, claude, gemini, responses, embedding, image, rerank). BodyStorage (memory and disk) already implements io.Seeker, so replay support only needed wiring: - NewOutboundJSONBody additionally returns a getBody that rewinds the storage and hands out a fresh non-closing reader. The transport only calls GetBody after the previous attempt's body has been abandoned, so the rewind cannot race an in-flight read. - RelayInfo carries it in the new UpstreamRequestGetBody field, set alongside UpstreamRequestBodySize by the handlers that build storage-backed bodies. - applyUpstreamGetBody (symmetric with applyUpstreamContentLength) wires it into DoApiRequest/DoFormRequest/DoTaskApiRequest, only when req.GetBody is still nil. Also remove the hand-rolled GetBody override in DoTaskApiRequest: it returned the same already-consumed reader, so any transport-level replay would have silently sent an empty body, and it clobbered the correct snapshot-based GetBody that net/http derives from the *bytes.Reader bodies the task adaptors pass in. For non-replayable bodies GetBody now stays nil, so a retry fails loudly instead of corrupting the request. Covered by unit tests plus an end-to-end raw-frame HTTP/2 test that resets the first stream with REFUSED_STREAM after the body is written and asserts the transport transparently retries with the complete body. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(relay): hand out independent readers from GetBody (address review) Per the http.Request.GetBody contract ("returns a new copy of Body"), each call must yield a reader with its own cursor. The previous implementation rewound and reused the shared BodyStorage, so two consecutive GetBody readers would interfere with each other, and a replay could disturb the primary body's offset under extreme transport timing (e.g. attempt N's body write not yet fully abandoned when the transport builds attempt N+1). Instead of snapshotting the payload (an extra copy), add BodyStorage.NewReader, which returns an independent zero-copy reader: - memory mode: a fresh bytes.Reader over the same immutable backing array; - disk mode: a separate file descriptor over the cache file, so the transport closing a replayed body only closes that descriptor. NewOutboundJSONBody's getBody now simply hands out storage.NewReader, and once the handler releases the storage, GetBody fails with ErrStorageClosed instead of replaying stale data. Tests: interleaved reads across two replay readers and the primary body each observe exactly their own byte stream, for both the memory and the disk-backed storage; the existing GetBody and HTTP/2 retry suites still pass (h2 e2e tests flake-free with -count=20). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(relay): bind replayable metadata on pass-through requests * fix(relay): reset upstream body metadata between channels * test(relay): cover replay across retries and channel attempts * fix(relay): stop following upstream redirects --------- Co-authored-by: Claude Fable 5 <noreply@anthropic.com> * refactor(relay): move replay metadata onto request bodies * Merge commit from fork * feat(channels): refine fetched model categorization (QuantumNous#6632) * feat(channels): refine fetched model categorization * fix: channel category * fix: hy3 category * fix: test Claude/Gemini endpoints with native request format (QuantumNous#6698) * feat(rate-limit): add user critical rate limit middleware for access token and aff transfer routes * fix: 修复兑换码额度精度损失 (QuantumNous#6685) * fix: 修复兑换码额度精度损失(QuantumNous#6680) * fix(redemption): guard update data integrity * CI: enhance release synchronization workflow with optional file syncing * fix(ali): stop injecting top_p into requests that omit it (QuantumNous#6674) * fix(channels): classify Qwen TTS models correctly (QuantumNous#6711) * feat(channels): add auto-disable-only channel test mode (QuantumNous#6728) * perf(web): debounce server and large-list searches (QuantumNous#6727) * fix: record reasoning effort consistently in usage logs (QuantumNous#6641) * feat(relay): expose user and group context to parameter overrides (QuantumNous#6534) * fix(ollama): preserve reasoning and tool-call context (QuantumNous#6605) * fix: backend length validation (QuantumNous#5548) * feat(billing): highlight matched conditional multipliers in logs (QuantumNous#6561) * feat(billing): highlight matched conditional multipliers in usage logs * fix(billing): make request rule tracing stable and type-safe * fix(web): require confirmation before rotating access token (QuantumNous#6749) --------- Co-authored-by: Lucas <hepo.lucas@gmail.com> Co-authored-by: Claude Fable 5 <noreply@anthropic.com> Co-authored-by: CaIon <i@caion.me> Co-authored-by: RedwindA <128586631+RedwindA@users.noreply.github.com> Co-authored-by: Seefs <40468931+seefs001@users.noreply.github.com> Co-authored-by: lihu-001 <lihu9048@gmail.com> Co-authored-by: ENCHIGO <38551565+ENCHIGO@users.noreply.github.com>
Important
📝 变更描述 / Description
(简述:做了什么?为什么这样改能生效?请基于你对代码逻辑的理解来写,避免粘贴未经整理的内容)
🚀 变更类型 / Type of change
🔗 关联任务 / Related Issue
✅ 提交前检查项 / Checklist
Bug fix,我已提交或关联对应 Issue,且不会将设计取舍、预期不一致或理解偏差直接归类为 bug。📸 运行证明 / Proof of Work
(请在此粘贴截图、关键日志或测试报告,以证明变更生效)
Summary by CodeRabbit
New Features
Tests