perf: optimize request metadata extraction and disabled field filtering - #5009
Conversation
|
No actionable comments were generated in the recent review. 🎉 ℹ️ Recent review info⚙️ Run configurationConfiguration used: Organization UI Review profile: CHILL Plan: Pro Run ID: 📒 Files selected for processing (5)
WalkthroughThis PR optimizes JSON request handling across the middleware and relay layers. It adds JSON body parsing for model selection in the distributor, refactors streaming token processing from batch buffering to incremental per-chunk parsing, and adds an early-return optimization for request field removal checks based on channel settings. ChangesJSON and Stream Processing Improvements
Estimated code review effort🎯 3 (Moderate) | ⏱️ ~22 minutes Poem
🚥 Pre-merge checks | ✅ 4 | ❌ 1❌ Failed checks (1 warning)
✅ Passed checks (4 passed)
✏️ Tip: You can configure your own custom pre-merge checks in the settings. ✨ Finishing Touches🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
Range: 18282e6..3b9ed0a8 (upstream/main as of fetch) Highlights: - feat: support request_header key source (QuantumNous#4903) - feat: Waffo Pancake gateway + admin catalog binding (QuantumNous#4935) - perf: optimize request metadata extraction, drop dead batch helpers in relay/channel/openai/helper.go (QuantumNous#5009) - perf: reduce heap residency for large base64 relay requests - fix(channel): evict auto-disabled multi-key channels from cache (QuantumNous#4983) - fix: resolve model owned_by from active channels (QuantumNous#4416) — introduces channelOwnerName/getPreferredModelOwners/buildOpenAIModel + ListModels refactor - fix: GetAllChannels respects group filter (QuantumNous#4847, QuantumNous#4885) - fix(auth): expose register_enabled, aff_code, localize reset (QuantumNous#4871, QuantumNous#4945, QuantumNous#4769) - fix(webhook): processing + Waffo subscription compliance (QuantumNous#5047, QuantumNous#5038) - refactor(ui): system settings drill-in sidebar + log filter responsiveness Conflicts resolved: - controller/model.go: kept local hiddenMappedModels filter (resolveAccessibleModelGroups + getHiddenMappedModelNamesForGroups) on top of upstream's ListModels refactor; adopted upstream channelOwnerName helper. - relay/channel/openai/helper.go: adopted upstream (HEAD's processChatCompletions/processCompletions were dead code after upstream's perf refactor in QuantumNous#5009). Local patches verified intact: Username + fillTopUpUsernames (model/topup.go, locked by topup_username_test.go), HideUpstreamErrors, Claude developer-role normalization, Gemini role fallback, Model Chat header nav entry, channel affinity auto-clear. Note: go build not run (no Go toolchain in this environment); CI to verify. Generated with [Claude Code](https://claude.ai/code) via [Happy](https://happy.engineering) Co-Authored-By: Claude <noreply@anthropic.com> Co-Authored-By: Happy <yesreply@happy.engineering>
…ng (QuantumNous#5009) * perf: optimize request metadata extraction and disabled field filtering * perf: optimize stream usage estimation path
Sync upstream QuantumNous/new-api QuantumNous#5009 (ae6a033). - distributor: gjson fast-path to read model/group from JSON bodies, avoiding a full struct unmarshal on the hot routing path; body is seeked back and reset so downstream re-reads are unaffected. Non-JSON content types keep the UnmarshalBodyReusable path. - relay_info: RemoveDisabledFields early-outs via a gjson presence check, skipping the unmarshal/marshal round-trip when no controlled field is present. - openai stream: parse each SSE chunk incrementally (processTokenData) instead of buffering all items and bulk-parsing at the end, cutting heap residency; token text / toolCount accounting is unchanged. Rewrote the added test to the file's existing style (no testify).
…ng (QuantumNous#5009) * perf: optimize request metadata extraction and disabled field filtering * perf: optimize stream usage estimation path
…ng (QuantumNous#5009) * perf: optimize request metadata extraction and disabled field filtering * perf: optimize stream usage estimation path
…ng (QuantumNous#5009) * perf: optimize request metadata extraction and disabled field filtering * perf: optimize stream usage estimation path
Important
📝 变更描述 / Description
(简述:做了什么?为什么这样改能生效?请基于你对代码逻辑的理解来写,避免粘贴未经整理的内容)
🚀 变更类型 / Type of change
🔗 关联任务 / Related Issue
✅ 提交前检查项 / Checklist
Bug fix,我已提交或关联对应 Issue,且不会将设计取舍、预期不一致或理解偏差直接归类为 bug。📸 运行证明 / Proof of Work
(请在此粘贴截图、关键日志或测试报告,以证明变更生效)
Summary by CodeRabbit
Release Notes
Performance
Bug Fixes
Tests