Skip to content

perf: optimize request metadata extraction and disabled field filtering - #5009

Merged
seefs001 merged 2 commits into
QuantumNous:mainfrom
seefs001:perf/complexity-optimizer
May 22, 2026
Merged

perf: optimize request metadata extraction and disabled field filtering#5009
seefs001 merged 2 commits into
QuantumNous:mainfrom
seefs001:perf/complexity-optimizer

Conversation

@seefs001

@seefs001 seefs001 commented May 21, 2026

Copy link
Copy Markdown
Collaborator

⚠️ 提交说明 / PR Notice

Important

  • 请提供人工撰写的简洁摘要,避免直接粘贴未经整理的 AI 输出。

📝 变更描述 / Description

(简述:做了什么?为什么这样改能生效?请基于你对代码逻辑的理解来写,避免粘贴未经整理的内容)

  • 轻量提取请求里的 model/group,避免分发阶段提前完整解析大请求体。
  • 跳过无受控字段时的请求 JSON 重写,减少不必要的反序列化和重新序列化。
  • 流式响应边接收边累计 usage 估算文本,避免保存全部 chunk 后再拼接重解析。

🚀 变更类型 / Type of change

  • 🐛 Bug 修复 (Bug fix) - 请关联对应 Issue,避免将设计取舍、理解偏差或预期不一致直接归类为 bug
  • ✨ 新功能 (New feature) - 重大特性建议先通过 Issue 沟通
  • ⚡ 性能优化 / 重构 (Refactor)
  • 📝 文档更新 (Documentation)

🔗 关联任务 / Related Issue

  • Closes # (如有)

✅ 提交前检查项 / Checklist

  • 人工确认: 我已亲自整理并撰写此描述,没有直接粘贴未经处理的 AI 输出。
  • 非重复提交: 我已搜索现有的 IssuesPRs,确认不是重复提交。
  • Bug fix 说明: 若此 PR 标记为 Bug fix,我已提交或关联对应 Issue,且不会将设计取舍、预期不一致或理解偏差直接归类为 bug。
  • 变更理解: 我已理解这些更改的工作原理及可能影响。
  • 范围聚焦: 本 PR 未包含任何与当前任务无关的代码改动。
  • 本地验证: 已在本地运行并通过测试或手动验证,维护者可以据此复核结果。
  • 安全合规: 代码中无敏感凭据,且符合项目代码规范。

📸 运行证明 / Proof of Work

(请在此粘贴截图、关键日志或测试报告,以证明变更生效)

Summary by CodeRabbit

Release Notes

  • Performance

    • Optimized streaming response processing with incremental token handling
    • Improved request field validation efficiency
  • Bug Fixes

    • Enhanced JSON request body parsing for reliable model selection
  • Tests

    • Added comprehensive test coverage for request handling scenarios

Review Change Stack

@coderabbitai

coderabbitai Bot commented May 21, 2026

Copy link
Copy Markdown
Contributor

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info
⚙️ Run configuration

Configuration used: Organization UI

Review profile: CHILL

Plan: Pro

Run ID: caa5775e-77b6-469e-bd86-c787eb046c76

📥 Commits

Reviewing files that changed from the base of the PR and between 006e801 and ef617d5.

📒 Files selected for processing (5)
  • middleware/distributor.go
  • relay/channel/openai/helper.go
  • relay/channel/openai/relay-openai.go
  • relay/common/override_test.go
  • relay/common/relay_info.go

Walkthrough

This PR optimizes JSON request handling across the middleware and relay layers. It adds JSON body parsing for model selection in the distributor, refactors streaming token processing from batch buffering to incremental per-chunk parsing, and adds an early-return optimization for request field removal checks based on channel settings.

Changes

JSON and Stream Processing Improvements

Layer / File(s) Summary
JSON Request Body Parsing for Model Selection
middleware/distributor.go
getModelFromRequest now detects application/json content-type and delegates to new getModelFromJSONBody, which validates JSON, extracts model and group fields using gjson, enforces string types, and rewinds the body for downstream handlers. Adds io and gjson imports.
Stream Token Processing Optimization
relay/channel/openai/helper.go, relay/channel/openai/relay-openai.go
Replaces batch-oriented stream helpers (processTokens, processChatCompletions, processCompletions) with incremental processTokenData that unmarshals each chunk on arrival. Removes streamItems buffering and post-stream processing; OaiStreamHandler now calls processTokenData inside the scan loop to accumulate into responseTextBuilder and toolCount.
Request Body Field Removal Optimization and Tests
relay/common/relay_info.go, relay/common/override_test.go
Adds hasRemovableDisabledField helper using gjson to check existence of removable fields (service_tier, inference_geo, speed, store, safety_identifier, stream_options.include_obfuscation) against permission flags. RemoveDisabledFields now returns early without unmarshalling when no removable fields are present. Includes test asserting no mutation when no controlled fields exist.

Estimated code review effort

🎯 3 (Moderate) | ⏱️ ~22 minutes

Poem

🐰 Hopping through JSON with gjson so fast,
Stream tokens no longer need buffering past,
Early returns save the day with a gleam,
Parsing and processing—a well-oiled team!
Bytes rewind smoothly, requests flow clean. 🌙

🚥 Pre-merge checks | ✅ 4 | ❌ 1

❌ Failed checks (1 warning)

Check name Status Explanation Resolution
Docstring Coverage ⚠️ Warning Docstring coverage is 20.00% which is insufficient. The required threshold is 80.00%. Write docstrings for the functions missing them to satisfy the coverage threshold.
✅ Passed checks (4 passed)
Check name Status Explanation
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check ✅ Passed The PR title accurately summarizes the main changes: optimizing request metadata extraction and disabled field filtering across multiple files.
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.

✏️ Tip: You can configure your own custom pre-merge checks in the settings.

✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands and usage tips.

@seefs001
seefs001 merged commit ae6a033 into QuantumNous:main May 22, 2026
2 checks passed
yiranxiaohui added a commit to yiranxiaohui/new-api that referenced this pull request May 25, 2026
Range: 18282e6..3b9ed0a8 (upstream/main as of fetch)

Highlights:
- feat: support request_header key source (QuantumNous#4903)
- feat: Waffo Pancake gateway + admin catalog binding (QuantumNous#4935)
- perf: optimize request metadata extraction, drop dead batch
  helpers in relay/channel/openai/helper.go (QuantumNous#5009)
- perf: reduce heap residency for large base64 relay requests
- fix(channel): evict auto-disabled multi-key channels from cache (QuantumNous#4983)
- fix: resolve model owned_by from active channels (QuantumNous#4416) — introduces
  channelOwnerName/getPreferredModelOwners/buildOpenAIModel + ListModels refactor
- fix: GetAllChannels respects group filter (QuantumNous#4847, QuantumNous#4885)
- fix(auth): expose register_enabled, aff_code, localize reset (QuantumNous#4871, QuantumNous#4945, QuantumNous#4769)
- fix(webhook): processing + Waffo subscription compliance (QuantumNous#5047, QuantumNous#5038)
- refactor(ui): system settings drill-in sidebar + log filter responsiveness

Conflicts resolved:
- controller/model.go: kept local hiddenMappedModels filter
  (resolveAccessibleModelGroups + getHiddenMappedModelNamesForGroups)
  on top of upstream's ListModels refactor; adopted upstream
  channelOwnerName helper.
- relay/channel/openai/helper.go: adopted upstream (HEAD's
  processChatCompletions/processCompletions were dead code after
  upstream's perf refactor in QuantumNous#5009).

Local patches verified intact: Username + fillTopUpUsernames (model/topup.go,
locked by topup_username_test.go), HideUpstreamErrors, Claude developer-role
normalization, Gemini role fallback, Model Chat header nav entry,
channel affinity auto-clear.

Note: go build not run (no Go toolchain in this environment); CI to verify.

Generated with [Claude Code](https://claude.ai/code)
via [Happy](https://happy.engineering)

Co-Authored-By: Claude <noreply@anthropic.com>
Co-Authored-By: Happy <yesreply@happy.engineering>
xyfacai pushed a commit to xyfacai/new-api that referenced this pull request May 30, 2026
…ng (QuantumNous#5009)

* perf: optimize request metadata extraction and disabled field filtering

* perf: optimize stream usage estimation path
isboyjc added a commit to isboyjc/amux-api that referenced this pull request May 31, 2026
Sync upstream QuantumNous/new-api QuantumNous#5009 (ae6a033).
- distributor: gjson fast-path to read model/group from JSON bodies,
  avoiding a full struct unmarshal on the hot routing path; body is
  seeked back and reset so downstream re-reads are unaffected. Non-JSON
  content types keep the UnmarshalBodyReusable path.
- relay_info: RemoveDisabledFields early-outs via a gjson presence check,
  skipping the unmarshal/marshal round-trip when no controlled field is
  present.
- openai stream: parse each SSE chunk incrementally (processTokenData)
  instead of buffering all items and bulk-parsing at the end, cutting
  heap residency; token text / toolCount accounting is unchanged.
Rewrote the added test to the file's existing style (no testify).
SamuelSxy pushed a commit to SamuelSxy/new-api-rh that referenced this pull request Jun 7, 2026
…ng (QuantumNous#5009)

* perf: optimize request metadata extraction and disabled field filtering

* perf: optimize stream usage estimation path
fx247562340 pushed a commit to fx247562340/vancine-platform that referenced this pull request Jun 11, 2026
…ng (QuantumNous#5009)

* perf: optimize request metadata extraction and disabled field filtering

* perf: optimize stream usage estimation path
330079598 pushed a commit to 330079598/new-api that referenced this pull request Aug 19, 2026
…ng (QuantumNous#5009)

* perf: optimize request metadata extraction and disabled field filtering

* perf: optimize stream usage estimation path
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant