feat: openai response /v1/response/compact - #2644
Conversation
WalkthroughAdds a new "responses compact" feature: POST /v1/responses/compact route, OpenAPI schemas and DTOs, upstream adaptor/handler for compact responses, compact-model suffix and ratio logic, model-mapping/middleware adjustments, and compact-specific pricing/quota wiring. Changes
Sequence DiagramsequenceDiagram
participant Client as Client
participant Router as Router
participant Controller as Controller
participant Validator as Validator
participant RelayInfo as RelayInfo
participant Adaptor as UpstreamAdaptor
participant Upstream as OpenAI
participant Handler as CompactionHandler
participant Billing as Billing/Quota
Client->>Router: POST /v1/responses/compact
Router->>Controller: Relay(format=openai_responses_compaction)
Controller->>Validator: GetAndValidateResponsesCompactionRequest
Validator-->>Controller: OpenAIResponsesCompactionRequest
Controller->>RelayInfo: GenRelayInfoResponsesCompaction
Controller->>Adaptor: Forward request (mapped model / compact suffix)
Adaptor->>Upstream: /backend-api/.../responses/compact
Upstream-->>Handler: Compaction response (id, output, usage)
Handler->>Handler: Unmarshal & check OpenAI error
Handler->>Billing: Post quota / pricing (usage)
Billing-->>Handler: Quota result
Handler-->>Controller: Usage + response forwarded
Controller-->>Client: Return compaction response + usage
Estimated code review effort🎯 4 (Complex) | ⏱️ ~45 minutes Possibly related PRs
Suggested reviewers
Poem
🚥 Pre-merge checks | ✅ 1 | ❌ 2❌ Failed checks (2 warnings)
✅ Passed checks (1 passed)
✏️ Tip: You can configure your own custom pre-merge checks in the settings. ✨ Finishing touches
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
There was a problem hiding this comment.
Actionable comments posted: 1
🤖 Fix all issues with AI agents
In @relay/responses_handler.go:
- Around line 158-162: When refunding pre-consumed quota the code currently
ignores errors from service.PostConsumeQuota, which can permanently lose user
quota; change the block that calls PostConsumeQuota (using relayInfoCopy and
FinalPreConsumedQuota) to capture the returned error and log it (using the
module's logger, e.g., processLogger or the existing logger instance) instead of
discarding it, and only clear info.FinalPreConsumedQuota after a successful
refund or record the failure for retry/alerting so the failed refund is
observable.
🧹 Nitpick comments (4)
dto/openai_responses_compaction_request.go (1)
19-30: Token count includes raw JSON syntax, which may inflate estimates.
string(r.Instructions)andstring(r.Input)include the raw JSON representation (quotes, brackets, etc.), not the actual text content. This could lead to inaccurate token count estimates if these fields contain complex JSON structures.If precise token counting is important for billing or quota management, consider unmarshaling the JSON to extract actual text content. However, if this is just an estimate for pre-flight checks, the current approach may be acceptable.
relay/helper/valid_request.go (1)
130-139: Consider validating that at least one ofInputorPreviousResponseIDis provided.The validation only checks for
Model, but bothInputandPreviousResponseIDare optional in the DTO. A compaction request should have at least one of these fields to be meaningful. This pattern aligns withGetAndValidateResponsesRequestwhich requiresInput.♻️ Optional: Add validation for required input fields
func GetAndValidateResponsesCompactionRequest(c *gin.Context) (*dto.OpenAIResponsesCompactionRequest, error) { request := &dto.OpenAIResponsesCompactionRequest{} if err := common.UnmarshalBodyReusable(c, request); err != nil { return nil, err } if request.Model == "" { return nil, errors.New("model is required") } + if len(request.Input) == 0 && request.PreviousResponseID == "" { + return nil, errors.New("input or previous_response_id is required") + } return request, nil }docs/openapi/relay.json (2)
288-326: Inconsistent indentation in new endpoint definition.The JSON uses mixed tabs and spaces for indentation (e.g., lines 288-307 use tabs, lines 308-326 use spaces). This may cause issues with some parsers or formatters and reduces maintainability.
3186-3199: Consider adding property definitions foroutputanderrorschemas.The
outputarray items anderrorobject have emptyproperties: {}, which makes the API contract less clear. Even if flexible, documenting expected fields improves API usability.Example structure based on similar schemas
"output": { "type": "array", "items": { "type": "object", - "properties": {} + "properties": { + "type": { "type": "string" }, + "id": { "type": "string" }, + "content": { "type": "array", "items": { "type": "object" } } + } } }, ... "error": { "type": "object", - "properties": {} + "properties": { + "message": { "type": "string" }, + "code": { "type": "string" } + } }
📜 Review details
Configuration used: Organization UI
Review profile: CHILL
Plan: Pro
📒 Files selected for processing (12)
controller/relay.godocs/openapi/relay.jsondto/openai_compaction.godto/openai_responses_compaction_request.gorelay/channel/openai/adaptor.gorelay/channel/openai/relay_responses_compact.gorelay/common/relay_info.gorelay/constant/relay_mode.gorelay/helper/valid_request.gorelay/responses_handler.gorouter/relay-router.gotypes/relay_format.go
🧰 Additional context used
🧬 Code graph analysis (7)
relay/channel/openai/relay_responses_compact.go (5)
dto/openai_response.go (2)
Usage(222-240)GetOpenAIError(378-417)types/error.go (5)
NewAPIError(89-98)NewOpenAIError(237-260)ErrorCodeReadResponseBodyFailed(71-71)ErrorCodeBadResponseBody(74-74)WithOpenAIError(288-318)service/http.go (2)
CloseResponseBodyGracefully(15-23)IOCopyBytesGracefully(25-61)dto/openai_compaction.go (1)
OpenAIResponsesCompactionResponse(9-16)common/json.go (1)
Unmarshal(9-11)
relay/channel/openai/adaptor.go (2)
relay/constant/relay_mode.go (1)
RelayModeResponsesCompact(54-54)relay/channel/openai/relay_responses_compact.go (1)
OaiResponsesCompactionHandler(15-44)
controller/relay.go (1)
relay/constant/relay_mode.go (2)
RelayModeResponses(48-48)RelayModeResponsesCompact(54-54)
relay/responses_handler.go (9)
relay/constant/relay_mode.go (1)
RelayModeResponsesCompact(54-54)constant/api_type.go (1)
APITypeOpenAI(4-4)dto/openai_request.go (2)
OpenAIResponsesRequest(795-820)Input(904-908)dto/openai_responses_compaction_request.go (1)
OpenAIResponsesCompactionRequest(12-17)dto/openai_response.go (1)
Usage(222-240)types/price_data.go (2)
PriceData(11-27)GroupRatioInfo(5-9)relay/helper/price.go (1)
ContainPriceOrRatio(165-175)relay/common/relay_info.go (1)
TokenCountMeta(77-80)service/quota.go (1)
PostConsumeQuota(504-533)
relay/helper/valid_request.go (3)
types/relay_format.go (1)
RelayFormatOpenAIResponsesCompaction(10-10)dto/openai_responses_compaction_request.go (1)
OpenAIResponsesCompactionRequest(12-17)common/gin.go (1)
UnmarshalBodyReusable(72-97)
router/relay-router.go (2)
controller/relay.go (1)
Relay(65-229)types/relay_format.go (1)
RelayFormatOpenAIResponsesCompaction(10-10)
relay/common/relay_info.go (3)
types/relay_format.go (2)
RelayFormatOpenAIResponsesCompaction(10-10)RelayFormat(3-3)dto/openai_responses_compaction_request.go (1)
OpenAIResponsesCompactionRequest(12-17)relay/constant/relay_mode.go (2)
RelayModeUnknown(9-9)RelayModeResponsesCompact(54-54)
🔇 Additional comments (21)
types/relay_format.go (1)
10-10: LGTM!The new constant
RelayFormatOpenAIResponsesCompactionfollows the established naming convention and is correctly positioned alongside other OpenAI-related formats.relay/constant/relay_mode.go (2)
53-54: LGTM!The new
RelayModeResponsesCompactconstant is correctly added to the enum group.
75-78: Correct path ordering for prefix matching.The more specific
/v1/responses/compactpath is correctly checked before the general/v1/responsespath, preventing incorrect matches due toHasPrefixbehavior.controller/relay.go (2)
47-48: LGTM!Routing both
RelayModeResponsesandRelayModeResponsesCompacttoResponsesHelperis appropriate, as the helper can differentiate behavior based on the relay mode internally.
244-269: Verify token count meta handling for compaction requests.
fastTokenCountMetaForPricingdoesn't explicitly handle*dto.OpenAIResponsesCompactionRequest, causing it to fall through to the default case with an emptyMaxTokens. Given the PR TODO mentions billing handling is pending, this may be intentional. Ensure this aligns with the planned billing implementation for compact mode.router/relay-router.go (1)
96-98: LGTM!The new
/v1/responses/compactroute is correctly added following the established pattern, with proper middleware inheritance from the parent router groups.relay/channel/openai/adaptor.go (2)
608-609: LGTM!The new case correctly routes
RelayModeResponsesCompacttoOaiResponsesCompactionHandler. Unlike the standard responses mode, this doesn't checkinfo.IsStream, which aligns with the handler implementation that reads the full response body synchronously.Confirm that the compact endpoint intentionally does not support streaming. If streaming should be supported in the future, a similar branching pattern as
RelayModeResponses(lines 602-607) would be needed.
139-154: Verify if Azure OpenAI API supports the/responses/compactendpoint.The Azure-specific URL handling (lines 139-154) covers
RelayModeResponseswith custom endpoint paths (/openai/v1/responsesor/openai/responses) and configurable API versions. However,RelayModeResponsesCompactlacks equivalent Azure-specific logic and would fall through to standard Azure deployment URL construction. If Azure supports the compact responses endpoint, it may require similar URL handling as the regular responses endpoint.relay/helper/valid_request.go (1)
37-38: LGTM!The new case for
RelayFormatOpenAIResponsesCompactioncorrectly routes to the dedicated validation function, following the established pattern.relay/common/relay_info.go (2)
474-478: LGTM!The type assertion and routing for
OpenAIResponsesCompactionRequestfollows the established pattern used for other request types likeRerankRequestandOpenAIResponsesRequest.
488-495: LGTM!The function correctly initializes the relay info for compaction requests. The explicit
RelayModefallback ensures proper mode assignment even whenPath2RelayModereturnsUnknown.relay/channel/openai/relay_responses_compact.go (1)
15-43: Response written before error check is complete — consider ordering.The response is copied to the client (line 31) before the usage extraction completes. If any issue occurs after
IOCopyBytesGracefully, the client has already received the response but the server might fail to properly account for billing. This is likely acceptable for this flow, but worth noting.Also, when
compactResp.Usageis nil (but no error), an emptyUsage{}is returned. Verify this is the intended behavior for billing calculations downstream.dto/openai_compaction.go (1)
9-20: LGTM!The response DTO structure is well-designed:
- Uses
json.RawMessagefor flexibleOutputhandling- Properly typed
Usagepointer for optional usage dataGetOpenAIErrorhelper follows the established pattern for error extractiondto/openai_responses_compaction_request.go (2)
12-17: LGTM!The request DTO correctly models the compaction request with appropriate JSON tags. Using
json.RawMessageforInputandInstructionsallows flexible handling of various input formats.
32-40: LGTM!
IsStreamcorrectly returnsfalsefor compaction requests (compaction is not a streaming operation), andSetModelNamefollows the established pattern with proper empty-check guard.relay/responses_handler.go (5)
25-36: LGTM: Appropriate API type validation for compact mode.The switch statement correctly gates the compact endpoint to only allow OpenAI API type, returning a clear error for unsupported types.
38-56: LGTM: Clean request type normalization.The type switch handles both
OpenAIResponsesRequestandOpenAIResponsesCompactionRequestgracefully, mapping the compaction request to the standard responses request format.
176-179: LGTM: Proper cleanup of temporary state.Restoring
OriginModelNameandPriceDataafter compact billing processing ensures no side effects leak to subsequent logic.
148-155: ThepostConsumeQuotafunction signature already supports the optional message parameter. The function is defined as:func postConsumeQuota(ctx *gin.Context, relayInfo *relaycommon.RelayInfo, usage *dto.Usage, extraContent ...string)The variadic parameter
extraContent ...stringallows zero or more message arguments, making both call patterns valid:
- Line 155:
postConsumeQuota(c, info, usageDto)✓- Line 173:
postConsumeQuota(c, info, usageDto, "Compaction free: no <model>-compact pricing configured")✓No changes needed.
136-136: Error check at lines 130-134 guards against nil errors.The error check properly prevents reaching line 136 when
newAPIErroris not nil. All examinedDoResponseimplementations (openai, zhipu, xai, etc.) maintain the invariant that if no error is returned, the usage value is always set to a non-nil pointer. The direct assertion is safe as currently written.docs/openapi/relay.json (1)
3202-3233: LGTM: Request schema aligns with DTO definition.The
ResponsesCompactionRequestschema matches the Go DTO (OpenAIResponsesCompactionRequest) with requiredmodeland optionalinput,instructions, andprevious_response_idfields.
| if info.FinalPreConsumedQuota != 0 { | ||
| relayInfoCopy := *info | ||
| _ = service.PostConsumeQuota(&relayInfoCopy, -relayInfoCopy.FinalPreConsumedQuota, 0, false) | ||
| info.FinalPreConsumedQuota = 0 | ||
| } |
There was a problem hiding this comment.
Silently ignoring quota refund error may cause user quota loss.
If PostConsumeQuota fails when refunding pre-consumed quota, the user silently loses that quota. At minimum, log the error for observability.
🔧 Proposed fix
if info.FinalPreConsumedQuota != 0 {
relayInfoCopy := *info
- _ = service.PostConsumeQuota(&relayInfoCopy, -relayInfoCopy.FinalPreConsumedQuota, 0, false)
+ if err := service.PostConsumeQuota(&relayInfoCopy, -relayInfoCopy.FinalPreConsumedQuota, 0, false); err != nil {
+ common.SysError(fmt.Sprintf("failed to refund pre-consumed quota for compaction: %v", err))
+ }
info.FinalPreConsumedQuota = 0
}🤖 Prompt for AI Agents
In @relay/responses_handler.go around lines 158 - 162, When refunding
pre-consumed quota the code currently ignores errors from
service.PostConsumeQuota, which can permanently lose user quota; change the
block that calls PostConsumeQuota (using relayInfoCopy and
FinalPreConsumedQuota) to capture the returned error and log it (using the
module's logger, e.g., processLogger or the existing logger instance) instead of
discarding it, and only clear info.FinalPreConsumedQuota after a successful
refund or record the failure for retry/alerting so the failed refund is
observable.
There was a problem hiding this comment.
Actionable comments posted: 1
🤖 Fix all issues with AI agents
In @relay/helper/model_mapped.go:
- Around line 77-79: When isResponsesCompact is false and info.IsModelMapped is
false, ensure info.UpstreamModelName falls back to info.OriginModelName instead
of leaving it empty; update the logic around
request.SetModelName(info.UpstreamModelName) in relay/helper/model_mapped.go to
compute a finalUpstreamModelName (use mappingModelName when compact, prefer
info.UpstreamModelName when mapped and non-empty, otherwise set
info.UpstreamModelName = info.OriginModelName) and also set info.OriginModelName
= ratio_setting.WithCompactModelSuffix(finalUpstreamModelName) when compact;
then call request.SetModelName(finalUpstreamModelName).
🧹 Nitpick comments (1)
relay/responses_handler.go (1)
137-152: Consider removing redundant state restoration.Lines 149-150 restore
OriginModelNameandPriceDataimmediately before returning. Since the function exits right after, this restoration has no observable effect.If this is defensive coding for future changes, consider adding a comment. Otherwise, these lines can be removed.
♻️ Suggested simplification
postConsumeQuota(c, info, usageDto) - - info.OriginModelName = originModelName - info.PriceData = originPriceData return nil }
📜 Review details
Configuration used: Organization UI
Review profile: CHILL
Plan: Pro
📒 Files selected for processing (5)
middleware/distributor.gorelay/helper/model_mapped.gorelay/responses_handler.gosetting/ratio_setting/compact_suffix.gosetting/ratio_setting/model_ratio.go
🧰 Additional context used
🧠 Learnings (1)
📚 Learning: 2025-08-05T17:14:17.246Z
Learnt from: neotf
Repo: QuantumNous/new-api PR: 1511
File: setting/ratio_setting/model_ratio.go:118-123
Timestamp: 2025-08-05T17:14:17.246Z
Learning: Claude models handle "-thinking" variants differently from Gemini models. For Claude models, only the base model (without "-thinking") gets an entry in defaultModelRatio map. The "-thinking" variants rely on the Claude relay handler stripping the suffix using strings.TrimSuffix(textRequest.Model, "-thinking") before looking up the ratio, so they automatically use the base model's ratio.
Applied to files:
setting/ratio_setting/compact_suffix.gosetting/ratio_setting/model_ratio.go
🧬 Code graph analysis (4)
middleware/distributor.go (1)
setting/ratio_setting/compact_suffix.go (1)
WithCompactModelSuffix(8-13)
relay/responses_handler.go (10)
relay/constant/relay_mode.go (1)
RelayModeResponsesCompact(54-54)constant/api_type.go (1)
APITypeOpenAI(4-4)dto/openai_request.go (2)
OpenAIResponsesRequest(795-820)Input(904-908)dto/request_common.go (1)
Request(8-12)dto/openai_responses_compaction_request.go (1)
OpenAIResponsesCompactionRequest(12-17)dto/openai_response.go (1)
Usage(222-240)types/price_data.go (1)
PriceData(11-27)relay/helper/price.go (1)
ModelPriceHelper(48-140)relay/common/relay_info.go (1)
TokenCountMeta(77-80)service/quota.go (1)
PostAudioConsumeQuota(377-478)
relay/helper/model_mapped.go (4)
relay/common/relay_info.go (2)
RelayInfo(82-130)ChannelMeta(57-75)dto/request_common.go (1)
Request(8-12)relay/constant/relay_mode.go (1)
RelayModeResponsesCompact(54-54)setting/ratio_setting/compact_suffix.go (2)
CompactModelSuffix(5-5)WithCompactModelSuffix(8-13)
setting/ratio_setting/model_ratio.go (1)
setting/ratio_setting/compact_suffix.go (2)
CompactModelSuffix(5-5)CompactWildcardModelKey(6-6)
🔇 Additional comments (7)
setting/ratio_setting/compact_suffix.go (1)
1-13: LGTM!Clean utility file with proper idempotency handling in
WithCompactModelSuffix. The wildcard key pattern aligns well with existing model suffix conventions (e.g.,-thinkingvariants mentioned in learnings).setting/ratio_setting/model_ratio.go (1)
451-456: LGTM - Compact model billing logic is correct.The fallback behavior correctly implements the PR requirements:
- First checks for a wildcard ratio (
*-openai-compact) to allow global compact pricing- Falls back to
0(free) when no wildcard is configured- Returns
ok=trueto prevent the default 37.5 SelfUseMode fallbackThis pattern aligns with how other suffix-based variants (like
-thinking) handle ratio lookups. Based on learnings, this approach is consistent with existing conventions.relay/helper/model_mapped.go (1)
69-76: LGTM - Compact mode model name handling.The logic correctly:
- Preserves the mapped model name (if any) as
finalUpstreamModelName- Sets
UpstreamModelNameto the unmapped/mapped name (without suffix) for the upstream API call- Re-applies the compact suffix to
OriginModelNamefor billing purposesThis ensures the upstream API receives the correct model name while billing uses the compact-suffixed name.
relay/responses_handler.go (3)
25-36: LGTM - API type validation for compact mode.The switch statement correctly restricts the compact endpoint to OpenAI API type only, with a clear error message for unsupported types.
38-56: LGTM - Request type handling.The type switch cleanly handles both request types, converting
OpenAIResponsesCompactionRequesttoOpenAIResponsesRequestfor uniform downstream processing. The field mapping is straightforward.
154-158: LGTM - Non-compact response handling.The existing audio/non-audio quota paths are preserved with the updated
usageDtovariable.middleware/distributor.go (1)
303-306: The code is correct and consistent. All references throughout the codebase use/v1/responses/compact(plural) — including the route definition in router/relay-router.go, path checks in relay/constant/relay_mode.go, and error messages in relay/responses_handler.go. No path discrepancy exists.
| if request != nil { | ||
| request.SetModelName(info.UpstreamModelName) | ||
| } |
There was a problem hiding this comment.
🧩 Analysis chain
🏁 Script executed:
# Find all SetModelName calls and their context
ast-grep --pattern 'SetModelName($_)'Repository: QuantumNous/new-api
Length of output: 45
🏁 Script executed:
# Read the relevant section of model_mapped.go to understand the complete logic flow
wc -l relay/helper/model_mapped.goRepository: QuantumNous/new-api
Length of output: 94
🏁 Script executed:
# Read the entire model_mapped.go file to understand the logic
cat -n relay/helper/model_mapped.goRepository: QuantumNous/new-api
Length of output: 2903
🏁 Script executed:
# Search for SetModelName implementation and where request comes from
rg -B 5 -A 5 "SetModelName" relay/Repository: QuantumNous/new-api
Length of output: 1778
🏁 Script executed:
# Check where UpstreamModelName is initialized and used
rg "UpstreamModelName\s*=" relay/ -B 2 -A 2Repository: QuantumNous/new-api
Length of output: 6974
🏁 Script executed:
# Find the Request type and SetModelName definition
fd "request\.go|dto\.go" relay/ | head -5Repository: QuantumNous/new-api
Length of output: 196
🏁 Script executed:
# Find RelayInfo struct definition
rg "type RelayInfo struct" relay/ -A 20Repository: QuantumNous/new-api
Length of output: 1263
🏁 Script executed:
# Check api_request.go to understand SetModelName
cat -n relay/channel/api_request.go | head -60Repository: QuantumNous/new-api
Length of output: 2127
🏁 Script executed:
# Search for SetModelName implementation
rg "func.*SetModelName" relay/ -A 5Repository: QuantumNous/new-api
Length of output: 45
🏁 Script executed:
# Look for the Request interface/struct definition and SetModelName
rg "SetModelName" dto/ -B 2 -A 3Repository: QuantumNous/new-api
Length of output: 3113
Use OriginModelName as fallback when no mapping occurs.
When isResponsesCompact is false and IsModelMapped is false, UpstreamModelName remains empty. While SetModelName("") is safely handled by guard clauses in all implementations and won't clear the model, this pattern is inconsistent with the approach in relay/common/relay_info.go (line 172), which explicitly uses OriginModelName as a fallback. For clarity and consistency, consider:
if isResponsesCompact {
finalUpstreamModelName := mappingModelName
if info.IsModelMapped && info.UpstreamModelName != "" {
finalUpstreamModelName = info.UpstreamModelName
}
info.UpstreamModelName = finalUpstreamModelName
info.OriginModelName = ratio_setting.WithCompactModelSuffix(finalUpstreamModelName)
} else if info.UpstreamModelName == "" {
info.UpstreamModelName = info.OriginModelName
}
🤖 Prompt for AI Agents
In @relay/helper/model_mapped.go around lines 77 - 79, When isResponsesCompact
is false and info.IsModelMapped is false, ensure info.UpstreamModelName falls
back to info.OriginModelName instead of leaving it empty; update the logic
around request.SetModelName(info.UpstreamModelName) in
relay/helper/model_mapped.go to compute a finalUpstreamModelName (use
mappingModelName when compact, prefer info.UpstreamModelName when mapped and
non-empty, otherwise set info.UpstreamModelName = info.OriginModelName) and also
set info.OriginModelName =
ratio_setting.WithCompactModelSuffix(finalUpstreamModelName) when compact; then
call request.SetModelName(finalUpstreamModelName).
# Conflicts: # relay/common/relay_info.go
There was a problem hiding this comment.
Actionable comments posted: 1
🤖 Fix all issues with AI agents
In `@relay/common/relay_info.go`:
- Around line 484-488: The compaction branch currently returns early and
prevents the common InitRequestConversionChain from running; change this branch
to mirror the others by assigning info := GenRelayInfoResponsesCompaction(c,
request) (after type-asserting request to *dto.OpenAIResponsesCompactionRequest)
instead of returning, so execution falls through to the shared initialization
logic (and remove the early return path that returns the error); keep the
type-check and still return an error only if the assertion fails.
| case types.RelayFormatOpenAIResponsesCompaction: | ||
| if request, ok := request.(*dto.OpenAIResponsesCompactionRequest); ok { | ||
| return GenRelayInfoResponsesCompaction(c, request), nil | ||
| } | ||
| return nil, errors.New("request is not a OpenAIResponsesCompactionRequest") |
There was a problem hiding this comment.
Initialize RequestConversionChain for compaction paths.
This branch returns early, so InitRequestConversionChain() never runs and the original relay format won’t be recorded. Align with the other formats by setting info and falling through to the common init.
💡 Suggested fix
case types.RelayFormatOpenAIResponsesCompaction:
if request, ok := request.(*dto.OpenAIResponsesCompactionRequest); ok {
- return GenRelayInfoResponsesCompaction(c, request), nil
+ info = GenRelayInfoResponsesCompaction(c, request)
+ break
}
- return nil, errors.New("request is not a OpenAIResponsesCompactionRequest")
+ err = errors.New("request is not a OpenAIResponsesCompactionRequest")🤖 Prompt for AI Agents
In `@relay/common/relay_info.go` around lines 484 - 488, The compaction branch
currently returns early and prevents the common InitRequestConversionChain from
running; change this branch to mirror the others by assigning info :=
GenRelayInfoResponsesCompaction(c, request) (after type-asserting request to
*dto.OpenAIResponsesCompactionRequest) instead of returning, so execution falls
through to the shared initialization logic (and remove the early return path
that returns the error); keep the type-check and still return an error only if
the assertion fails.
There was a problem hiding this comment.
Actionable comments posted: 1
Caution
Some comments are outside the diff and can’t be posted inline due to platform limitations.
⚠️ Outside diff range comments (1)
relay/channel/codex/adaptor.go (1)
104-113: Route compact mode through the compaction handler.Right now compact requests still go through the standard responses handler, which can mis-parse response shape/usage if compaction differs. Consider branching to the compaction handler (and decide how streaming should behave for compact).
🛠️ Proposed fix
- if info.IsStream { - return openai.OaiResponsesStreamHandler(c, info, resp) - } - return openai.OaiResponsesHandler(c, info, resp) + if info.IsStream { + return openai.OaiResponsesStreamHandler(c, info, resp) + } + if info.RelayMode == relayconstant.RelayModeResponsesCompact { + return openai.OaiResponsesCompactionHandler(c, info, resp) + } + return openai.OaiResponsesHandler(c, info, resp)
🤖 Fix all issues with AI agents
In `@relay/responses_handler.go`:
- Around line 137-159: The code currently does an unchecked type assertion
usage.(*dto.Usage) which can panic; change the extraction of usage into a
guarded check (e.g., usageDto, ok := usage.(*dto.Usage)) and if !ok or usage ==
nil return a typed error (use types.NewError with an appropriate ErrorCode)
instead of proceeding; ensure both branches that call postConsumeQuota and
service.PostAudioConsumeQuota only run when usageDto is valid and preserve the
existing restore of info.OriginModelName/info.PriceData behavior.
🧹 Nitpick comments (1)
relay/responses_handler.go (1)
38-55: Centralize compaction → responses mapping to avoid field drift.As the compaction DTO evolves, manual field copies can miss new fields. Consider a helper (e.g.,
ToResponsesRequest()) ondto.OpenAIResponsesCompactionRequestand call it here.
| usageDto := usage.(*dto.Usage) | ||
| if info.RelayMode == relayconstant.RelayModeResponsesCompact { | ||
| originModelName := info.OriginModelName | ||
| originPriceData := info.PriceData | ||
|
|
||
| _, err := helper.ModelPriceHelper(c, info, info.GetEstimatePromptTokens(), &types.TokenCountMeta{}) | ||
| if err != nil { | ||
| info.OriginModelName = originModelName | ||
| info.PriceData = originPriceData | ||
| return types.NewError(err, types.ErrorCodeModelPriceError, types.ErrOptionWithSkipRetry()) | ||
| } | ||
| postConsumeQuota(c, info, usageDto) | ||
|
|
||
| info.OriginModelName = originModelName | ||
| info.PriceData = originPriceData | ||
| return nil | ||
| } | ||
|
|
||
| if strings.HasPrefix(info.OriginModelName, "gpt-4o-audio") { | ||
| service.PostAudioConsumeQuota(c, info, usage.(*dto.Usage), "") | ||
| service.PostAudioConsumeQuota(c, info, usageDto, "") | ||
| } else { | ||
| postConsumeQuota(c, info, usage.(*dto.Usage)) | ||
| postConsumeQuota(c, info, usageDto) | ||
| } |
There was a problem hiding this comment.
Guard the usage type assertion to avoid panics.
If the adaptor returns a nil or non-*dto.Usage payload (especially for compact or streaming flows), this will panic. Add an ok check and return a typed error instead.
🛠️ Proposed fix
- usageDto := usage.(*dto.Usage)
+ usageDto, ok := usage.(*dto.Usage)
+ if !ok || usageDto == nil {
+ return types.NewError(
+ fmt.Errorf("invalid usage payload type: %T", usage),
+ types.ErrorCodeInvalidRequest,
+ types.ErrOptionWithSkipRetry(),
+ )
+ }📝 Committable suggestion
‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.
| usageDto := usage.(*dto.Usage) | |
| if info.RelayMode == relayconstant.RelayModeResponsesCompact { | |
| originModelName := info.OriginModelName | |
| originPriceData := info.PriceData | |
| _, err := helper.ModelPriceHelper(c, info, info.GetEstimatePromptTokens(), &types.TokenCountMeta{}) | |
| if err != nil { | |
| info.OriginModelName = originModelName | |
| info.PriceData = originPriceData | |
| return types.NewError(err, types.ErrorCodeModelPriceError, types.ErrOptionWithSkipRetry()) | |
| } | |
| postConsumeQuota(c, info, usageDto) | |
| info.OriginModelName = originModelName | |
| info.PriceData = originPriceData | |
| return nil | |
| } | |
| if strings.HasPrefix(info.OriginModelName, "gpt-4o-audio") { | |
| service.PostAudioConsumeQuota(c, info, usage.(*dto.Usage), "") | |
| service.PostAudioConsumeQuota(c, info, usageDto, "") | |
| } else { | |
| postConsumeQuota(c, info, usage.(*dto.Usage)) | |
| postConsumeQuota(c, info, usageDto) | |
| } | |
| usageDto, ok := usage.(*dto.Usage) | |
| if !ok || usageDto == nil { | |
| return types.NewError( | |
| fmt.Errorf("invalid usage payload type: %T", usage), | |
| types.ErrorCodeInvalidRequest, | |
| types.ErrOptionWithSkipRetry(), | |
| ) | |
| } | |
| if info.RelayMode == relayconstant.RelayModeResponsesCompact { | |
| originModelName := info.OriginModelName | |
| originPriceData := info.PriceData | |
| _, err := helper.ModelPriceHelper(c, info, info.GetEstimatePromptTokens(), &types.TokenCountMeta{}) | |
| if err != nil { | |
| info.OriginModelName = originModelName | |
| info.PriceData = originPriceData | |
| return types.NewError(err, types.ErrorCodeModelPriceError, types.ErrOptionWithSkipRetry()) | |
| } | |
| postConsumeQuota(c, info, usageDto) | |
| info.OriginModelName = originModelName | |
| info.PriceData = originPriceData | |
| return nil | |
| } | |
| if strings.HasPrefix(info.OriginModelName, "gpt-4o-audio") { | |
| service.PostAudioConsumeQuota(c, info, usageDto, "") | |
| } else { | |
| postConsumeQuota(c, info, usageDto) | |
| } |
🤖 Prompt for AI Agents
In `@relay/responses_handler.go` around lines 137 - 159, The code currently does
an unchecked type assertion usage.(*dto.Usage) which can panic; change the
extraction of usage into a guarded check (e.g., usageDto, ok :=
usage.(*dto.Usage)) and if !ok or usage == nil return a typed error (use
types.NewError with an appropriate ErrorCode) instead of proceeding; ensure both
branches that call postConsumeQuota and service.PostAudioConsumeQuota only run
when usageDto is valid and preserve the existing restore of
info.OriginModelName/info.PriceData behavior.
There was a problem hiding this comment.
Actionable comments posted: 1
Caution
Some comments are outside the diff and can’t be posted inline due to platform limitations.
⚠️ Outside diff range comments (1)
setting/ratio_setting/model_ratio.go (1)
458-468: Compact models without wildcard ratio incorrectly mark models as "containing ratio" in validation.The
ContainPriceOrRatiofunction (relay/helper/price.go:166-173) returnstruebased solely on theokflag, without checking the ratio value. When no compact wildcard is configured,GetModelRatioreturns(0, true, name), causing unconfigured compact models to pass the "model has price/ratio" check. This is inconsistent with non-compact models, which returnokbased onoperation_setting.SelfUseModeEnabled. Theok=truereturn for compact models with ratio0is semantically misleading and allows unconfigured compact models to bypass ratio validation when they shouldn't.
🤖 Fix all issues with AI agents
In `@setting/ratio_setting/model_ratio.go`:
- Around line 412-423: The compact suffix check is run after
FormatMatchingModelName which can normalize away the suffix and bypass compact
pricing; change the logic in the block that uses FormatMatchingModelName,
CompactModelSuffix, CompactWildcardModelKey and modelPriceMap so you first
detect and temporarily strip/record the raw suffix from the incoming name (e.g.,
store rawName and hasCompactSuffix), then call FormatMatchingModelName on the
base name, then when resolving price: 1) check modelPriceMap for the explicit
compact key (rawName + CompactModelSuffix) if hasCompactSuffix, 2) if not found
and hasCompactSuffix check the CompactWildcardModelKey price, and 3) otherwise
fall back to the normalized name lookup — this preserves explicit compact
entries and only uses the wildcard compact price when no explicit compact key
exists.
| name = FormatMatchingModelName(name) | ||
|
|
||
| if strings.HasSuffix(name, CompactModelSuffix) { | ||
| price, ok := modelPriceMap[CompactWildcardModelKey] | ||
| if !ok { | ||
| if printErr { | ||
| common.SysError("model price not found: " + name) | ||
| } | ||
| return -1, false | ||
| } | ||
| return price, true | ||
| } |
There was a problem hiding this comment.
Compact suffix can be stripped by normalization, bypassing compact pricing.
FormatMatchingModelName runs before the compact check, which can rewrite names like gpt-4-gizmo-openai-compact into gpt-4-gizmo-* and drop the suffix. That skips the compact branch and charges the base model. Also, the compact branch runs before modelPriceMap[name], so explicit compact prices are never used. Consider detecting/stripping the suffix before normalization, then re‑attaching it (or checking the raw name) and only falling back to the wildcard if no explicit compact key is present.
💡 Possible fix (preserve suffix + allow explicit compact key)
- name = FormatMatchingModelName(name)
-
- if strings.HasSuffix(name, CompactModelSuffix) {
- price, ok := modelPriceMap[CompactWildcardModelKey]
+ isCompact := strings.HasSuffix(name, CompactModelSuffix)
+ if isCompact {
+ name = strings.TrimSuffix(name, CompactModelSuffix)
+ }
+ name = FormatMatchingModelName(name)
+ if isCompact {
+ name = name + CompactModelSuffix
+ }
+ if price, ok := modelPriceMap[name]; ok {
+ return price, true
+ }
+ if isCompact {
+ price, ok := modelPriceMap[CompactWildcardModelKey]
if !ok {
if printErr {
common.SysError("model price not found: " + name)
}
return -1, false
}
return price, true
}🤖 Prompt for AI Agents
In `@setting/ratio_setting/model_ratio.go` around lines 412 - 423, The compact
suffix check is run after FormatMatchingModelName which can normalize away the
suffix and bypass compact pricing; change the logic in the block that uses
FormatMatchingModelName, CompactModelSuffix, CompactWildcardModelKey and
modelPriceMap so you first detect and temporarily strip/record the raw suffix
from the incoming name (e.g., store rawName and hasCompactSuffix), then call
FormatMatchingModelName on the base name, then when resolving price: 1) check
modelPriceMap for the explicit compact key (rawName + CompactModelSuffix) if
hasCompactSuffix, 2) if not found and hasCompactSuffix check the
CompactWildcardModelKey price, and 3) otherwise fall back to the normalized name
lookup — this preserves explicit compact entries and only uses the wildcard
compact price when no explicit compact key exists.
# Conflicts: # common/endpoint_defaults.go
主要更新: - feat: 磁盘请求体缓存 (QuantumNous#2780) - feat: OpenAI Response API /v1/response/compact (QuantumNous#2644) - feat: 渠道亲和性 (Channel Affinity) (QuantumNous#2669) - feat: Codex渠道支持 (QuantumNous#2652) - feat: Claude/Grok refusal reason显示 - feat: 性能监控和GC控制API - fix: 用户配额获取逻辑 (QuantumNous#2749) - fix: Gemini多工具调用索引问题 - fix: 错误时仍本地计费 冲突解决: - README.md: 保留中文版本 - go.mod: 采用上游较新版本 Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
* feat: openai response /v1/response/compact * feat: /v1/response/compact bill * feat: /v1/response/compact * feat: /v1/responses/compact -> codex channel * feat: /v1/responses/compact -> codex channel * feat: /v1/responses/compact -> codex channel * feat: codex channel default models * feat: compact model price * feat: /v1/responses/comapct test
2 participants
#2418
需要某个渠道模型列表里有对应的模型才可以调用，例如调用的是gpt-5.2-codex，那么就需要加入gpt-5.2-codex-openai-compact作为模型名。
支持设置价格（固定价格和按倍率）
"*-openai-compact": 0.01
如果不设置默认为免费，倍率为0，但是会显示在日志中。
当前只支持OpenAI渠道和Codex渠道，Codex已经预设好带有-openai-compact后缀的模型名。
Summary by CodeRabbit
New Features
Improvements
Documentation
Chores
✏️ Tip: You can customize this high-level summary in your review settings.