Skip to content

fix: fix the proxyURL is empty, not using the default HTTP client configuration && the AWS calling side did not apply the relay timeout. #2580

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
Calcium-Ion merged 1 commit into from
Jan 5, 2026
Merged

fix: fix the proxyURL is empty, not using the default HTTP client configuration && the AWS calling side did not apply the relay timeout. #2580

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
25 changes: 22 additions & 3 deletions relay/channel/aws/relay-aws.go
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,11 +1,13 @@
package aws

import (
"context"
"encoding/json"
"fmt"
"io"
"net/http"
"strings"
"time"

"github.com/QuantumNous/new-api/common"
"github.com/QuantumNous/new-api/dto"
Expand Down Expand Up @@ -37,6 +39,13 @@ func getAwsErrorStatusCode(err error) int {
return http.StatusInternalServerError
}

func newAwsInvokeContext() (context.Context, context.CancelFunc) {
if common.RelayTimeout <= 0 {
return context.Background(), func() {}
}
return context.WithTimeout(context.Background(), time.Duration(common.RelayTimeout)*time.Second)
}
Comment on lines +42 to +47

@coderabbitai coderabbitai Bot Jan 5, 2026 •
edited
Loading

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Potential issue | 🔴 Critical

Critical: Context should inherit from request context.

The function creates a new context from context.Background(), which breaks request cancellation propagation. When a client disconnects or cancels their request, the AWS SDK calls will continue executing, leading to:

  • Wasted AWS API invocations and costs
  • Resource leaks (uncancelled network connections)
  • Delayed error responses
🔎 Proposed fix to inherit from request context

The context should be created from the HTTP request context to properly propagate cancellations:

-func newAwsInvokeContext() (context.Context, context.CancelFunc) {
+func newAwsInvokeContext(c *gin.Context) (context.Context, context.CancelFunc) {
+	reqCtx := c.Request.Context()
 	if common.RelayTimeout <= 0 {
-		return context.Background(), func() {}
+		return reqCtx, func() {}
 	}
-	return context.WithTimeout(context.Background(), time.Duration(common.RelayTimeout)*time.Second)
+	return context.WithTimeout(reqCtx, time.Duration(common.RelayTimeout)*time.Second)
 }

Then update all call sites to pass the gin context:

  • Line 214: ctx, cancel := newAwsInvokeContext(c)
  • Line 244: ctx, cancel := newAwsInvokeContext(c)
  • Line 287: ctx, cancel := newAwsInvokeContext(c)
📝 Committable suggestion

‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.

Suggested change
func newAwsInvokeContext() (context.Context, context.CancelFunc) {
if common.RelayTimeout <= 0 {
return context.Background(), func() {}
}
return context.WithTimeout(context.Background(), time.Duration(common.RelayTimeout)*time.Second)
}
func newAwsInvokeContext(c *gin.Context) (context.Context, context.CancelFunc) {
reqCtx := c.Request.Context()
if common.RelayTimeout <= 0 {
return reqCtx, func() {}
}
return context.WithTimeout(reqCtx, time.Duration(common.RelayTimeout)*time.Second)
}
🤖 Prompt for AI Agents 
In @relay/channel/aws/relay-aws.go around lines 42-47, newAwsInvokeContext
currently uses context.Background() which prevents request cancellation from
propagating; change its signature to accept the request context (e.g.,
newAwsInvokeContext(c *gin.Context) or newAwsInvokeContext(ctx
context.Context)), create the child context from the incoming request context
(use c.Request.Context() or the passed ctx) and apply the timeout only if
common.RelayTimeout > 0, returning the derived context and cancel func; then
update all call sites that invoked newAwsInvokeContext (the three places noted)
to pass the gin context variable (c) or the request context so cancellations
from the client propagate into AWS SDK calls.

@seefs001 seefs001 Jan 5, 2026

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, this is exactly what I'm going to do.

@coderabbitai coderabbitai Bot Jan 5, 2026

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@seefs001, great! Thank you for addressing this. The fix will ensure proper request cancellation propagation and prevent resource leaks. Let me know if you need any assistance with the implementation.


func newAwsClient(c *gin.Context, info *relaycommon.RelayInfo) (*bedrockruntime.Client, error) {
var (
httpClient *http.Client
Expand Down Expand Up @@ -117,6 +126,7 @@ func doAwsClientRequest(c *gin.Context, info *relaycommon.RelayInfo, a *Adaptor,
return nil, types.NewError(errors.Wrap(err, "marshal nova request"), types.ErrorCodeBadResponseBody)
}
awsReq.Body = reqBody
a.AwsReq = awsReq
return nil, nil
} else {
awsClaudeReq, err := formatRequest(requestBody, requestHeader)
Expand Down Expand Up @@ -201,7 +211,10 @@ func getAwsModelID(requestModel string) string {

func awsHandler(c *gin.Context, info *relaycommon.RelayInfo, a *Adaptor) (*types.NewAPIError, *dto.Usage) {

awsResp, err := a.AwsClient.InvokeModel(c.Request.Context(), a.AwsReq.(*bedrockruntime.InvokeModelInput))
ctx, cancel := newAwsInvokeContext()
defer cancel()

awsResp, err := a.AwsClient.InvokeModel(ctx, a.AwsReq.(*bedrockruntime.InvokeModelInput))
if err != nil {
statusCode := getAwsErrorStatusCode(err)
return types.NewOpenAIError(errors.Wrap(err, "InvokeModel"), types.ErrorCodeAwsInvokeError, statusCode), nil
Expand All @@ -228,7 +241,10 @@ func awsHandler(c *gin.Context, info *relaycommon.RelayInfo, a *Adaptor) (*types
}

func awsStreamHandler(c *gin.Context, info *relaycommon.RelayInfo, a *Adaptor) (*types.NewAPIError, *dto.Usage) {
awsResp, err := a.AwsClient.InvokeModelWithResponseStream(c.Request.Context(), a.AwsReq.(*bedrockruntime.InvokeModelWithResponseStreamInput))
ctx, cancel := newAwsInvokeContext()
defer cancel()

awsResp, err := a.AwsClient.InvokeModelWithResponseStream(ctx, a.AwsReq.(*bedrockruntime.InvokeModelWithResponseStreamInput))
if err != nil {
statusCode := getAwsErrorStatusCode(err)
return types.NewOpenAIError(errors.Wrap(err, "InvokeModelWithResponseStream"), types.ErrorCodeAwsInvokeError, statusCode), nil
Expand Down Expand Up @@ -268,7 +284,10 @@ func awsStreamHandler(c *gin.Context, info *relaycommon.RelayInfo, a *Adaptor) (
// Nova模型处理函数
func handleNovaRequest(c *gin.Context, info *relaycommon.RelayInfo, a *Adaptor) (*types.NewAPIError, *dto.Usage) {

awsResp, err := a.AwsClient.InvokeModel(c.Request.Context(), a.AwsReq.(*bedrockruntime.InvokeModelInput))
ctx, cancel := newAwsInvokeContext()
defer cancel()

awsResp, err := a.AwsClient.InvokeModel(ctx, a.AwsReq.(*bedrockruntime.InvokeModelInput))
if err != nil {
statusCode := getAwsErrorStatusCode(err)
return types.NewOpenAIError(errors.Wrap(err, "InvokeModel"), types.ErrorCodeAwsInvokeError, statusCode), nil
Expand Down
3 changes: 3 additions & 0 deletions service/http_client.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -82,6 +82,9 @@ func ResetProxyClientCache() {
// NewProxyHttpClient 创建支持代理的 HTTP 客户端
func NewProxyHttpClient(proxyURL string) (*http.Client, error) {
if proxyURL == "" {
if client := GetHttpClient(); client != nil {
return client, nil
}
return http.DefaultClient, nil
}

Expand Down