fix: 流式请求客户端断连后继续读取上游响应，避免 usage 丢失

[zuiho-kai]

问题

StreamScannerHandler（relay/helper/stream_scanner.go）中，scanner goroutine 和主循环都监听了 c.Request.Context().Done()。客户端断连时 scanner 立即停止读取上游响应流，导致最后的 message_delta 事件（含 output_tokens 等 usage 信息）丢失，计费记录不完整，造成上下游账单差异。

根因

• scanner goroutine（第 232 行）：case <-c.Request.Context().Done() → 客户端断连立即 return
• 主循环（第 290 行）：case <-c.Request.Context().Done() → 客户端断连结束整个流程

修复

去掉 scanner goroutine 和主循环中的 c.Request.Context().Done() 监听。客户端断连后 scanner 继续读完上游流拿到完整 usage 数据再退出。

不影响资源回收：

• 上游响应流会自然结束（EOF 或 [DONE]）
• streaming timeout（ticker）兜底防止无限阻塞
• stopChan 机制确保所有 goroutine 正确退出
• ping goroutine 保留 c.Request.Context().Done() 检查（客户端断连后无需继续 ping，且 ping goroutine 正常退出不会触发 stopChan）

实际案例

通过对比上下游账单发现：上游 100 条记录中有 2 条在 new-api 侧缺失计费记录（quota=0），均为客户端断连导致 scanner 提前停止、未读到 usage 事件。

Fixes #4463

Summary by CodeRabbit

• Bug Fixes
  • Improved stream handling to ensure final usage-related events are properly captured during stream completion, rather than being lost during termination. Stream timeout and shutdown coordination now work more reliably.

[coderabbitai]

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info

⚙️ Run configuration

Configuration used: Organization UI

Review profile: CHILL

Plan: Pro

Run ID: 45c51776-c29e-4071-b5e2-9172710f4d3f

📥 Commits

Reviewing files that changed from the base of the PR and between f2f3410 and fbddb71.

📒 Files selected for processing (1)

• relay/helper/stream_scanner.go

---

Walkthrough

The handler removes checks for client disconnection (c.Request.Context().Done()) from the scanner loop and main select statement. The scanner now continues reading upstream until EOF, [DONE], or streaming timeout, relying on stopChan and internal context (ctx.Done()) for termination. Comments clarify this preserves final usage events needed for complete billing records.

Changes

Cohort / File(s)	Summary
Stream Scanner Control Flow relay/helper/stream_scanner.go	Removed c.Request.Context().Done() checks from scanner goroutine and main loop select statements. Scanner now continues reading upstream until natural EOF/[DONE] or streaming timeout, preventing loss of final message_delta events containing token usage data. Added clarifying comments on shutdown coordination strategy.

Estimated code review effort

🎯 2 (Simple) | ⏱️ ~12 minutes

Suggested reviewers

• seefs001

Poem

🐰 A scanner once stopped when clients would flee,
Leaving tokens uncounted, the ledgers askew—
Now it reads until [DONE], complete billing spree,
No more phantom accounts, the math finally true! ✨

🚥 Pre-merge checks | ✅ 4 | ❌ 1

❌ Failed checks (1 warning)

Check name	Status	Explanation	Resolution
Docstring Coverage	⚠️ Warning	Docstring coverage is 0.00% which is insufficient. The required threshold is 80.00%.	Write docstrings for the functions missing them to satisfy the coverage threshold.

✅ Passed checks (4 passed)

Check name	Status	Explanation
Description Check	✅ Passed	Check skipped - CodeRabbit’s high-level summary is enabled.
Title check	✅ Passed	The title accurately describes the main change: continuing to read upstream responses after client disconnection to avoid loss of usage data.
Linked Issues check	✅ Passed	The PR implementation directly addresses issue #4463 by removing client context monitoring from scanner and main loop to allow complete upstream response reading.
Out of Scope Changes check	✅ Passed	All changes are strictly scoped to removing c.Request.Context().Done() monitoring from stream_scanner.go as required by issue #4463.

_{✏️ Tip: You can configure your own custom pre-merge checks in the settings.}

✨ Finishing Touches

🧪 Generate unit tests (beta)

• Create PR with unit tests

---

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

• X
• Mastodon
• Reddit
• LinkedIn

_{Comment @coderabbitai help to get the list of available commands and usage tips.}

[Calcium-Ion]

这不应该算是bug，不能算作修复，大部分提供商支持流模式中断，如果需要忽略客户端断开可以用nginx等网关配置

[zuiho-kai]

这不应该算是bug，不能算作修复，大部分提供商支持流模式中断，如果需要忽略客户端断开可以用nginx等网关配置

看你决策吧，因为有些供应商就算支持了，但是可能因为回传的各种网络波动什么的，导致超时失败，这个是我发现计费异常定位发现的，如果什么依赖第三方，例如供应商，例如网关配置，那我觉得这个有点太多约束了

这个问题让我昨天发现丢了10个请求的钱，我搞了个财务统计系统才发现，否则在大流量下有盈利就被覆盖看不出来了

[tointcn]

这不应该算是bug，不能算作修复，大部分提供商支持流模式中断，如果需要忽略客户端断开可以用nginx等网关配置

网关忽略没有用

[fengerwoo]

这不应该算是bug，不能算作修复，大部分提供商支持流模式中断，如果需要忽略客户端断开可以用nginx等网关配置

但是这个问题会导致有人在请求是恶意取消，实现请求免单，是否最起码应该只要开始响应了，最起码计费个预估输入？

[Calcium-Ion]

目前就是按照已输出文本估算计费的，计费为0可以看看自己是不是关了流模式本地计费

[fengerwoo]

这不应该算是bug，不能算作修复，大部分提供商支持流模式中断，如果需要忽略客户端断开可以用nginx等网关配置

Nginx的location 无论设置了 proxy_ignore_client_abort on; 还是如何设置，都没有效果，都无法实现 Nginx 层面实现 客户端中断，newapi继续请求。是否可以加个设置？

[fengerwoo]

这不应该算是bug，不能算作修复，大部分提供商支持流模式中断，如果需要忽略客户端断开可以用nginx等网关配置

Nginx的location 无论设置了 proxy_ignore_client_abort on; 还是如何设置，都没有效果，都无法实现 Nginx 层面实现 客户端中断，newapi继续请求。是否可以加个设置？

成功了，在 nginx 的 location 配置加以下参数就行

# 下游取消/中断，上游继续
proxy_ignore_client_abort on;
proxy_ignore_headers X-Accel-Buffering;
proxy_buffering on;
proxy_cache off;
proxy_max_temp_file_size 1024m;

set $responses_drain_sink "";
proxy_store $responses_drain_sink;

proxy_set_header Upgrade $http_upgrade;
proxy_set_header Connection $connection_upgrade;

如果希望只改 /v1/responses 的 endpoint 不影响其他 endpoint 话，可以把 location 复制一份，然后 location ^~ / 匹配改为 location = /v1/responses ，后面增加以上配置就行，完整 location 示例如下

    location ^~ / {
      proxy_pass http://127.0.0.1:3000;
      proxy_set_header Host $http_host;
      proxy_set_header X-Real-IP $remote_addr;
      proxy_set_header X-Real-Port $remote_port;
      proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
      proxy_set_header X-Forwarded-Proto $scheme;
      proxy_set_header X-Forwarded-Host $host;
      proxy_set_header X-Forwarded-Port $server_port;
      proxy_set_header REMOTE-HOST $remote_addr;
      
      proxy_buffering off;
      proxy_connect_timeout 60s;
      proxy_send_timeout 1800s;
      proxy_read_timeout 1800s;
      proxy_http_version 1.1;
      proxy_set_header Upgrade $http_upgrade;
      proxy_set_header Connection $connection_upgrade;
    }
    
    location = /v1/responses {
      proxy_pass http://127.0.0.1:3000;
      proxy_set_header Host $http_host;
      proxy_set_header X-Real-IP $remote_addr;
      proxy_set_header X-Real-Port $remote_port;
      proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
      proxy_set_header X-Forwarded-Proto $scheme;
      proxy_set_header X-Forwarded-Host $host;
      proxy_set_header X-Forwarded-Port $server_port;
      proxy_set_header REMOTE-HOST $remote_addr;
      
      proxy_connect_timeout 60s;
      proxy_send_timeout 1800s;
      proxy_read_timeout 1800s;
      proxy_http_version 1.1;
      proxy_set_header Upgrade $http_upgrade;
      proxy_set_header Connection $connection_upgrade;
      
      
      # 下游取消/中断，上游继续
      proxy_ignore_client_abort on;
      proxy_ignore_headers X-Accel-Buffering;
      proxy_buffering on;
      proxy_cache off;
      proxy_max_temp_file_size 1024m;
    
      set $responses_drain_sink "";
      proxy_store $responses_drain_sink;
    
      proxy_set_header Upgrade $http_upgrade;
      proxy_set_header Connection $connection_upgrade;
    }