We read every piece of feedback, and take your input very seriously.
To see all available qualifiers, see our documentation.
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
测试时,为什么平均延迟和平均生成第一个token的时间是一样的?
The text was updated successfully, but these errors were encountered:
同问,求问是什么原因
Sorry, something went wrong.
请问您是在非 Steam 模式下的压测吗?
在非 Stream 模式返回的情况下,客户端收到第一个 Token 的延迟 == 整个 Response 的延迟。因此计算结果是相等的。
lxline
No branches or pull requests
测试时,为什么平均延迟和平均生成第一个token的时间是一样的?
The text was updated successfully, but these errors were encountered: