[pizzav-xyz]

Apologies for the PR spam — this is one of several small, focused PRs split from a larger batch to make review easier.

Summary

Tracks latency metrics from tool-call completion to first content token, both within a single request (gap latency) and across requests (cross-request TTFT).

Changes

• Gap latency: Time from last tool-call chunk to first content token
• Cross-request TTFT: Time from tool-finish to first content across requests (persisted via session manager)
• Per-provider aggregation: Metrics grouped by provider (resets on server restart)

Files Changed

• open-sse/services/toolLatencyTracker.ts — per-provider TTFT/gap metrics (new)
• open-sse/services/sessionManager.ts — markToolFinish/consumeToolFinishTime
• open-sse/utils/stream.ts — latency recording in SSE pipeline
• src/sse/handlers/chat.ts — chat handler integration
• src/sse/handlers/chatHelpers.ts — helper integration

[gemini-code-assist]

Code Review

This pull request introduces per-provider tool call latency tracking (TTFT and gap latency) across requests by persisting tool-finish timestamps in the session manager and recording metrics in a new tool latency tracker service. It also updates proxy resolution to pass the API key ID and adds type safety checks when resolving models. The review feedback highlights critical issues in the latency tracking implementation: checking for non-empty strings instead of any string type in the stream parser can delay latency tracking on initial empty chunks, and the recordToolTtft function fails to increment the total request count, which would lead to inaccurate metrics.

Important

The consumer version of Gemini Code Assist on GitHub is being sunset. Starting June 18, 2026, new organization installations will be blocked, and all code review activity will officially cease on July 17, 2026.
For more details on the timeline and next steps, please review the Help Documentation.

[gemini-code-assist]

The current check content && typeof content === "string" will evaluate to falsy if the first content chunk contains an empty string (""). In streaming responses, the first chunk often contains an empty string to signal the start of the content stream. By ignoring empty strings, the latency tracking (TTFT and gap latency) is delayed until the first non-empty chunk, which can lead to inaccurate metrics.

We should explicitly check if content is a string, even if it is empty, to accurately capture the true start of the content.

Suggested change

	const content = delta?.content || delta?.reasoning_content;
	if (content && typeof content === "string") {
	const content = delta?.content !== undefined ? delta.content : delta?.reasoning_content;
	if (typeof content === "string") {

[gemini-code-assist]

In translate mode, the check translateHasContent uses logical OR on the string values. If the first content chunk contains an empty string (""), translateHasContent will be falsy, delaying the latency tracking until the first non-empty chunk.

We should check if any of the content fields are of type "string" to ensure empty strings are correctly recognized as the start of content.

Suggested change

	const translateHasContent =
	parsed.delta?.text ||
	parsed.choices?.[0]?.delta?.content ||
	parsed.choices?.[0]?.delta?.reasoning_content;
	const translateHasContent =
	typeof parsed.delta?.text === "string" ||
	typeof parsed.choices?.[0]?.delta?.content === "string" ||
	typeof parsed.choices?.[0]?.delta?.reasoning_content === "string";

[gemini-code-assist]

In recordToolTtft, entry.totalRequests is not incremented when recording the TTFT latency. Since getToolLatencyByProvider uses entry.totalRequests as the measurementCount, any requests that only record TTFT via recordToolTtft will not be counted in the total measurement count, leading to inaccurate metrics reporting.

We should increment entry.totalRequests in recordToolTtft just like we do in recordToolLatency.

  const entry = metrics.get(provider)!;
  entry.totalRequests++;
  entry.ttftAfterToolSum += ttftMs;
  entry.ttftCount++;

[diegosouzapw]

Thanks @pizzav-xyz! The cross-request tool-latency tracking (one-shot markToolFinish/consumeToolFinishTime on the session + the per-provider toolLatencyTracker) is a nice observability addition, and the stream wiring is well-guarded with try/catch so it never affects the response path. I pushed a unit test covering the tracker aggregation + the session one-shot. (Note: the apiKeyId arg added to safeResolveProxy/resolveProxyForConnection is a harmless no-op until the per-key proxy work lands — left in as forward-prep.) Merging into release/v3.8.10. 🙏