Skip to content

feat(ui): add TTFT column alongside TPS in RequestLoggerV2 with backend telemetry - #12148

Closed
b3nw wants to merge 3 commits into
cheaper-inference:release/v3.8.51from
b3nw:feat/ttft-column-requestlogger
Closed

b3nw wants to merge 3 commits into
cheaper-inference:release/v3.8.51from
b3nw:feat/ttft-column-requestlogger

Conversation

@b3nw

@b3nw b3nw commented Aug 30, 2026

Copy link
Copy Markdown
Contributor

Summary

Adds Time-to-First-Token (TTFT) metrics to call log telemetry and surfaces a sortable TTFT column in the RequestLogger dashboard.

  • Adds migration 171_call_logs_ttft_ms.sql adding nullable ttft_ms INTEGER to call_logs.
  • Records TTFT during SSE streaming responses in chatCore.ts and attemptLogging.ts. Failed or aborted streams safely record NULL to avoid poisoning statistical aggregations.
  • Surfaces TTFT column in RequestLoggerV2.tsx with adaptive formatting (ms for <1000ms, s for >=1000ms) and full null-aware sorting (ttft_asc and ttft_desc).
  • Adds changelog fragment changelog.d/features/ttft-column-requestlogger.md.

Related Issues

N/A

Validation

  • Change type: UI / DB / routing
  • Focused tests and category gates from the golden path
  • npm run lint
  • Reconciled with the current active release base; focused checks rerun afterward
  • Production-code changes include a new or updated automated test in this PR

Tests Added Or Updated

  • tests/unit/call-logs-ttft.test.ts
  • tests/unit/shared/components/RequestLoggerV2.test.ts

Coverage Notes

  • Covers src/lib/usage/callLogs.ts, open-sse/handlers/chatCore/attemptLogging.ts, and src/shared/components/RequestLoggerV2.tsx.
  • Validates TTFT calculation, null persistence on failure, formatting thresholds, and table sort invariants.

Reviewer Notes

  • Includes database migration 171_call_logs_ttft_ms.sql (additive nullable column, backward-compatible with existing call log records).

@b3nw
b3nw requested a review from diegosouzapw as a code owner August 30, 2026 18:58
@b3nw
b3nw force-pushed the feat/ttft-column-requestlogger branch 3 times, most recently from 35eb8da to 94bccfb Compare August 31, 2026 00:28
b3nw and others added 2 commits September 4, 2026 00:33
Preserve the TTFT column, telemetry, and call-log persistence. Extract
RequestLogger metrics/sort helpers so file-size and complexity stay
inside the frozen ratchets. Renumber the ttft_ms migration to 173.

Co-authored-by: diegosouzapw <8016841+diegosouzapw@users.noreply.github.com>
Preserve the TTFT column, telemetry, and call-log persistence.

Co-authored-by: diegosouzapw <8016841+diegosouzapw@users.noreply.github.com>
@diegosouzapw

Copy link
Copy Markdown
Collaborator

Thanks for this — TTFT visibility is genuinely useful. In reviewing the group we found
#13373 covers the same ground (identical ALTER TABLE call_logs ADD COLUMN ttft_ms
statement, same TTFT column in RequestLoggerV2) and goes further: it also fixes the
underlying TPS formula bug (wall-clock vs. generation-time) that affects thinking-model rows,
and threads the fix through getModelLatencyStats (auto-combo speed ranking) and the
playground, which this PR doesn't touch. To avoid a duplicate migration/column, we're
planning to move forward with #13373 and close this one as subsumed — flagging here first
in case there's something in requestLoggerMetrics.ts or the normalizeTtftMs edge-case
test worth porting over with credit before that happens.

Triage note: this is the review recommendation — the close itself happens only after the maintainer's per-PR sign-off (and, where a superseding PR is named, after it has landed). Nothing is being closed by this comment.

@diegosouzapw diegosouzapw added the protected-surface Touches an agent-instruction surface (AGENTS/CLAUDE/llm.txt/SKILL.md) — per-PR operator OK to merge label Sep 15, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

protected-surface Touches an agent-instruction surface (AGENTS/CLAUDE/llm.txt/SKILL.md) — per-PR operator OK to merge

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants