feat(obs): add ttft_ms (time to first token) to UsageEvent

[jarvis9443]

Add a ttft_ms field to UsageEvent to track first-token latency on the streaming path.

What changed:

• UsageEvent gains ttft_ms: u32 — time from request entry to first upstream SSE chunk carrying delta.content or delta.tool_calls (milliseconds). Role-only chunks are skipped. Omitted from the wire when 0 (backwards-compatible with older cp-api).
• Streaming path in build_sse_stream records elapsed time on the first content/tool-call chunk, stored in StreamCompletion and propagated through UsageExtras → emit_usage_event.
• Non-streaming, error, and cache-hit paths keep ttft_ms = 0.
• OTLP trace export includes aisix.ttft_ms span attribute when > 0.

Files:

• crates/aisix-obs/src/usage.rs — new field + serialization tests
• crates/aisix-proxy/src/chat.rs — TTFT capture + wiring
• crates/aisix-obs/src/otlp_http_sink.rs — OTLP attribute + test

Follow-up: cp-api side (telemetryEvent struct, DB migration, dashboard) to persist and surface TTFT.

Summary by CodeRabbit

• New Features

  • Added time-to-first-token (TTFT) telemetry for chat completions; streaming paths now capture and include TTFT in telemetry while non-streaming emits zero.

• Tests

  • Extended tests to ensure TTFT is collected for streaming, serialized into telemetry when set, and omitted from payloads when zero.

[coderabbitai]

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info

⚙️ Run configuration

Configuration used: Organization UI

Review profile: CHILL

Plan: Pro

Run ID: 4e2994f8-c99a-478e-bd75-c4be95e9ac5c

📥 Commits

Reviewing files that changed from the base of the PR and between c919a4a and bfe1ab9.

📒 Files selected for processing (1)

• crates/aisix-proxy/src/chat.rs

🚧 Files skipped from review as they are similar to previous changes (1)

• crates/aisix-proxy/src/chat.rs

---

📝 Walkthrough

Walkthrough

This PR adds time-to-first-token (TTFT) telemetry for chat completions. The UsageEvent struct gains a ttft_ms field that streams populate by tracking the first upstream chunk's arrival time; non-streaming paths set it to zero. TTFT is exported as an OTLP span attribute aisix.ttft_ms when present.

Changes

TTFT Telemetry Integration

Layer / File(s)	Summary
UsageEvent TTFT field and serialization crates/aisix-obs/src/usage.rs	UsageEvent gains ttft_ms: u32 field with serde(default, skip_serializing_if = "is_zero_u32") to omit zero values from JSON; tests assert the field is absent when unset and present when assigned.
Chat handler TTFT collection and emission crates/aisix-proxy/src/chat.rs	UsageExtras and StreamCompletion structures add ttft_ms fields; build_sse_stream accepts started: Instant and streaming loop computes TTFT from first upstream chunk arrival; non-streaming and error paths set ttft_ms to 0; emit_usage_event copies the value into final UsageEvent.
OTLP span attribute encoding crates/aisix-obs/src/otlp_http_sink.rs	build_otlp_traces_payload conditionally encodes event.ttft_ms as OTLP intValue attribute aisix.ttft_ms only when ttft_ms > 0; tests verify omission when zero and correct JSON encoding when set.

🎯 2 (Simple) | ⏱️ ~10 minutes

🚥 Pre-merge checks | ✅ 4

✅ Passed checks (4 passed)

Check name	Status	Explanation
Description Check	✅ Passed	Check skipped - CodeRabbit’s high-level summary is enabled.
Title check	✅ Passed	The title 'feat(obs): add ttft_ms (time to first token) to UsageEvent' accurately reflects the main change: adding a new ttft_ms field to the UsageEvent struct to track time-to-first-token latency.
Linked Issues check	✅ Passed	Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check	✅ Passed	Check skipped because no linked issues were found for this pull request.

_{✏️ Tip: You can configure your own custom pre-merge checks in the settings.}

✨ Finishing Touches

📝 Generate docstrings

• Create stacked PR
• Commit on current branch

---

_{Comment @coderabbitai help to get the list of available commands and usage tips.}

[Copilot]

Pull request overview

Adds TTFT telemetry to usage/observability so streaming chat requests can report first-token latency through UsageEvent and OTLP traces.

Changes:

• Added ttft_ms to UsageEvent, omitted when zero.
• Wired streaming completion telemetry to carry TTFT from SSE processing.
• Added OTLP trace attribute emission and serialization tests.

Reviewed changes

Copilot reviewed 3 out of 3 changed files in this pull request and generated 1 comment.

File	Description
crates/aisix-proxy/src/chat.rs	Captures TTFT during streaming and forwards it into usage telemetry.
crates/aisix-obs/src/usage.rs	Adds the ttft_ms UsageEvent field and serialization coverage.
crates/aisix-obs/src/otlp_http_sink.rs	Emits aisix.ttft_ms OTLP span attribute when set.

---

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.