Repository navigation
Conversation
A request_end row carried the client's x-request-id and the model only when the request had agent context. The frontend now assigns its own request_id, so a plain request had no key a client could join on. Read x-request-id once, through one helper, for the agent path and the plain path, and record the model on both. Signed-off-by: Ryan Olson <rolson@nvidia.com>
ryanolson
temporarily deployed
to
external_collaborator
November 4, 2026 19:29 — with
GitHub Actions
Inactive
ryanolson
temporarily deployed
to
external_collaborator
November 4, 2026 19:29 — with
GitHub Actions
Inactive
This was referenced Nov 4, 2026
Contributor
Contributor
|
/ok to test 967a494 |
Contributor
✅ Dynamo PR CI passed — run 41897439797 (attempt 1) on
|
This branch was previously deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Overview
The frontend mints its own
request_id(#17032), and that is the key trace consumers should use to tie a request's rows together inside Dynamo. Two things were still missing onrequest_endfor a plain request (one with no agent context):model. A consumer that keeps separate state per model, such as one that follows KV block reuse per model, needs it to route the row to the right state without first looking up the request.x_request_id. It is not a key inside Dynamo. It is the client's own id, from thex-request-idheader, for joining a trace row to records kept outside Dynamo, such as a load generator's results or the client's logs.Both were written only when the request had agent context. With this change every
request_endrow carriesmodel, andx_request_idwhen the client sent the header.Key Changes
About 25 lines of core logic, plus a test and a docs paragraph.
lib/llm/src/request_trace/agent_context.rs: one helper,x_request_id_from_context, readsx-request-idfrom the distributed tracing context, else from the request context. The agent path and the plain path both use it, so they cannot disagree.lib/llm/src/request_trace/integration.rsandlib/llm/src/request_trace/record.rs: the request-end state keepsmodelandx_request_id, andemit_request_endwrites them on the plain path.docs/fern/pages/reference/observability/request-traces.mdx: says that every row carriesmodel, andx_request_idwhen the header is sent.Validation
x-request-idheaderrequest_endhas nox_request_id, nomodelx-request-idheaderTest added:
plain_request_end_carries_x_request_id_and_modelinrequest_trace/integration.rs. It fails without the change.cargo test --locked -p dynamo-llm --libpasses: 4,001 passed, 0 failed.No new environment variables. The fields appear when request tracing is on (
DYN_REQUEST_TRACE).Issues and Ongoing Work
request_id.request_trace/integration.rsfor different reasons: fix(frontend): emit request_end trace records on native generate endpoints #17036 (request_end rows on native generate endpoints) and feat(request-trace): retain generated sequence hashes #14636 (output sequence hashes). No overlap in behavior.engine_reused_tokensfield onrequest_end) and feat(request-trace): add an opt-in request_start row with routing hashes #18105 (the opt-inrequest_startrow).Pull Request Checklist
/pr-checklistskill.