[xiaoyaner0201]

Summary

Add an OTLP/HTTP traces destination for persisted call logs, with optional Langfuse mapping and opt-in original request Header attributes. Each exported call-log row becomes a span; the existing isolated export process and cursor semantics remain in place.

This is PR B of #16454, building on the merged #16663. The branch includes release/v3.8.53 through e9cb075cf6db161661b01f5dde2a1f0b033f7660. This update adds Header export without changing the database schema or importing a newer base.

Why

Standard trace export makes persisted gateway activity available to an OpenTelemetry backend: model usage, latency, TTFT, errors, sessions and optional request/response inspection. Original client Headers add context such as the calling application, user agent or SDK version, without hard-coding an application taxonomy into the exporter.

Changes

• Register otlp-http with configurable endpoint, encrypted collector authentication Headers, service/environment, allowed request paths, timeout, optional Langfuse mapping and user identity mode.
• Export OpenTelemetry GenAI and omniroute.* attributes. Optional Langfuse mapping supplies model, usage, input/output, session, user, level and TTFT. Bodies are emitted once, rather than duplicated under both mappings.
• Derive trace/span IDs from the call-log ID; resending a row retains its IDs. Langfuse v4 can retain multiple rows with the same IDs, so this is not a deduplication guarantee.
• Bound requests to 500 spans and 4 MiB. Check every span before sending a cursor batch, preserve the cursor on failure or partial acceptance, and use bounded retries for 408/429/5xx/network errors. Redirects are not followed.
• Keep diagnostic messages limited to exporter-owned facts. Collector response bodies and Headers are not echoed into persisted errors, the dashboard or logs.
• Carry the existing ttft_ms column through the export record. New BigQuery tables include a nullable ttft_ms; old tables without it remain supported.
• Add create-time startFromCurrentPosition, with its UI string in all 67 catalogs.
• Update LOG-EXPORT.md, the changelog fragment, packaging coverage and focused tests. Refresh the document's 66 translated mirrors through the official pipeline.

Original request Headers

requestHeaderAllowlist defaults to empty. It accepts exact names or a trailing-prefix wildcard, as an array or comma/newline-separated text:

{
  "requestHeaderAllowlist": [
    "user-agent",
    "x-stainless-package-version",
    "x-omniroute-meta-*"
  ]
}

With the original client snapshot available, the span contains ordinary OpenTelemetry attributes:

http.request.header.user-agent = ["SyntheticHarness/2.0.0"]
http.request.header.x-stainless-package-version = ["9.8.7"]
http.request.header.x-omniroute-meta-app = ["synthetic-test-platform"]
user_agent.original = "SyntheticHarness/2.0.0"

• Header names become lowercase; values remain string arrays. A selected, single-valued User-Agent also supplies user_agent.original.
• X-OmniRoute-Meta-* has no special interpretation: the complete name is preserved, without prefix stripping or a Langfuse-specific business-field mapping. SDK/version parsing belongs to downstream analysis.
• The source is the existing hydrated pipelineClientRequest, not the Headers sent to a model provider. includeBodies=true and retained detailed logging are required. Missing, malformed, oversized or truncated snapshots contribute no Header attributes; the exporter does not reconstruct them.
• Credential Header names and recognized credential values are excluded even if explicitly allowlisted. Types, control characters, Unicode validity, counts and byte sizes are bounded. With the default empty allowlist, the Header path does not read the snapshot.
• Allowlisting remains an operator privacy decision. Arbitrary sensitive business data is not automatically classified. Enabling includeBodies also retains its existing prompt/response export behavior; identity pseudonymization applies only to the user identifier.

The destination's existing headers option is separate: it authenticates exporter → collector HTTP requests and is not an input to span attributes.

Validation

• 479 focused tests passed on each of Node 22, 24 and 26 at d953b1be9eeef847810d1fc4f19b2a03f7a5de4d, covering Header export, OTLP payload/error handling, adjacent export paths and packaging. Each test file ran in its own process; the Header file contains 68 passing cases, including real compiled-child tests.
• 22 tests across four independent regression probes passed, including literal and escaped-quote credential counterexamples through the real logger, persistence/hydration, compiled export child and loopback collector; default-off output matched the pre-Header implementation in 12 byte-level controls with zero snapshot reads. Credential selection reuses the existing complete public recognition path as an omission signal, while safe Header values remain unchanged.
• Lint, file-size, complexity, changelog integrity, pack policy and repository inventory checks passed on the final commit; the verification checkout remained clean.
• 66 locale mirrors refreshed and source/target hashes independently verified. The strict structural checker passed 62 mirrors directly; four contained one additional, unchanged inline-code literal each (null in ka; a repeated headers in kn/mr/te). These presentation-only differences were reviewed separately, with no missing literals or other structural failures. The new Header subsection was read in zh-CN, ja, de, ar, es and ru. The same-scope --all drift comparison adds no entries relative to the frozen base, and contains no LOG-EXPORT.md drift.
• TypeScript compiler checks were not rerun locally for the final credential-filter repair. Earlier scoped typecheck results are historical; the current compiler result must come from CI.

An independent read-only review of this exact commit/tree found no remaining blocking findings. The original escaped-credential regression was reproduced against the parent and rejected by the candidate; a further eight-case canonical/adversarial suite passed, including safe-value parity and default-off zero reads.

A synthetic receiver acceptance test used the candidate's actual OTLP client against Langfuse 4.56.0. Readback through both the v2 observations API and ClickHouse confirmed the complete Header names, string-array values and scalar User-Agent. Credential attributes were absent. Header arrays are stored in Langfuse metadata as JSON-array strings, so a scalar equality filter and an array-string equality filter are different queries. No model request or production exporter activation was involved.

Earlier receiver testing also covered input/output, usage, model, session, user, error level, TTFT and same-ID resends. It is historical evidence for those unchanged mappings, not a substitute for the latest Header checks.

Known limits

• Complete credential recognition has an opt-in CPU cost near the snapshot limits. A synthetic 985,933-byte snapshot with 240 maximum-sized selected values took 7.38 seconds in a comparison run versus 1.48 seconds with the previous helper; separate bounded stress runs remained below 15 seconds. These are extreme local helper measurements, not production latency figures. Work stays in the isolated export child, but larger detailed batches can take longer; Header capture remains off by default.
• Full local unit/Vitest/E2E suites and a production application build were not run for this update. Read the checks attached to the published head for full CI status.
• The sandboxed real-child proxy fixture targeting hostname sidecar is blocked by the loopback-only test guard. The remaining child cases were exercised without removing the guard.
• Four documentation consistency tests reproduce with the same failing assertions on the frozen upstream base and the candidate: README/package version consistency, docs-counts, and provenance for RESILIENCE_GUIDE and ENVIRONMENT. These unrelated failures are left unchanged. The scoped LOG-EXPORT.md translation result is reported separately above.
• This remains persisted-log export, not live request instrumentation. Detail retention, includeBodies and downstream handling determine which optional data is available.

Infographic

The restaurant serves the client while a separate courier delivers copies of the saved receipts. Streaming pieces are assembled before export; body export is opt-in. The bookmark (cursor) advances only after an acknowledged batch; a delivery failure leaves it at the last confirmed position. The isolated courier is the prerequisite from #16663; this PR adds the OTLP/HTTP delivery format. This illustrates the mechanism, not a product screenshot or an exactly-once guarantee.

中文说明

概要

新增 OTLP/HTTP traces 日志导出目的地，并提供可选的 Langfuse 字段映射及默认关闭的原始请求 Header 导出。每条调用日志对应一个 span，复用已有的隔离子进程与游标处理。

这是 #16454 的 PR B，前置 #16663 已合并。当前分支包含 release/v3.8.53 的固定基线 e9cb075cf6db161661b01f5dde2a1f0b033f7660；本轮新增 Header 支持，不改数据库结构，也没有再次追合上游。

Header 的处理方式

• 使用 requestHeaderAllowlist 选择需要的 Header，支持精确名字及末尾前缀通配符；默认空列表，不读取请求快照。
• 从已有详细日志的原始客户端快照取值，输出 http.request.header.<完整小写名字>，值为字符串数组。选中的单值 User-Agent 额外输出 user_agent.original。
• X-OmniRoute-Meta-* 与普通 Header 一样处理：保留完整名字，不剥前缀、不生成专用业务字段，也不在导出器里解析 CLI 或 SDK 版本。
• 需要 includeBodies=true 和仍可读取的详细日志。快照缺失、损坏、过大或截断时不生成 Header 属性；不会用模型厂商侧的请求头代替。
• 凭据头及可识别的凭据值强制排除，并限制类型、字符、条数和字节数。业务敏感数据仍需管理员通过白名单自行控制。开启 includeBodies 同时沿用原有的提示词/响应正文导出行为。
• 配置里的 headers 仍是导出器请求 collector 时的鉴权头，与上述 span 属性无关。

原有功能与验证

保留 OTLP/GenAI 属性、可选 Langfuse 映射、稳定 trace/span ID、有界重试与批次大小、失败不前移游标、仅含导出器自有事实的错误信息，以及 BigQuery 的 ttft_ms 兼容和从当前位置开始导出的选项。

本轮最终测试、独立审查及 66 份翻译的结果见上方 Validation。真实 Langfuse 4.56.0 验收使用合成数据和候选自己的导出客户端，分别从 observations API 与 ClickHouse 回读，确认原始 Header 名、字符串数组及 User-Agent 都已保存，凭据属性不存在。没有发起模型请求，也没有启用生产导出器。

独立只读复审未发现剩余阻塞项。完整凭据识别存在极限输入下的 CPU 代价：约 986KB、240 个大值的合成快照在一次对照中由 1.48 秒增加到 7.38 秒；这是本地 helper 的压力测量，不是生产请求延迟。处理仍在隔离导出子进程内，Header 捕获默认关闭。

完整本地 unit/Vitest/E2E、生产应用构建及最终修复后的 TypeScript 编译检查未跑；完整 CI 以发布后准确 head 的 checks 为准。sidecar 代理夹具受本地网络保护限制；另外四项文档一致性失败已在固定上游基线上同命令复现，未扩大修改范围。日志详情和保留策略仍决定可导出的数据范围。

比喻图

餐厅照常给顾客出餐，独立邮递员另送已保存的账单副本。流式片段先组装为回答，正文导出需要显式开启；收到一批数据的确认后，书签（游标）才向前移动，发送失败则停在上次确认的位置。独立邮递员对应前置 PR #16663，本 PR 增加 OTLP/HTTP 这种寄送格式。此图解释机制，不是产品截图，也不承诺恰好一次投递。

[xiaoyaner0201]

Thanks for merging #16663. Following the A-first order, #16664 is now refreshed on release/v3.8.52: an ordinary merge commit (no rebase or force-push), so the PR diff is the OTLP destination only. The shared export code from #16663 and your refactors are unchanged; the only shared-query change is mapping the existing call_logs.ttft_ms column. Focused log-export tests pass on Node 22/24/26 (399/399 each), and the refreshed code was re-verified against a self-hosted Langfuse v4.56.0. Details are in the PR body.

A note on why we would like this to land: we run OmniRoute as a shared gateway and use this OTLP path to send its call logs to a self-hosted Langfuse. It is how we measure per-key and per-session usage, latency/TTFT and error rates, and it shows where the gateway's capability limits are. More generally, a standard OpenTelemetry destination lets any OmniRoute deployment plug into Langfuse, Phoenix, Tempo or an OTel Collector instead of building one integration per tool, and it stays off unless a destination is created.

I see it is labelled for v3.8.53. If anything would help it move (a narrower first slice, dropping the BigQuery ttft_ms part into a follow-up, or a specific test you want), tell me and I will turn it around quickly. Converting it to Ready now so the full CI runs.

中文说明

感谢合并 #16663。按先 A 后 B 的顺序，#16664 已基于 release/v3.8.52 更新：用的是普通 merge 提交（没有 rebase 或 force-push），所以 PR 的 diff 只剩 OTLP 目的地本身。#16663 的共享导出代码和你的重构都没有改动；共享查询唯一的改动，是映射 call_logs 里已有的 ttft_ms 列。日志导出的定向测试在 Node 22/24/26 上各 399 项全部通过，更新后的代码也重新对自建的 Langfuse v4.56.0 做了验证。细节见 PR 正文。

说明一下我们为什么希望它能合入：我们把 OmniRoute 当作共享网关运行，用这条 OTLP 路径把调用日志送进自建的 Langfuse。我们靠它按 Key、按会话衡量用量，看延迟、首字耗时和错误率，也靠它看清网关的能力边界。更一般地说，有了一个标准的 OpenTelemetry 目的地，任何 OmniRoute 部署都能接入 Langfuse、Phoenix、Tempo 或 OTel Collector，不用每个工具各做一套集成；而且不建目的地就不会启用。

我看到它标了 v3.8.53。如果有什么能帮它推进（比如先合一个更窄的切片、把 BigQuery 的 ttft_ms 拆到后续 PR，或者你想要某个具体测试），告诉我，我会尽快处理。现在把它转为正式待审，好让完整 CI 跑起来。

[diegosouzapw]

Re-homed to release/v3.8.53: v3.8.52 entered its release freeze, so the branch now belongs to the release captain and development continues on the next cycle. Nothing is wrong with this PR — it just needed a live base. No action needed from you; CI will re-run against the new base.

[xiaoyaner0201]

Thanks for clarifying and moving this to the live branch, @diegosouzapw. We are not asking for an exception to the v3.8.52 freeze. Could you prioritize review and landing of this PR early in the v3.8.53 development cycle, once the required checks are satisfied, and point out any remaining merge blockers?

Our shared gateway's planned production collection integration is waiting on this upstream-supported exporter. The branch has been tested against a self-hosted Langfuse; we have not deployed it to our production gateway. A standard OTLP destination would make retained call logs usable for per-key/session usage, latency/TTFT, error analysis and request/response inspection, without a separate gateway integration for each backend.

The scope is bounded: export is opt-in and runs in the isolated child from #16663; this is call-log export, not application-internal agent/skill tracing or proof that a tool executed successfully. It does not export monetary cost, automatically delete source logs, or provide prompt/response redaction—identity pseudonymization only covers the user identifier. Retry delivery is not an exactly-once guarantee.

I see CI has restarted against the new base, and I am not claiming it is green. I can address PR-specific CI/review findings, or move the optional BigQuery TTFT addition into a follow-up if that would help the OTLP feature land sooner. The concrete request is review and a path to merge into the active 53 branch.

[xiaoyaner0201]

Updated the branch with the LOG-EXPORT.md translations for all 66 locales using the official translation pipeline. Only those mirrors and their state entry changed in the translation commit; existing drift for other documents is untouched.

• 66/66 passed structural checks; semantic spot-checks of the new OTLP section passed in six languages.
• Focused log-export suite: 399/399 pass; typecheck:core: pass.
• LOG-EXPORT.md has no remaining source, target or stale-locale drift. The full drift check still reports the same pre-existing entries as the frozen release/v3.8.53 base (e9cb075).

The branch includes that base through an ordinary merge; the feature delta is unchanged. The PR description now reflects the v3.8.53 base and the translation refresh.