feat(obs): Langfuse exporter wired into chat completions - #21
Merged
Conversation
Per spec §3.6 / §9 / plan §4.9. New aisix-obs::langfuse module exports:
- LangfuseEvent / LangfuseSender / LangfuseHandle types
- spawn() that returns Ok(None) when disabled, or starts a background
batch flusher (50 events or 1s, whichever comes first) when enabled
The exporter authenticates with HTTP basic (public_key:secret_key),
POSTs to {host}/api/public/ingestion in the documented batch shape
({batch: [{type: 'generation-create', body: {...}}]}), and never
blocks the request hot path — full queues drop the event silently.
Wired into:
- aisix-server bootstrap (spawn after metrics, hold handle for life
of process)
- ProxyState (new optional langfuse: Option<Arc<LangfuseSender>>)
- chat::chat_completions (emits one event per request, success or
failure)
Tests (20 new):
- Disabled config returns None
- Enabled-without-host errors clearly
- Enabled-without-key-env errors with the missing env name
- Wiremock round-trip: emit one event, wait for the 1s flush
interval, assert the upstream received exactly one POST
- ISO timestamp round-trip including unix epoch + leap year
- Base64 basic auth encoding matches expected output
- Channel-full does not block the emitter
Streaming chat handler emission, plus other endpoints
(messages/embeddings/etc.), follow in a separate PR.
There was a problem hiding this comment.
Pull request overview
Adds an optional Langfuse ingestion exporter to the observability layer and wires it into the non-streaming chat completions handler so each request emits a generation event when enabled.
Changes:
- Introduces
aisix-obs::langfusewith a bounded, non-blocking event channel and background batch flusher to Langfuse/api/public/ingestion. - Wires a
LangfuseSenderintoProxyStateand emits oneLangfuseEventper/v1/chat/completionsrequest (success or failure). - Updates workspace plumbing (exports/deps) to support the new module.
Reviewed changes
Copilot reviewed 7 out of 8 changed files in this pull request and generated 5 comments.
Show a summary per file
| File | Description |
|---|---|
| crates/aisix-server/src/main.rs | Spawns optional Langfuse exporter and injects sender into ProxyState. |
| crates/aisix-proxy/src/state.rs | Adds optional langfuse sender field + builder. |
| crates/aisix-proxy/src/chat.rs | Emits LangfuseEvent on completion for success/failure paths. |
| crates/aisix-obs/src/lib.rs | Exposes new langfuse module and re-exports its types. |
| crates/aisix-obs/src/langfuse.rs | Implements batching exporter + tests. |
| crates/aisix-obs/Cargo.toml | Adds deps for HTTP/JSON/base64/uuid and wiremock for tests. |
| crates/aisix-core/src/lib.rs | Re-exports LangfuseConfig. |
| Cargo.lock | Locks new dependencies pulled in by the exporter/tests. |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
| start_time: iso_offset(ev.latency), | ||
| end_time: now_iso.clone(), | ||
| status_message: (ev.status_code != 200) | ||
| .then(|| format!("upstream status {}", ev.status_code)), |
Comment on lines
+462
to
+467
| // SAFETY: tests run single-threaded by default and we only mutate | ||
| // env vars that are scoped to this test's spawn() call. | ||
| unsafe { | ||
| std::env::set_var("LANGFUSE_PUBLIC_KEY", "pk-test"); | ||
| std::env::set_var("LANGFUSE_SECRET_KEY", "sk-test"); | ||
| } |
Comment on lines
+88
to
+99
| // Optional Langfuse exporter — disabled in config by default. | ||
| // When enabled, the proxy gets an Arc<LangfuseSender> through | ||
| // ProxyState and emits one event per chat completion at | ||
| // end-of-request. We keep the handle alive for the lifetime of | ||
| // the process so the background flush task continues running. | ||
| let langfuse_handle = match langfuse::spawn(&cfg.observability) { | ||
| Ok(h) => h, | ||
| Err(e) => { | ||
| tracing::warn!(error = %e, "langfuse exporter disabled"); | ||
| None | ||
| } | ||
| }; |
|
|
||
| loop { | ||
| tokio::select! { | ||
| biased; |
Comment on lines
+31
to
+32
| /// never blocks the proxy thread; if Langfuse is offline we drop the | ||
| /// oldest events at the edges. |
20 tasks
jarvis9443
added a commit
that referenced
this pull request
Jun 2, 2026
Output guardrails only inspected message.content, so client-visible output that lives elsewhere bypassed content/DLP checks: - tool_calls / Anthropic tool_use (normalized into message.extra) are now folded into a single guardrail-inspected text view via ChatResponse::guardrail_output_text(), used by the keyword, text- moderation, Bedrock, and Prompt Shield output checks (#3/#18/#21). Reasoning/thinking content is intentionally left out of scope. - Non-streaming cache hits now run the resolved output guardrail chain before returning the stored body, instead of replaying it unchecked (#28). Streaming output guardrails already run end-of-stream. Part of #448 (findings #3, #18, #21, #28)
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
New `aisix-obs::langfuse` module that pushes per-chat-completion
generation events to a Langfuse `/api/public/ingestion` endpoint.
There is no first-party Rust SDK for Langfuse, so this is a hand-rolled
HTTP client that:
non-blocking, drops on overflow.
first).
`{host}/api/public/ingestion`.
Wiring
returns `Ok(None)` when disabled, an opaque `LangfuseHandle` when
enabled. The handle is held for the lifetime of the process.
`with_langfuse(...)` builder.
failure — no behavior change when langfuse is `None`.
Tests (20 new)
flush, assert exactly one POST received
Test plan
rerank/audio/images/responses) emit langfuse events — deferred to
a follow-up PR to keep this one reviewable
🤖 Generated with Claude Code