feat(langfuse): widen tracing to errors, sessions, subagents, and MoA fan-out - #85439
Merged
kshitijk4poor merged 3 commits intoAug 13, 2026
Merged
Conversation
… fan-out Salvaged from PR NousResearch#83437 by @erosika, with adopted fixes from @bgodlin (NousResearch#81054), @aldoeliacim (NousResearch#82332), @nftpoetrist (NousResearch#42326), @rodboev (NousResearch#39653), @FnExpress (NousResearch#64292, supersedes NousResearch#32175 by @db-aeon), @Per0-1 (NousResearch#61166), @NaMinhyeok (NousResearch#64797), and @liuhao1024 (NousResearch#43130). Widens the bundled Langfuse plugin from 6 to 11 hooks and fixes two attribution bugs. Also adopts shutdown/atexit lifecycle fixes and composes 8 prior community PRs with interaction-fix follow-ups. Model attribution: on_pre_llm_request and on_post_llm_call now prefer the wire value (request body model, response model) over the agent attribute, which goes stale after /model switch or provider fallback. Cost total: both cost paths now send a summed total alongside the per-type breakdown, since Langfuse does not derive calculatedTotalCost from cost_details keys. Subscription-included routes send no cost keys at all. New coverage: api_request_error closes failed generations with ERROR level; on_session_finalize/on_session_end close dangling traces for tool-only and interrupted turns; subagent_start/subagent_stop trace delegated children as spans; MoA advisor fan-out emits one generation per advisor priced at the advisor's own model. Capture modes: HERMES_LANGFUSE_CAPTURE=metadata|sanitized|full (default sanitized). Sanitized mode redacts secret patterns before truncation. Adopted lifecycle fixes: shutdown client at session finalize when reason=shutdown (not on session rotation); atexit finalizer ends open root spans for short-lived processes; root context manager exited to prevent interpreter-teardown TypeError; TOCTOU on _get_langfuse() fixed with lock; reasoning_content surfaced in traces; system prompt included in generation input for Anthropic/Codex/Bedrock; SDK v3 update_trace replaces set_trace_io. Closes NousResearch#29482, NousResearch#43129, NousResearch#72661. Supersedes NousResearch#81054, NousResearch#82332, NousResearch#42326, NousResearch#39653, NousResearch#64292, NousResearch#32175, NousResearch#61166, NousResearch#64797, NousResearch#43130. Partially addresses NousResearch#67544 (capture modes + secret redaction; user_id remains open).
Follow-up fixes from /hermes-pr-review + /simplify-code on PR NousResearch#83437: 1. Replace _redact_secrets with agent.redact.redact_sensitive_text(force=True) — the plugin's 11-pattern list was a strict subset of the 50+ patterns in agent/redact.py. Secrets like Stripe keys, Google API keys, GitLab tokens, HuggingFace tokens, DB connection strings, and Telegram bot tokens would all leak through the plugin's list but are caught by the existing redactor. Added pk-lf- (Langfuse public key) to _PREFIX_PATTERNS in agent/redact.py. 2. Remove dead 'not isinstance(client, object)' check in on_session_finalize — always False for any Python value. 3. Fix MoAClient.last_reference_metrics() to call the public self.chat.completions.last_reference_metrics() instead of reaching into the private _last_reference_metrics attribute via getattr. 4. Deduplicate _coerce_request_messages call in on_pre_llm_request — pass pre_coerced=input_messages to _messages_for_langfuse_input to avoid double-coercion + double _capture_content serialization per API request. 5. Add HERMES_LANGFUSE_CAPTURE to OPTIONAL_ENV_VARS in hermes_cli/config.py for consistency with the other HERMES_LANGFUSE_* env vars. 6. Fix test_sanitized_mode_redacts_secrets test data — the old samples ('sk-abc...1234', 'sk-ant...1234', 'Authorization: Bearer ***') were too short to match the regex thresholds and never actually tested redaction. Updated to realistic-length secrets and changed assertions to check that the output differs from input (redact_sensitive_text masks rather than inserting the literal string 'REDACTED').
alt-glitch
added
type/feature
…loop.py The diff-apply salvage introduced stale-base revert hunks — the PR was 1246 commits behind main, and its diff for conversation_loop.py and moa_loop.py silently dropped symbols added after the PR's base (e.g. _CODEX_ACK_CONTINUATION_NUDGE, _INTERRUPT_SCAFFOLD_MARKER, cache_ttl plumbing, finalize_turn import, _restore_user_after_reference_handoff). Restored both files to origin/main and re-applied only the PR's additive changes: _moa_reference_metrics_for_hook, _system_prompt_for_hooks, the system_prompt= and moa_references= hook kwargs, _last_reference_metrics attribute and accessors, and the slot_metrics population in the fan-out path. Fixes CI ImportError: cannot import name '_CODEX_ACK_CONTINUATION_NUDGE' from 'agent.conversation_loop'.
This was referenced Aug 13, 2026
Closed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Widens the bundled Langfuse observability plugin from 6 hooks to 11 and fixes two attribution bugs, salvaged from PR #83437 by @erosika with adopted fixes from 8 prior community PRs.
Changes
on_pre_llm_request/on_post_llm_callnow prefer the wire value (request["body"]["model"],response_model) over the stale agent attribute after/modelswitch or provider fallback.totalalongside the per-type breakdown. Subscription-included routes send no cost keys at all (Langfuse treats explicit$0as authoritative).api_request_error(closes failed generations with ERROR level),on_session_finalize/on_session_end(closes dangling traces),subagent_start/subagent_stop(traces delegated children as spans), MoA advisor fan-out (one generation per advisor priced at the advisor's own model).HERMES_LANGFUSE_CAPTURE=metadata|sanitized|full(defaultsanitized). Sanitized mode redacts secrets using the project-wideagent.redact.redact_sensitive_text(force=True).reason=shutdown(not on session rotation); atexit finalizer ends open root spans for short-lived processes; root context manager exited to prevent interpreter-teardown TypeError; TOCTOU on_get_langfuse()fixed with lock; reasoning_content surfaced; system prompt included in generation input for Anthropic/Codex/Bedrock; SDK v3update_tracereplacesset_trace_io.Follow-up fixes applied during salvage (from /hermes-pr-review + /simplify-code)
_redact_secretswithredact_sensitive_text(force=True)fromagent/redact.py— the plugin's 11-pattern list was a strict subset of the 50+ patterns in the existing redactor. Addedpk-lf-to_PREFIX_PATTERNS.not isinstance(client, object)check inon_session_finalize.MoAClient.last_reference_metrics()to call the public method instead of reaching into private_last_reference_metricsviagetattr._coerce_request_messagescall inon_pre_llm_request— passpre_coercedto avoid double-coercion + double_capture_contentserialization.HERMES_LANGFUSE_CAPTUREtoOPTIONAL_ENV_VARSinhermes_cli/config.py.test_sanitized_mode_redacts_secretstest data — old samples were too short to match regex thresholds and never actually tested redaction.Attribution
Salvaged from PR #83437 by @erosika, with adopted fixes from:
Validation
Closes / supersedes
Closes #29482, #43129, #72661.
Supersedes #81054, #82332, #42326, #39653, #64292, #32175, #61166, #64797, #43130.
Partially addresses #67544 (capture modes + secret redaction; user_id remains open).