Skip to content

fix(langfuse): include Anthropic system prompt in generation input - #32175

Closed
db-aeon wants to merge 1 commit into
NousResearch:mainfrom
db-aeon:fix/langfuse-include-anthropic-system-prompt
Closed

fix(langfuse): include Anthropic system prompt in generation input#32175
db-aeon wants to merge 1 commit into
NousResearch:mainfrom
db-aeon:fix/langfuse-include-anthropic-system-prompt

Conversation

@db-aeon

@db-aeon db-aeon commented May 25, 2026

Copy link
Copy Markdown
Contributor

Summary

When using Anthropic (and other providers that split system out of messages), Langfuse LLM call generation observations only showed user/assistant turns — the Hermes system prompt (skills, memory, context) was missing from the trace input.

This change:

  • Passes system_prompt from api_kwargs["system"] (with fallback when the first message is role: system) through the pre_api_request hook in agent/conversation_loop.py
  • Prepends a serialized role: system message to Langfuse generation input in plugins/observability/langfuse
  • Adds system_prompt_chars metadata when a system block is logged
  • Documents verification in the Langfuse plugin README

Why

Anthropic Messages API does not include the system prompt in messages; Hermes already builds api_kwargs["system"] separately. The Langfuse plugin only serialized request_messages, so hosted traces looked like conversations without the agent instructions — making debugging prompt/cache behavior difficult.

Test plan

  • pytest tests/plugins/test_langfuse_plugin.py (39 passed locally)
  • Enable observability/langfuse, run hermes chat -q "hello", open LLM call in Langfuse and confirm role: system appears first in generation input

Made with Cursor

@db-aeon @cursoragent
fix(langfuse): include Anthropic system prompt in generation input
1700a87 
Anthropic Messages API passes system via a separate `system` kwarg, so
Langfuse LLM call observations only showed user/assistant messages.
Pass `system_prompt` through pre_api_request and prepend it to the
serialized generation input when it is not already in messages.

Co-authored-by: Cursor <cursoragent@cursor.com>
@db-aeon

db-aeon commented May 25, 2026

Copy link
Copy Markdown
Contributor Author

Human here: pretty self explanatory: system messages were not being displayed in the langfuse traces. Not sure if this is the best approach, but does work.

@alt-glitch alt-glitch added type/bug Something isn't working P3 Low — cosmetic, nice to have comp/plugins Plugin system and bundled plugins comp/agent Core agent runtime: loop, agent_init, prompt builder, context-compression, responses endpoint labels May 25, 2026
@teknium1

teknium1 commented Jul 13, 2026

Copy link
Copy Markdown
Contributor

Thanks for tracing the provider-specific request shape. The premise is confirmed on current main: agent/anthropic_adapter.py:2421-2435 extracts system messages and agent/anthropic_adapter.py:2601-2602 sends them as a separate system kwarg, while agent/conversation_loop.py:1212-1259 forwards only request messages to the hook. plugins/observability/langfuse/__init__.py:854-896 then serializes that message list as generation input.

Suggested changes

  • Add one regression test that crosses the hook boundary and verifies the separate Anthropic system kwarg reaches the Langfuse generation input. The tests introduced by commit 1700a87131f8 directly cover the helper, but not the conversation_loop.py forwarding path that caused the regression.

Automated hermes-sweeper review.

@teknium1 teknium1 added sweeper:risk-security-boundary Sweeper risk: may affect sandboxing, auth, credentials, or sensitive data sweeper:risk-compatibility Sweeper risk: may break existing users, config, migrations, defaults, or upgrades sweeper:blast-contained Sweeper blast radius: contained — one narrow path / opt-in / few users labels Jul 13, 2026
@FnExpress FnExpress mentioned this pull request Jul 14, 2026
Closed
4 tasks
This was referenced Jul 27, 2026
Open
Open
This was referenced Aug 3, 2026
Open
Open
Open
Open
Open
Open
Open
Open
Open
erosika added a commit to erosika/hermes-agent that referenced this pull request Aug 10, 2026
@erosika @FnExpress @db-aeon
fix(langfuse): include system prompt in generation input
09e7745 
Providers that move the system prompt out of messages made it vanish
from traces: Anthropic Messages carries it as a separate system kwarg
(str or content-block list) and the Responses/Codex API as top-level
instructions, so generation inputs showed conversations without the
agent's instructions, skills, or memory. conversation_loop now derives
the system prompt as actually sent to the provider and forwards it to
hooks; the plugin prepends a role: system entry when messages don't
already carry one. Serialization routes through _capture_content so
capture modes apply to system prompts too.

Adopted from NousResearch#64292, which extends NousResearch#32175's Anthropic fix to the
Codex/Responses path — thanks @FnExpress and @db-aeon.

Co-authored-by: FnExpress <37214785+FnExpress@users.noreply.github.com>
Co-authored-by: Dan Benyamin <db@project-aeon.com>
@erosika erosika mentioned this pull request Aug 10, 2026
Closed
@erosika

erosika commented Aug 10, 2026

Copy link
Copy Markdown
Contributor

the Anthropic-system fix landed in #83437 via #64292, which extends it to Codex instructions and Bedrock Converse blocks — with co-author credit to you on the commit. thanks for identifying the original gap.

@alt-glitch alt-glitch added provider/anthropic Anthropic native Messages API and removed sweeper:risk-security-boundary Sweeper risk: may affect sandboxing, auth, credentials, or sensitive data labels Aug 11, 2026
@kshitijk4poor kshitijk4poor mentioned this pull request Aug 13, 2026
Merged
kshitijk4poor added a commit that referenced this pull request Aug 13, 2026
@kshitijk4poor
feat(langfuse): widen tracing to errors, sessions, subagents, and MoA…
e665300 
… fan-out

Salvaged from PR #83437 by @erosika, with adopted fixes from @bgodlin (#81054),
@aldoeliacim (#82332), @nftpoetrist (#42326), @rodboev (#39653), @FnExpress
(#64292, supersedes #32175 by @db-aeon), @Per0-1 (#61166), @NaMinhyeok (#64797),
and @liuhao1024 (#43130).

Widens the bundled Langfuse plugin from 6 to 11 hooks and fixes two
attribution bugs. Also adopts shutdown/atexit lifecycle fixes and composes
8 prior community PRs with interaction-fix follow-ups.

Model attribution: on_pre_llm_request and on_post_llm_call now prefer the
wire value (request body model, response model) over the agent attribute,
which goes stale after /model switch or provider fallback.

Cost total: both cost paths now send a summed total alongside the per-type
breakdown, since Langfuse does not derive calculatedTotalCost from
cost_details keys. Subscription-included routes send no cost keys at all.

New coverage: api_request_error closes failed generations with ERROR level;
on_session_finalize/on_session_end close dangling traces for tool-only and
interrupted turns; subagent_start/subagent_stop trace delegated children as
spans; MoA advisor fan-out emits one generation per advisor priced at the
advisor's own model.

Capture modes: HERMES_LANGFUSE_CAPTURE=metadata|sanitized|full (default
sanitized). Sanitized mode redacts secret patterns before truncation.

Adopted lifecycle fixes: shutdown client at session finalize when
reason=shutdown (not on session rotation); atexit finalizer ends open root
spans for short-lived processes; root context manager exited to prevent
interpreter-teardown TypeError; TOCTOU on _get_langfuse() fixed with lock;
reasoning_content surfaced in traces; system prompt included in generation
input for Anthropic/Codex/Bedrock; SDK v3 update_trace replaces set_trace_io.

Closes #29482, #43129, #72661.
Supersedes #81054, #82332, #42326, #39653, #64292, #32175, #61166, #64797, #43130.
Partially addresses #67544 (capture modes + secret redaction; user_id remains open).
@kshitijk4poor

kshitijk4poor commented Aug 13, 2026

Copy link
Copy Markdown
Collaborator

Merged via #85439 — your fix was adopted and composed into the wider Langfuse tracing PR by @erosika. Your contribution is credited in the commit body. Thanks @db-aeon!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

No reviews

Assignees

No one assigned

Labels

comp/agent Core agent runtime: loop, agent_init, prompt builder, context-compression, responses endpoint comp/plugins Plugin system and bundled plugins P3 Low — cosmetic, nice to have provider/anthropic Anthropic native Messages API sweeper:blast-contained Sweeper blast radius: contained — one narrow path / opt-in / few users sweeper:risk-compatibility Sweeper risk: may break existing users, config, migrations, defaults, or upgrades type/bug Something isn't working

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

5 participants

@db-aeon @teknium1 @erosika @kshitijk4poor @alt-glitch