fix(langfuse): omit explicit zero cost_details for subscription-included providers (#43129) - #43130
Closed
liuhao1024 wants to merge 1 commit into
Closed
Conversation
…ded providers
For subscription-included providers (e.g. openai-codex), get_pricing_entry()
returns a PricingEntry with all-zero costs. Both Langfuse code paths
(_usage_and_cost response-object path and post_api_request dict path) checked
`entry.input_cost_per_million is not None` which is True for Decimal("0"),
leading to explicit 0.0 values in cost_details.
Langfuse treats provided cost_details as authoritative and will not
recalculate estimated cost from model pricing when zeros are present.
Omitting cost_details lets Langfuse fall back to its own model-based cost
estimation.
Fixes NousResearch#43129
alt-glitch
added
type/bug
1 task
9 tasks
Contributor
|
Thanks for the focused regression fix. The current-main code still emits zero-valued Langfuse costs in both paths: the response-object path at The PR’s two guards cover those two active paths and its tests preserve non-subscription behavior. It adds no core surface, configuration, or cache-sensitive behavior. Its June base is older, but the target logic remains structurally present on current main, making this a high-value salvage candidate. Automated hermes-sweeper review. |
teknium1
added
sweeper:risk-compatibility
19 tasks
teknium1
added
the
area/usage-cost
erosika
added a commit
to erosika/hermes-agent
that referenced
this pull request
Aug 10, 2026
Subscription-included routes (e.g. openai-codex) priced every request at explicit $0: get_pricing_entry returns zero rates for these routes, and Langfuse treats provided cost_details as authoritative — the zeros blocked its own model-based cost estimation, so every generation showed $0 forever (NousResearch#43129). _canonical_usage_and_cost now resolves the billing route first and sends no cost keys at all for included routes, letting Langfuse fall back to its own pricing. Usage details still export. Adopted from NousResearch#43130 — thanks @liuhao1024. Co-authored-by: liuhao1024 <sunsky.lau@gmail.com>
Contributor
kshitijk4poor
added a commit
that referenced
this pull request
Aug 13, 2026
… fan-out Salvaged from PR #83437 by @erosika, with adopted fixes from @bgodlin (#81054), @aldoeliacim (#82332), @nftpoetrist (#42326), @rodboev (#39653), @FnExpress (#64292, supersedes #32175 by @db-aeon), @Per0-1 (#61166), @NaMinhyeok (#64797), and @liuhao1024 (#43130). Widens the bundled Langfuse plugin from 6 to 11 hooks and fixes two attribution bugs. Also adopts shutdown/atexit lifecycle fixes and composes 8 prior community PRs with interaction-fix follow-ups. Model attribution: on_pre_llm_request and on_post_llm_call now prefer the wire value (request body model, response model) over the agent attribute, which goes stale after /model switch or provider fallback. Cost total: both cost paths now send a summed total alongside the per-type breakdown, since Langfuse does not derive calculatedTotalCost from cost_details keys. Subscription-included routes send no cost keys at all. New coverage: api_request_error closes failed generations with ERROR level; on_session_finalize/on_session_end close dangling traces for tool-only and interrupted turns; subagent_start/subagent_stop trace delegated children as spans; MoA advisor fan-out emits one generation per advisor priced at the advisor's own model. Capture modes: HERMES_LANGFUSE_CAPTURE=metadata|sanitized|full (default sanitized). Sanitized mode redacts secret patterns before truncation. Adopted lifecycle fixes: shutdown client at session finalize when reason=shutdown (not on session rotation); atexit finalizer ends open root spans for short-lived processes; root context manager exited to prevent interpreter-teardown TypeError; TOCTOU on _get_langfuse() fixed with lock; reasoning_content surfaced in traces; system prompt included in generation input for Anthropic/Codex/Bedrock; SDK v3 update_trace replaces set_trace_io. Closes #29482, #43129, #72661. Supersedes #81054, #82332, #42326, #39653, #64292, #32175, #61166, #64797, #43130. Partially addresses #67544 (capture modes + secret redaction; user_id remains open).
Collaborator
|
Merged via #85439 — your fix was adopted and composed into the wider Langfuse tracing PR by @erosika. Your contribution is credited in the commit body. Thanks @liuhao1024! |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
5 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What does this PR do?
Fixes Langfuse showing $0 cost for subscription-included providers (e.g.
openai-codex). Both the response-object path and thepost_api_requestdict path were sending explicit zero-valuedcost_detailsto Langfuse, which treats them as authoritative and never recalculates from model pricing.Related Issue
Fixes #43129
Type of Change
Changes Made
plugins/observability/langfuse/__init__.py: In_usage_and_cost(), gate cost_details population oncost.status != "included"so subscription-included routes produce empty cost_details instead of explicit zeros. In thepost_api_requestdict path, checkresolve_billing_route().billing_mode != "subscription_included"before computing per-type costs from the pricing entry.tests/plugins/test_langfuse_plugin.py: AddedTestSubscriptionIncludedCostOmissionwith 4 regression tests covering both code paths (subscription-included → empty cost_details, normal provider → populated cost_details).How to Test
pytest tests/plugins/test_langfuse_plugin.py -xvs— all 43 tests pass_usage_and_cost()returns emptycost_detailsforopenai-codex(subscription_included)_usage_and_cost()returns populatedcost_detailsforopenai(normal provider)on_post_llm_calldict path returns emptycost_detailsforopenai-codexon_post_llm_calldict path returns populatedcost_detailsforopenaiChecklist
Code
fix(scope):,feat(scope):, etc.)pytest tests/ -qand all tests passDocumentation & Housekeeping
docs/, docstrings) — or N/Acli-config.yaml.exampleif I added/changed config keys — or N/ACONTRIBUTING.mdorAGENTS.mdif I changed architecture or workflows — or N/ACode Intelligence
_usage_and_cost,on_post_llm_call,get_pricing_entry,estimate_usage_cost,resolve_billing_routeestimate_usage_costreturnsCostResult(amount_usd=Decimal("0"), status="included")for subscription routes;get_pricing_entryreturnsPricingEntrywith all_ZEROcosts. Both paths now check the billing status before populating cost_details.