feat(code): running session cost in status bar and usage

[Mason Daugherty (mdrxy)]

Closes #4519

Deep Agents Code now keeps a running estimated USD total for the active thread. The status bar shows it beside token usage, /cost and /tokens break it down on demand, and usage summaries include per-model cost. The total covers everything the thread spends — assistant turns, subagents, offload/compaction, and Auto mode's classifier — and it is restored on resume and thread switches.

---

Where the number lives

A thread's lifetime cost should be durable to support --resume and switching between threads. Consequently, the graph owns it and the client (e.g. TUI / headless CLI) can read it.

Concretely, _session_cost_usd is a new channel on the agent's state schema (CostState, which extends ResumeState), which means it is persisted by the graph's checkpointer, in the same checkpoint as messages.

FAQ: CostState, and why it extends ResumeState

CostState is the state schema CostTrackingMiddleware declares — the agent's state plus one channel:

class CostState(ResumeState):
    _session_cost_usd: Annotated[NotRequired[float], PrivateStateAttr, operator.add]

What ResumeState already holds. It's the schema for facts that belong to the thread as a whole rather than to any one message — the things the TUI/client needs the moment you open a thread, before the agent runs again:

• _context_tokens — how full the context window was after the last turn. This drives token readout — e.g., the display in the TUI's footer status bar, where the cost figure now also sits.
• _model_spec / _model_params — the model and settings the thread was using, so -r resumes on that model instead of whatever your current global default might happen to be.
• _goal_objective, _goal_rubric, _goal_status, _goal_status_note, _sticky_rubric, and the pending-proposal fields — the goal the thread is working toward (if one is set) and where progress on said goal stands.

Every field here is private, including the new one. A graph has a public interface: its input and output schemas say what callers may pass in and receive back (messages is the obvious example). None of the fields above belong in that interface, as they aren't things a caller supplies or asks for — they're notes that dcode writes for itself so it can pick the thread back up later.

PrivateStateAttr omits a field from those public input/output schemas. It does not make the field ephemeral or unreadable: the channel remains ordinary checkpointed state, is available to graph nodes, and is returned by get_state. Its writes can also appear as raw deltas in the updates stream; the live status bar uses a dedicated absolute-total event instead so the client does not have to reproduce the reducer (details below).

These values are stored rather than recomputed, because recomputing them is expensive or impossible after the fact. Reopening a thread has to show the context gauge and the current model immediately. The context number originally came from the last response's usage_metadata — the provider counted it for us. If it weren't saved, the CLI would have to load the whole message history and count tokens itself to guess at it: slow, dependent on having a tokenizer that matches the provider's, and still only an approximation of what was actually billed. Cost is the same shape of problem and worse — re-deriving it would mean re-pricing every message in the thread every time you open it.

So why extend ResumeState? Cost is the same kind of resumable thread fact, and pricing also needs the model information that ResumeState already carries. LangChain's AIMessage contract does not require a model identifier: usage_metadata standardizes token counts, while response_metadata is optional. Provider integrations commonly include a model name, but custom and third-party integrations are allowed to omit it. When a response does, pricing falls back to the _model_spec persisted for that turn.

Riding the checkpoint is a deliberate choice over an alternative where the client prices a turn and then pushes the new total with aupdate_state. (That's a second round trip that can be interrupted — e.g. Ctrl-C, a crash, a dropped connection — after the messages have already been committed.) Writing from inside the graph puts the cost in the same commit as the response that incurred it. Importantly, it also keeps an UpdateState run/span out of LangSmith traces if configured. Restoring is then a single aget_state read from the checkpoint.

Two things about the channel's declaration matter downstream:

• Additive. Its reducer is operator.add, so a writer returns only the delta it just priced — literally {"_session_cost_usd": 0.02}, never a replacement total. If a thread starts a turn at $1.00, after_model contributes $0.02 and a later rubric-grading drain contributes $0.005, the checkpoint ends at $1.025. Neither hook has to perform its own read-modify-write.
• Private. It carries PrivateStateAttr, keeping cost out of the graph's public input/output contract while leaving it checkpointed and readable through get_state. Deep Agents also uses this marker to identify state that must not cross the main-agent/subagent boundary.

The cumulative channel has one writer: the CostTrackingMiddleware instance belonging to the thread's main agent. Subagents run the same middleware, but theirs is constructed with nested=True, which switches its pricing off — a subagent's spend is charged by its parent instead, as the next section explains.

To get every completed model call to that writer, the cost-tracking module installs one process-wide Recorder. The Recorder is an in-memory inbox for completed model calls, implemented as a LangChain callback (BaseCallbackHandler). When a request finishes, it files the request's usage, model/provider metadata, and message ID under that thread. It does not calculate cost or write graph state; it only holds records until the main agent's middleware drains them.

flowchart LR
    subgraph calls["every model call in the run"]
        direction TB
        A["assistant"]
        B["subagent"]
        C["offload / summarization"]
        D["Auto mode classifier"]
        E["rubric grader"]
    end
    calls --> REC["in-memory model-call recorder<br/>collects usage, keyed by thread"]
    REC --> MW["CostTrackingMiddleware<br/>drains, prices, adds the delta"]
    MW --> CK[("checkpoint<br/>_session_cost_usd")]
    MW -. "stream event: absolute total" .-> BAR["status bar and /cost"]
    CK -. "resume, thread switch, end of turn" .-> BAR

Loading

The client is purely a reader; nothing outside the graph writes to the channel.

Why collecting is split from pricing

The obvious place to price a request is after_model, but that hook only ever sees the agent's own model node, and a large share of a thread's spend can happen somewhere else entirely:

Model call	Runs in
Assistant turn, Auto mode classifier	model node
Offload / summarization	before_model (stock) or the model wrap chain (deepagents)
Subagent turns	tool node, in a child graph
compact_conversation	tool node
Rubric grading	after_agent

The Recorder and the middleware have complementary access:

• The Recorder sits at LangChain's callback layer, so it can observe model calls made by the main agent, a subagent, summarization, or another direct invocation. But a callback is not a graph node and cannot return an update to _session_cost_usd.
• CostTrackingMiddleware runs inside the main agent's graph, so its hooks can return an additive update that the checkpointer persists. But those hooks do not run inside a child graph or around a bare side invocation, so they cannot discover all spend on their own.

The in-memory queue bridges those two scopes. The callback places one record in the queue when a request completes; CostTrackingMiddleware drains the current thread's records in after_model and after_agent, prices them, and returns their sum as one state update.

Example: one turn with hidden model work

Suppose a thread already sits at $1.00. Before the main response, summarization spends $0.004; the main response then spends $0.020. The Recorder files both completions under the thread. after_model drains those two records and writes +0.024, taking the checkpoint to $1.024.

If rubric grading then spends $0.005 after the final model step, after_agent drains that later record and writes +0.005. The committed total is $1.029. A subagent call works the same way: its record waits in the shared thread queue until the parent's next drain.

Why the main response is also checked in state

Normally, a main-agent response with message ID m1 produces a Recorder entry for m1. The middleware prices that entry, remembers that m1 was charged, and skips the copy of m1 already present in graph state.

If a custom or third-party integration returns an AIMessage with usage metadata but no usable callback record was filed — for example, the callback could not associate the request with a thread — after_model can still price that main response from state. Comparing message IDs lets the normal and fallback paths coexist without double-counting. This fallback is limited to the main response; hidden calls are not in the parent's state and therefore still rely on the Recorder.

How subagent spend is counted

An earlier version of this PR had subagents accumulate into their own copy of the channel and merge the total back. That silently did nothing because Deep Agents intentionally treats PrivateStateAttr as agent-local state: SubAgentMiddleware discovers those keys and strips them both when seeding a child and when merging its result. This prevents one agent's internal bookkeeping from leaking into another and avoids parallel subagents racing to write private LastValue channels. A parent-visible aggregate would need a separate shared/public contract; for cost, the thread-keyed recorder supplies that bridge without weakening private-state isolation.

Since a subagent inherits its parent's thread ID, its completed model calls land in the same in-memory recorder queue. The parent drains and prices those records on its next after_model or after_agent hook:

sequenceDiagram
    participant P as Main agent
    participant R as In-memory recorder
    participant S as Subagent
    participant C as Checkpoint
    P->>R: assistant call usage
    R-->>P: drain 1 record
    P->>C: add priced delta
    P->>S: task tool
    S->>R: nested call usage
    Note over S: Nested cost middleware does not<br/>price or accumulate spend
    S-->>P: result
    P->>R: next assistant call usage
    R-->>P: drain nested + assistant records
    P->>C: add priced delta

Loading

Keeping the status bar live

The updates stream can expose the private channel's writes, but those writes are reducer inputs such as +0.02, not the post-reducer lifetime total. Making the client accumulate them would create a second copy of the graph's state machine, vulnerable to missed stream chunks. Instead, the writer emits a small custom event carrying the absolute total and the client applies it with set_cost.

How an absolute event recovers from a dropped update

Suppose the client shows $1.00. The graph commits another $0.02 and emits $1.02, but that event is dropped. A later $0.005 charge emits $1.025. Because the payload is an absolute value, the client replaces $1.00 with $1.025 and is correct again; a delta-only stream would leave it permanently short by the missed $0.02. The end-of-turn checkpoint read provides another reconciliation point if the final event is the one that disappears.

The provisional number exists only to make the status bar react before the graph reaches its next cost-tracking hook. These two client-side fields have different jobs:

• _session_cost_usd is the latest absolute total received from the graph. It is the durable value restored from the checkpoint.
• _provisional_cost_usd is a temporary display estimate derived from usage on the message stream. It is never written to graph state.

For a request estimated at $0.02, the display moves through this sequence:

Moment	Durable total	Provisional	Status bar
Before the request	$1.00	$0.00	$1.00
Usage arrives on the message stream	$1.00	$0.02	$1.02
The graph emits its new absolute total	$1.02	$0.00	$1.02

The last step replaces the durable total and clears the provisional bucket, so the same request is not shown twice. The displayed number stays steady at $1.02; only its source changes from an early estimate to checkpointed state. On a completed turn, the final checkpoint read provides the same reconciliation if the custom event was missed.

/cost's per-type and per-model rows remain client-side and stay labeled “since this thread was loaded” — only the lifetime total above them comes from the graph.

Pricing

Pricing stays in one place: estimate_cost is the only function that touches canonical price data, currently through genai-prices. For each request it needs split input/output token counts, the model, and — when the model name is available from more than one host — the provider. Provider names are normalized to the identifiers in the price catalog; if response metadata omits the model or provider, the checkpointed model spec is the fallback.

Here are the main pricing paths.

Ordinary input and output. Suppose a catalog entry charges $2 per million input tokens and $8 per million output tokens. A request with 10,000 input tokens and 2,000 output tokens is estimated as:

input:  10,000 / 1,000,000 × $2 = $0.020
output:  2,000 / 1,000,000 × $8 = $0.016
total:                              $0.036

(Those rates are illustrative; genai-prices supplies the actual model/provider rates.)

Provider-specific lookup. The response's provider matters when the same model family is sold through different hosts. For example, LangChain's azure_openai provider name is normalized to the catalog's azure identifier, while a direct OpenAI request uses openai. This prevents a recognizable model name from silently selecting the wrong host's price.

Cached input. LangChain's input_tokens is inclusive of cache reads and writes. If a response reports 1,000 input tokens with 600 cache-read tokens and 100 cache-write tokens, only 300 are ordinary input:

ordinary input: 1,000 - 600 - 100 = 300
cache reads:                            600
cache writes:                           100

estimate_cost passes all three buckets to genai-prices, which applies the ordinary-input, cache-read, and cache-write rates separately. Output tokens are priced independently. If malformed provider metadata says the cache buckets exceed the inclusive input count, the tracker clamps them to the input total rather than creating a negative ordinary-input count.

Anthropic cache TTLs. When Anthropic reports its TTL breakdown, LangChain sets generic cache_creation to zero and exposes ephemeral_5m_input_tokens and ephemeral_1h_input_tokens instead. If those fields contain 100 and 50 tokens, the tracker passes 150 cache-write tokens rather than accidentally billing them as ordinary input. genai-prices currently exposes one cache-write bucket, so the two TTL buckets share that catalog estimate.

The estimator declines to guess when the available data cannot support a defensible calculation:

• A response with only total_tokens=1_200 is unpriceable because those tokens could be input, output, or a mixture, and the rates usually differ.
• An unknown model has no canonical catalog rate to apply.
• Subscription-style access such as the Codex provider is not equivalent to per-token API billing.
• Malformed or non-finite usage/price data is ignored rather than allowed to fail the model turn.

In each case estimate_cost returns None; the request is omitted from the dollar total while its token usage can still appear in /tokens.

[socket-security]

Review the following changes in direct dependencies. Learn more about Socket for GitHub.

Diff	Package	Supply Chain Security	Vulnerability	Quality	Maintenance	License
	pypi/​genai-prices@​0.1.0

View full report

[open-swe]

Open SWE Review found 2 potential issues.

Open in Web • View Open SWE trace

[open-swe]

Open SWE Review found 1 potential issue.

Open in Web • View Open SWE trace

[Alexander Olsen (aolsenjazz)]

I'm not convinced that we're helping ourselves with graph-state-level cost tracking here. Imo it's making the system more complex than it needs to be and is out of scope of main lift for this feature

Given that we're already accumulating the cost client-side and the client-side accumulation is built to segment by model etc, the server-side accumulation feels unrelated + like it's shoehorned into the client-side impl

Better imo to simplify and make client-side SSoT. What do you think?

[Mason Daugherty (mdrxy)]

Alexander Olsen (@aolsenjazz) This was a fair concern with the version you reviewed. At that point, the graph and client both participated in accumulation/persistence, so the graph-side piece did feel bolted onto the client implementation.

I’ve since reworked the ownership boundary: the graph is now the sole writer of the total, while the client only renders that total and keeps an explicitly provisional, since-loaded breakdown for /cost. There is no client-side checkpoint write anymore.

I think this is the better long-term boundary because the graph is the only layer that can reliably account for the entire thread—not just assistant messages, but also subagents, summarization/offload, Auto classification, compaction, and rubric grading. Making each client reconstruct and persist that total would duplicate pricing logic across TUI, headless, and future clients, and hidden invocation paths would be easy to miss.

Keeping the total in private graph state also makes the response and its cost part of the same checkpoint. That avoids a second aupdate_state round trip that could be interrupted after the response was already committed, and resume/thread switching becomes a single state read.

So I agree that graph-level tracking was hard to justify in the earlier dual-writer shape. In the current shape, it gives us one durable source of truth at the layer that sees all spend, while leaving presentation-only detail in the client. I think that separation will hold up much better as we add clients and new model-call paths.

[open-swe]

Open SWE Review found 1 potential issue.

Open in Web • View Open SWE trace

[open-swe]

Open SWE Review found 1 potential issue.

Open in Web • View Open SWE trace

[Mason Daugherty (mdrxy)]

Looked into models.dev as an alternative pricing source. It's a static JSON catalog (models.dev/api.json, ~3 MB) with an npm SDK — no Python package and no pricing function, so we'd vendor a snapshot and write the rate arithmetic ourselves. That's feasible since everything funnels through estimate_cost, but comparing the data on what this PR actually uses:

• Cache writes: models.dev has a single flat cache_write rate. No 5m/1h TTL split, so Anthropic 1h cache writes lose their premium rate (genai-prices prices them).
• Tiered pricing: ~250 models have context_over_200k-style tiers. genai-prices applies these; with models.dev we'd implement tier selection ourselves or underprice long-context requests.
• Provider coverage: no aws/bedrock providers at all, and IDs differ elsewhere (xai vs x-ai), so _PROVIDER_ALIASES would need rework and Bedrock pricing would become a coverage gap.
• Freshness: models.dev is actively community-maintained, but new models frequently land with $0.00 placeholder pricing before real rates are filled in.

The one thing models.dev offers that genai-prices doesn't is a long tail of obscure providers, but we only ever price against the provider the request ran on, so that breadth doesn't help us.

Net: genai-prices is the better fit on rate fidelity and provider coverage, and the integration here is already written against its usage model (inclusive token totals, TTL buckets, the audio/cache-read overlap constraint). Keeping it. If data staleness in genai-prices becomes a problem, bumping the pin is a much smaller change than swapping sources.

[Mason Daugherty (mdrxy)]

A question came up about whether the cost data source here could come from LangSmith instead of being computed locally with genai-prices. I dug into what's actually possible and wanted to write it down, since "use LangSmith" turns out to be several different things with very different implications:

1. Usage + cost from the LangSmith API (query runs server-side). Technically possible via list_runs, but it inverts this PR's architecture. Trace ingestion is async, so the status bar and /cost couldn't be live mid-turn; the feature silently degrades whenever LANGSMITH_TRACING is off, a self-hosted/proxied endpoint is in use, or there's no API key; and the checkpointed _session_cost_usd channel + session_cost stream events would have to be reworked around eventual consistency and a network dependency. This loses the durable, offline, tracing-optional, works-with-remote-graph properties the design is built around.

2. Rates from LangSmith's pricing table, computed locally. Doesn't exist — there's no public API to pull the model pricing map, and the langsmith SDK ships no local rate catalog. Cost is only ever computed server-side at ingestion.

3. Send client-computed costs to LangSmith (usage_metadata.total_cost et al.). This is supported per the LangSmith cost-tracking docs: costs you send are stored verbatim and the server doesn't re-run its own pricing on that run (note it's all-or-nothing per run — send total_cost only and the prompt/completion breakdown isn't backfilled). But this is additive, not a replacement: the client still needs its own price table to produce the dollars, so genai-prices stays. It also only flows when tracing is enabled.

Net: keeping local estimation with genai-prices is the right call for what this PR is doing. Option 3 is worth considering separately and only if we want the LangSmith UI dashboards to show the exact same numbers the CLI does — it's a small, optional, tracing-only addition (write total_cost onto the run tree where _SessionCostRecorder already sees the priced usage), not a data-source swap.