Skip to content

fix(agent): sticky cost_status priority ladder (#67764) - #67790

Open
DavidMetcalfe wants to merge 2 commits into
NousResearch:mainfrom
DavidMetcalfe:fix/cost-status-priority-ladder
Open

fix(agent): sticky cost_status priority ladder (#67764)#67790
DavidMetcalfe wants to merge 2 commits into
NousResearch:mainfrom
DavidMetcalfe:fix/cost-status-priority-ladder

Conversation

@DavidMetcalfe

Copy link
Copy Markdown
Contributor

Fixes #67764.

Problem

cost_status is supposed to communicate the accuracy confidence of an
accumulated dollar value across a session's per-call rows. Today it's
overwritten by every new API call (latest-call-wins), so a session with
100 actual-confidence calls followed by 1 estimated call downgrades the
whole accumulated row to "estimated" — even though the user-visible
fact "the cost is whatever OpenRouter said" hasn't changed.

Three layers exhibit the bug: SQL UPDATE on sessions (lines 2887, 2908),
SQL UPSERT on session_model_usage (line 3087), and the in-memory
agent.session_cost_status writes at agent/conversation_loop.py:2321,
agent/codex_runtime.py:150. The /insights aggregator (agent/insights.py:583)
had the same shape in Python. tools/delegate_tool.py:2862 had a bespoke
"sticky-ish" guard which is unified onto the same helper.

Fix

Sticky-max rule across all layers: actual (3) > included (2) > estimated
(1) > unknown (0). Once any call has reported actual, the accumulated
row stays actual forever.

  • New helper agent/usage_pricing.py::sticky_cost_status(current, new)
    returns the higher-rank value (with sentinel -1 for missing/invalid
    current, so any valid new wins; equal ranks return the new value; bad
    values fall through to "estimated").
  • SQL COALESCE replaced with a CASE-expression ladder at three sites.
    The interpolated new value is matched against a Literal allow-list
    before embedding (SQL-injection safety). Verified by
    test_sessions_status_with_unknown_literal_input.
  • In-memory assignments in agent/conversation_loop.py and
    agent/codex_runtime.py now call sticky_cost_status(...)
    instead of direct =. agent/insights.py:583 does the same for the
    per-model dict loop.
  • tools/delegate_tool.py replaces its bespoke guard with the same
    helper for codebase consistency (and so the rule evolves in one place).
  • cost_source is deliberately unchanged at all four sites (per the brief's
    Layer 4: no clean priority ladder for cost_source).

Tests

  • 30 tests in tests/test_cost_status_priority_ladder.py (new file)
    covering: rank mapping, sticky behavior on each pair of statuses,
    return-type Literal guarantee, SQL UPSERT accumulation, sessions
    aggregate SQL aggregation, SQL injection sanitization, in-memory
    dispatch in conversation_loop / codex_runtime / insights / delegate_tool,
    end-to-end brief reproduction scenarios.
  • Existing tests/hermes_state/, tests/run_agent/test_notice_spine.py,
    tests/agent/test_insights.py, tests/tools/test_delegate*.py: no regressions.

Out of scope

  • Phase 2 (wire actual_cost_usd from provider responses): still
    a separate deferred issue. Once Phase 2 lands, the priority ladder
    here will gain a real "actual" promotion path.
  • cost_source aggregation: not changed (no clean priority ladder).
  • Memory / rehydration compatibility: this PR is independent of
    PR fix(agent): rehydrate session cost counters on agent construction (#67762) #67770 (rehydration). Rehydration stays correct; cost_status
    reaches the agent via the same in-memory attribute paths and is
    now also sticky across those writes.

@alt-glitch alt-glitch added type/bug Something isn't working P2 Medium — degraded but workaround exists comp/agent Core agent runtime: loop, agent_init, prompt builder, context-compression, responses endpoint area/billing Account usage, credit usage, billing (cross-cutting) area/usage-cost Token accounting, usage reporting, billing, cost tracking sweeper:risk-session-state Sweeper risk: may lose/corrupt/mis-associate session or context state needs-decision Awaiting maintainer decision before any implementation labels Jul 20, 2026
@alt-glitch

Copy link
Copy Markdown
Collaborator

This was generated by AI during triage.

Related to #67764 and open #67770. This is the active reimplementation of closed, unmerged #67774's sticky-status repair; #67770 separately handles restart rehydration. The overlapping SessionDB work needs consolidation rather than duplicate closure.

Replaces the previous "most-recent-call-wins" semantics across all
four layers of cost_status aggregation:

- SQL COALESCE in hermes_state.py at three sites (sessions aggregate
  absolute + incremental paths, session_model_usage ON CONFLICT)
- In-memory attribute writes in agent/conversation_loop.py:2321
  and agent/codex_runtime.py:150
- agent/insights.py:583 per-model dict aggregation
- tools/delegate_tool.py:2862 subagent-fold guard (now via the same
  helper for codebase consistency)

Sticky-max rule: actual (rank 3) > included (rank 2) > estimated
(rank 1) > unknown (rank 0). Once any call has reported actual,
the accumulated row stays actual forever — even if subsequent calls
report estimated or unknown. SQL injection safety via allow-list
sanitization of the interpolated new value.

The new sticky_cost_status helper lives in agent/usage_pricing.py
alongside the CostStatus literal. cost_source is deliberately
unchanged at all four sites (per the brief's Layer 4: no clean
priority ladder).

Closes NousResearch#67764

Tests: tests/test_cost_status_priority_ladder.py (30 tests covering
helper, SQL, dispatch shape, dispatch behavior, end-to-end brief
scenarios). 174+ existing adjacent tests pass with no regressions.
When cost_status=None (token-only calls with no new cost information),
the SQL CASE ladder was coercing None to 'estimated' and overwriting
the accumulated status — diverging from both the Python
sticky_cost_status() helper and the original COALESCE(?, cost_status)
semantics.

Fix: skip the sticky ladder when cost_status is None, falling back to
COALESCE(NULL, cost_status) which preserves the existing row value.

Also sanitize the INSERT binding in _record_model_usage() to match
the ON CONFLICT DO UPDATE path — unrecognised non-None values now
fall back to 'estimated' on initial insert instead of raw literal
injection.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

area/billing Account usage, credit usage, billing (cross-cutting) area/usage-cost Token accounting, usage reporting, billing, cost tracking comp/agent Core agent runtime: loop, agent_init, prompt builder, context-compression, responses endpoint needs-decision Awaiting maintainer decision before any implementation P2 Medium — degraded but workaround exists sweeper:blast-broad Sweeper blast radius: broad — a core path most sessions hit sweeper:risk-compatibility Sweeper risk: may break existing users, config, migrations, defaults, or upgrades sweeper:risk-session-state Sweeper risk: may lose/corrupt/mis-associate session or context state type/bug Something isn't working

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[Bug]: cost_status overwritten on every API call across SQL, in-memory, and /insights aggregations

3 participants