feat(acp): report standard adapter usage - #4950
Conversation
e20cb57 to
0540e3d
Compare
|
I validated this problem independently while building an owner-side NIP-AM usage projection for a mixed Codex/Claude/Goose fleet. This PR covers the important upstream seam, and its separate standard-adapter tracker is a better fit than duplicating the local integration. Three attribution/coverage details from that deployment may be useful here or in a focused follow-up:
Cache-write preservation and the prompt-response token path are already covered here (and overlap #4000), so I do not plan to open a competing ACP usage PR. I can prepare a small dependent follow-up for these three points once the desired behavior and landing order are clear. |
0540e3d to
b542ac8
Compare
wesbillman
left a comment
There was a problem hiding this comment.
Reviewing on Wes's behalf.
The new standard-adapter mapping publishes usage.inputTokens directly while also publishing cachedReadTokens/cachedWriteTokens. Both supported adapters define the ACP prompt-response inputTokens as non-cached input: Codex explicitly subtracts cached input in toTokenCount(), and Claude's sessionUsage() keeps fresh input, cache reads, and cache writes as separate accumulators. NIP-AM requires the opposite representation: inputTokens is the inclusive input-side total, and cache fields are informational subsets (docs/nips/NIP-AM.md, Numeric validity and token semantics; also TokenCounts' contract in crates/buzz-core/src/agent_turn_metric.rs).
As written, a Claude turn with inputTokens=100, cachedReadTokens=30, and cachedWriteTokens=25 publishes input=100 rather than 155; the added test currently pins that undercount. Codex has the same issue for cache reads. This corrupts NIP-AM input totals and any aggregation/cost logic built on them.
Please normalize standard usage before constructing TurnUsage: checked-add cache read/write into turn_input_tokens, fail closed on overflow, and update the wire tests to assert the inclusive input value while preserving each cache subset separately.
b542ac8 to
b259a8b
Compare
|
Rebased onto latest PR #4000 does not make this PR redundant. #4000 adds the cumulative Per Wes's review, this revision now normalizes standard-adapter input to NIP-AM's inclusive contract with checked arithmetic:
It also preserves #4000/current-main's fail-closed optional cumulative counters, cache-write support, and pricing identity, while keeping Goose/buzz-agent usage exclusive if both producer paths emit data. Wes's broader attribution/coverage suggestions—effective model fallback, normalized provider-neutral harness labels, and optional coverage-only records for completed turns with no usage—remain useful follow-up scope; they are not supplied by #4000 and are not folded into this narrowly rebased ingestion change. Validation: — Brainy Bumble |
wpfleger96
left a comment
There was a problem hiding this comment.
🤖 Combined + deduped feedback from the dual review I ran on this (two independent passes; full writeups on our side). The current head already fixes the inclusive-input issue from the earlier review round — checked-add of cache read/write with fail-closed overflow looks right. What's left: two accounting gaps both passes converged on (the cost baseline and the initial_message lifecycle), plus a few nits. Overall the shape is exactly what we hoped for next to #4000: goose stays exclusive, standard adapters fill the gap, and omitting pricingIdentity/model for these records is the correct fail-closed call vs #4629's exact-lookup contract.
Co-authored-by: Atish Patel <atish@squareup.com> Signed-off-by: Atish Patel <atish@squareup.com> Signed-off-by: Brainy Bumble <0ed7657b57c0e8a9f5288390dd6c8d5d0a3a06abe9b01b9006814f52077d6cdf@buzz.block.builderlab.xyz>
Co-authored-by: Atish Patel <atish@squareup.com> Signed-off-by: Atish Patel <atish@squareup.com>
b259a8b to
001af63
Compare
wpfleger96
left a comment
There was a problem hiding this comment.
🤖 Verified all five findings from the combined review are addressed at head 001af63a:
- First-turn Claude cost: standard tracker now has per-session cost state seeded to zero via
notify_session_spawned()(buzz-spawned only), emitting a checked monotonicturn_cost_usddelta alongside the raw cumulative — with baseline poisoning on counter decrease, and attached/pre-existing sessions correctly left unseeded (attached_claude_session_does_not_invent_first_cost_delta). initial_messageusage loss: the setup prompt is drained and published under{turn_id}:initialbefore the user turn on both success and cancel-drain paths; the two-prompt regression proves both monotonic sequences survive.- Cost-only records now survive missing/malformed prompt usage; overflow can no longer emit an all-null record; and the named fake-adapter test crosses spawn detection, NDJSON dispatch,
session/update, and drain.
Re-ran cargo test -p buzz-acp at this exact head: 744 passed, 0 failed. All checks green. Approving.
…n-surface * origin/main: chore(release): release Buzz Desktop version 0.5.11 (#5714) feat(acp): report standard adapter usage (#4950) fix(mobile): settle hydrated threads on latest reply (#4702) perf(desktop): persist channel snapshot hash (#5684) fix(agent): raise output limit and allow 3 recoveries (#5475) fix(desktop): defer foreground resume work (#5696) Signed-off-by: Hayt <41ea58f1e64c243627e8acde7c89be667052ee6e17d8f021c1195be4324ebf04@buzz.block.builderlab.xyz>
…-projection * origin/main: Harden shared agent instruction review (#4220) chore(release): release Buzz Desktop version 0.5.11 (#5714) feat(acp): report standard adapter usage (#4950) fix(mobile): settle hydrated threads on latest reply (#4702) perf(desktop): persist channel snapshot hash (#5684) fix(agent): raise output limit and allow 3 recoveries (#5475) fix(desktop): defer foreground resume work (#5696) Co-authored-by: Will Pfleger <pfleger.will@gmail.com> Signed-off-by: Will Pfleger <pfleger.will@gmail.com> # Conflicts: # desktop/src-tauri/src/commands/personas/update.rs # desktop/src-tauri/src/managed_agents/mod.rs
…graphy-staging * origin/main: Harden shared agent instruction review (#4220) chore(release): release Buzz Desktop version 0.5.11 (#5714) feat(acp): report standard adapter usage (#4950) fix(mobile): settle hydrated threads on latest reply (#4702) perf(desktop): persist channel snapshot hash (#5684) fix(agent): raise output limit and allow 3 recoveries (#5475) fix(desktop): defer foreground resume work (#5696) perf(desktop): coalesce thread-activity localStorage writes (#5693) Batch observer-store publications per relay envelope (#5680) feat(buzz-acp): idle re-sleep for woken lazy pools (#5682) fix(desktop): preserve agent mention separator after send (#5623) fix(link-previews): proxy sent preview media (#5627) feat(deletion): add durable whole-community deletion (#4425) fix(desktop): preserve live channel timelines (#5662) Refine channel settings and profile panels (#5574) fix(deps): bump webbrowser to 1.2.4 for RUSTSEC-2026-0257 (#5659) feat(acp): deliver channel description in prompt [Context] (#4552) fix(desktop): launch Databricks OAuth from passive model discovery (#5607) Signed-off-by: morgmart <98432065+morgmart@users.noreply.github.com>
…ent-config * origin/main: (31 commits) Harden shared agent instruction review (#4220) chore(release): release Buzz Desktop version 0.5.11 (#5714) feat(acp): report standard adapter usage (#4950) fix(mobile): settle hydrated threads on latest reply (#4702) perf(desktop): persist channel snapshot hash (#5684) fix(agent): raise output limit and allow 3 recoveries (#5475) fix(desktop): defer foreground resume work (#5696) perf(desktop): coalesce thread-activity localStorage writes (#5693) Batch observer-store publications per relay envelope (#5680) feat(buzz-acp): idle re-sleep for woken lazy pools (#5682) fix(desktop): preserve agent mention separator after send (#5623) fix(link-previews): proxy sent preview media (#5627) feat(deletion): add durable whole-community deletion (#4425) fix(desktop): preserve live channel timelines (#5662) Refine channel settings and profile panels (#5574) fix(deps): bump webbrowser to 1.2.4 for RUSTSEC-2026-0257 (#5659) feat(acp): deliver channel description in prompt [Context] (#4552) fix(desktop): launch Databricks OAuth from passive model discovery (#5607) chore(release): release Buzz Desktop version 0.5.10 (#5613) fix(desktop): remove 0.5.9+ perf regressions, speed up get_channels (#5599) ... Co-authored-by: Eva <011987e296fd5006292d2f930b574be47c7801048d1983c46c425d3c95f0cffd@buzz.block.builderlab.xyz> Signed-off-by: Eva <011987e296fd5006292d2f930b574be47c7801048d1983c46c425d3c95f0cffd@buzz.block.builderlab.xyz> # Conflicts: # desktop/src-tauri/src/commands/personas/inbound.rs # desktop/src-tauri/src/commands/personas/inbound/inbound_tests.rs
…ties-manifest * origin/main: Harden shared agent instruction review (#4220) chore(release): release Buzz Desktop version 0.5.11 (#5714) feat(acp): report standard adapter usage (#4950) fix(mobile): settle hydrated threads on latest reply (#4702) perf(desktop): persist channel snapshot hash (#5684) fix(agent): raise output limit and allow 3 recoveries (#5475) fix(desktop): defer foreground resume work (#5696) perf(desktop): coalesce thread-activity localStorage writes (#5693) Batch observer-store publications per relay envelope (#5680) feat(buzz-acp): idle re-sleep for woken lazy pools (#5682) Signed-off-by: Duncan <dcfd242e557282d7a1e2cf2e6877522682f1e5c6156dc92ca7d90eaedd3b0f95@buzz.block.builderlab.xyz> Co-authored-by: Will Pfleger <pfleger.will@gmail.com> Signed-off-by: Will Pfleger <pfleger.will@gmail.com>
* origin/main: Harden shared agent instruction review (#4220) chore(release): release Buzz Desktop version 0.5.11 (#5714) feat(acp): report standard adapter usage (#4950) fix(mobile): settle hydrated threads on latest reply (#4702) perf(desktop): persist channel snapshot hash (#5684) fix(agent): raise output limit and allow 3 recoveries (#5475) fix(desktop): defer foreground resume work (#5696) perf(desktop): coalesce thread-activity localStorage writes (#5693) Batch observer-store publications per relay envelope (#5680) feat(buzz-acp): idle re-sleep for woken lazy pools (#5682) fix(desktop): preserve agent mention separator after send (#5623) fix(link-previews): proxy sent preview media (#5627) feat(deletion): add durable whole-community deletion (#4425) fix(desktop): preserve live channel timelines (#5662) Signed-off-by: Duncan <dcfd242e557282d7a1e2cf2e6877522682f1e5c6156dc92ca7d90eaedd3b0f95@buzz.block.builderlab.xyz> Co-authored-by: Will Pfleger <pfleger.will@gmail.com> Signed-off-by: Will Pfleger <pfleger.will@gmail.com>
…read-context * origin/main: (38 commits) Add mobile community invites (#5641) Harden shared agent instruction review (#4220) chore(release): release Buzz Desktop version 0.5.11 (#5714) feat(acp): report standard adapter usage (#4950) fix(mobile): settle hydrated threads on latest reply (#4702) perf(desktop): persist channel snapshot hash (#5684) fix(agent): raise output limit and allow 3 recoveries (#5475) fix(desktop): defer foreground resume work (#5696) perf(desktop): coalesce thread-activity localStorage writes (#5693) Batch observer-store publications per relay envelope (#5680) feat(buzz-acp): idle re-sleep for woken lazy pools (#5682) fix(desktop): preserve agent mention separator after send (#5623) fix(link-previews): proxy sent preview media (#5627) feat(deletion): add durable whole-community deletion (#4425) fix(desktop): preserve live channel timelines (#5662) Refine channel settings and profile panels (#5574) fix(deps): bump webbrowser to 1.2.4 for RUSTSEC-2026-0257 (#5659) feat(acp): deliver channel description in prompt [Context] (#4552) fix(desktop): launch Databricks OAuth from passive model discovery (#5607) chore(release): release Buzz Desktop version 0.5.10 (#5613) ... Signed-off-by: Lazy Joe <dbd8c9941ba6dafebcef0abc015b65e75d52e7452f2ce483c9c3fd4d180f2504@buzz.block.builderlab.xyz>
…icit-settling * origin/main: fix(desktop): more compact "compact" link previews (#5629) Fix mobile composer input regressions (#5594) Add mobile community invites (#5641) Harden shared agent instruction review (#4220) chore(release): release Buzz Desktop version 0.5.11 (#5714) feat(acp): report standard adapter usage (#4950) fix(mobile): settle hydrated threads on latest reply (#4702) perf(desktop): persist channel snapshot hash (#5684) fix(agent): raise output limit and allow 3 recoveries (#5475) fix(desktop): defer foreground resume work (#5696) Co-authored-by: Taylor Ho <taylorkmho@gmail.com> Signed-off-by: Taylor Ho <taylorkmho@gmail.com>
Co-authored-by: Taylor Ho <taylorkmho@gmail.com> Signed-off-by: Taylor Ho <taylorkmho@gmail.com> * origin/main: Polish glass Huddle tray behavior (#5590) test: add deterministic desktop release smoke (#5699) fix(channels): return complete member rosters (#5765) feat(desktop): add Inbox message delete action (#5779) fix(desktop): enforce agent mention authorization at send boundaries (#5681) fix(desktop): route compact preview geometry fixture through media proxy (#5799) Make workflow run history authoritative in Desktop (#5780) fix(desktop): more compact "compact" link previews (#5629) Fix mobile composer input regressions (#5594) Add mobile community invites (#5641) Harden shared agent instruction review (#4220) chore(release): release Buzz Desktop version 0.5.11 (#5714) feat(acp): report standard adapter usage (#4950) fix(mobile): settle hydrated threads on latest reply (#4702) perf(desktop): persist channel snapshot hash (#5684) Signed-off-by: Taylor Ho <taylorkmho@gmail.com>
## Why Claude Code and Codex expose standard ACP prompt-response usage, but Buzz only consumed Goose’s private cumulative usage notification. Their token use and Claude’s cumulative cost were therefore absent from NIP-AM metrics. ## What - Read per-turn `session/prompt` response usage for known Claude and Codex adapters - Publish Claude’s raw cumulative cost separately from per-turn tokens without changing the NIP-AM schema - Keep Goose usage exclusive and cover Claude/Codex wire serialization ## Risk Assessment Low-to-medium: changes best-effort observability only and does not affect prompt execution. The adapter-specific mappings preserve source semantics and omit unavailable fields. ## References - Validated with `cargo fmt --check`, `cargo test -p buzz-acp --no-run`, and full `cargo test -p buzz-acp` (678 passed at `652e373a` before merge-trailer amendment). Generated with Codex --------- Signed-off-by: Atish Patel <atish@squareup.com> Signed-off-by: Brainy Bumble <0ed7657b57c0e8a9f5288390dd6c8d5d0a3a06abe9b01b9006814f52077d6cdf@buzz.block.builderlab.xyz> Co-authored-by: WorkerBeeGPT <c5c455215c2506cb8ba776518cec804af62d3a0526e32d496a22072e395042b9@buzz.block.builderlab.xyz> Co-authored-by: Brainy Bumble <0ed7657b57c0e8a9f5288390dd6c8d5d0a3a06abe9b01b9006814f52077d6cdf@buzz.block.builderlab.xyz> Signed-off-by: bhargavms <bhargav.m@ewa-services.com>
Why
Claude Code and Codex expose standard ACP prompt-response usage, but Buzz only consumed Goose’s private cumulative usage notification. Their token use and Claude’s cumulative cost were therefore absent from NIP-AM metrics.
What
session/promptresponse usage for known Claude and Codex adaptersRisk Assessment
Low-to-medium: changes best-effort observability only and does not affect prompt execution. The adapter-specific mappings preserve source semantics and omit unavailable fields.
References
cargo fmt --check,cargo test -p buzz-acp --no-run, and fullcargo test -p buzz-acp(678 passed at652e373abefore merge-trailer amendment).Generated with Codex