Skip to content

feat(blackbox): record per-call subscription attribution - #828

Closed
Kyzcreig wants to merge 7 commits into
mainfrom
daedalus-opus/t_00bb0df4-accumulator
Closed

Kyzcreig wants to merge 7 commits into
mainfrom
daedalus-opus/t_00bb0df4-accumulator

Conversation

@Kyzcreig

@Kyzcreig Kyzcreig commented Sep 21, 2026 •

Copy link
Copy Markdown
Collaborator

Summary

Stacked on #799. Records every Hermes-managed upstream completion attempt in the C1 turn_api_calls ledger, pairing raw relay identity headers with the exact call usage object.

 completion attempt
+  snapshot x-pool-served-by / x-pool-route-id
   parse response usage
+  allocate turn-local seq under lock
+  append turn_api_calls row
 retry / fallback classification
  • Success rows carry normalized usage, provider/model, HTTP 200, sub identity, route ID, and wire/pinned provenance.
  • Failed attempts append zero-token rows before Hermes retry/failover classification.
  • Pooled Anthropic and chat-completions paths preserve raw response headers without echoing them into subsequent requests.
  • Partial-stream stubs retain the failure-recorded marker so one physical attempt cannot be counted twice.
  • Missing/invalid attribution remains fail-open for inference and increments an in-process anomaly counter.

Acceptance coverage

  • AC3: mid-turn two-sub failover; 429→200; pinned providers; headerless 200; delegated independent turn; concurrent parent/subagent.
  • AC4: production build_api_kwargs echo probe with mutation control.
  • I4: per-call token sum equals parent turn totals in SQLite integration.
  • Mutation receipts: deleting direct recorder wiring, streaming recorder wiring, partial-stub marker, response carrier, or accumulator emission makes the targeted tests red.

Verification

  • scripts/run_tests.sh tests/agent/test_api_call_attribution.py tests/agent/test_stream_surrogate_splicer.py tests/plugins/blackbox -q → 187 passed, 0 failed.
  • scripts/run_tests.sh tests/agent/test_anthropic_adapter.py -q → 108 passed, 0 failed.
  • Broad suite completed 7,227 passes; five resource-saturation failures/timeouts passed immediate isolated rerun (635 passed, 0 failed).
  • git diff --check → clean.
  • Independent adversarial review pass 3: APPROVE-WITH-CHANGES; all required changes applied, including live partial-stub wiring gate.

Stack

Base branch is daedalus-opus/t_75b1f823-api-calls / #799. Retarget this PR to main after #799 lands.


View with [code]smith Autofix with [code]smith
Need help on this PR? Tag @codesmith-bot with what you need. Autofix is disabled.

… upsert

Three P1 findings from FleetReview on d7fdcac:
- turn_api_calls PRIMARY KEY(turn_id, seq) accepted NULLs (SQLite allows NULL
  in non-INTEGER PK columns, and NULLs never collide) -> NOT NULL on both
  columns plus a boundary reject in insert_api_call.
- sweep() cascaded call rows only through the selected turns, so a call whose
  parent insert_turn never landed outlived retention forever -> bounded
  parentless delete on the call's own ts, same transaction.
- insert_turn used INSERT OR REPLACE, which deletes the row first and NULLed
  served_subs_json/attribution on any re-finalize -> INSERT ... ON CONFLICT
  DO UPDATE over the TurnRecord-owned columns only.
… not OR REPLACE

The probe matched the literal 'INSERT OR REPLACE INTO turns', so the upsert
rewrite made its mutation inert (caught by the probe's own self-check).
Match the turns INSERT regardless of conflict strategy.
Parentless turn_api_calls rows are the normal state of an in-flight turn
(calls are appended as they happen; the parent turns row only lands at
finalize). Sweeping them on the retention cutoff alone deletes a live
turn's call ledger whenever retention is short or the turn outruns it.

Orphan deletion now requires the row to be past BOTH retention and
_ORPHAN_GRACE_S (24h): orphan_cutoff = now - max(grace, retention).

Verified: scripts/run_tests.sh tests/plugins/blackbox -q -> 15 files,
156 tests passed, 0 failed, EXIT=0. Mutation gate: replacing
orphan_cutoff with the retention cutoff fails exactly
test_orphan_sweep_respects_grace_period_independent_of_retention
(1 failed, 12 passed).
Capture pooled response identity beside each completion's usage, append zero-token failure attempts before retry classification, and preserve per-turn sequence ownership across delegated/concurrent turns.\n\nVerified: 187 scoped tests passed; Blackbox 156 passed; Anthropic adapter 108 passed; direct and streaming wiring/stub mutations went red.
@Kyzcreig
Kyzcreig force-pushed the daedalus-opus/t_00bb0df4-accumulator branch from 51621f9 to 8425583 Compare September 21, 2026 11:51
Base automatically changed from daedalus-opus/t_75b1f823-api-calls to main September 21, 2026 13:45
@Kyzcreig
Kyzcreig marked this pull request as draft September 23, 2026 17:58
@Kyzcreig

Copy link
Copy Markdown
Collaborator Author

apollo/merge-pass 2026-09-23: not landable as-is — rebase onto current main conflicts (conflicts in plugins/blackbox/store.py against #915 + #787; 3 slices red on the stale head.). Re-port by content is card t_aa0a07fd. Draft until then.

@Kyzcreig

Copy link
Copy Markdown
Collaborator Author

Superseded by merged #913 (52ba20d). That PR explicitly cherry-picked this PR's accumulator commit 842558300d14e8cabff8e6b0c26c02770c4b8552, then fixed the Anthropic non-streaming transport seam and per-turn sequence test, and added legacy-ledger fixture coverage. #799 had already landed the turn_api_calls schema, guarded migration, insert, cascade retention, and rollup; #913 carried the residual _snapshot_pool_headers, _record_successful_api_call / _record_failed_api_call, record_api_call, streaming/non-streaming attribution, and source tests. Main also contains #915's legacy index migration fix and #787's UNKNOWN usage plumbing, neither of which should be reverted by re-porting this older branch.

Verified on current fork main at 2605de02cf0d98a048fe2d13d65608f8db51c200: tests/agent/test_api_call_attribution.py, test_stream_surrogate_splicer.py, tests/plugins/blackbox/test_api_calls.py, and test_api_calls_compat.py → 62 passed. No residual #828 behavior to re-port. Closing this stale vehicle, not dropping the feature.

@Kyzcreig Kyzcreig closed this Sep 23, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant