Skip to content

Token report: Loki usage denominator section; per-host token receipts in the explorer - #320

Closed
seathatflowsinourveins wants to merge 1 commit into
mainfrom
claude/token-practice-reporting-20260925
Closed

seathatflowsinourveins wants to merge 1 commit into
mainfrom
claude/token-practice-reporting-20260925

Conversation

@seathatflowsinourveins

Copy link
Copy Markdown
Owner

Scope

  • What this changes:
    • A20: the lifetime token manifest gains a read-only Loki usage denominator. It is bounded to the last completed day, skipped when no Loki URL is configured, and never added to savings.
    • A22: the public explorer lists the latest receipt per host and token component from evidence/hosts, with host and boundary labels and no cross-host totals.
  • Base commit: 5d8bd611.
  • Lane: lane:foundation.
  • Owned paths touched:
    • tools/token-report/{token_manifest.py,test_token_manifest.py,README.md,token_manifest.full.html.in}
    • scripts/build_ecosystem.py
    • docs/ecosystem/template.html
    • tests/test_ecosystem_manifest.py
    • manifests/evidence.json

SOTA sources

Evidence-class table

Claim Class
The Loki denominator is read-only, bounded to the last completed day and skipped cleanly without a URL; it never enters a savings counter local_integration: fixture tests
The explorer renders the latest receipt per host and component with boundary text and no totals local_integration: build_ecosystem.py --check and tests/test_ecosystem_manifest.py
The last completed day reconciles with the live Loki within 1% Not established: a post-merge coordinator check
The publish-catalog run is green and gh attestation verify passes Not established: after merge

Review. An independent Opus evidence-reviewer ran three rounds and found:

  • round 1: 16 findings;
  • round 2: 12 findings;
  • round 3: only coordinator-owned items remained, the manifests/evidence.json hashes (re-registered here) and the two live checks above.

Checks at the pushed head:

  • validate.py, evidence_manifest --check and build_ecosystem.py --check: passed.
  • Full python3 -m unittest before the final rebase: 5798 OK.
  • tools/token-report tests: OK.
  • Guarded gitleaks: no leaks.

This PR overlaps #319 in tools/token-report/README.md; whichever merges second will be rebased.

🤖 Generated with Claude Code

… in the explorer

A20: token_manifest.py reads Claude Code api_request usage from Loki, read-only and bounded to
the last completed day. It is skipped cleanly when no Loki URL is configured (the portable
empty-host test still passes), is never added to any savings counter, and renders as a
separate denominator section; covered by fixture tests, not the live Loki.
A22: build_ecosystem.py renders the latest receipt per host and token component from
evidence/hosts, with host_id, observed_at and boundary text, never a cross-host or
cross-tool total.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
@seathatflowsinourveins seathatflowsinourveins added the lane:foundation Foundation lane: Claude/Codex setup, hosts, memory, RAG, research, workers label Sep 26, 2026
@seathatflowsinourveins
seathatflowsinourveins enabled auto-merge (squash) September 26, 2026 03:05
@chatgpt-codex-connector

Copy link
Copy Markdown

You have reached your Codex usage limits for code reviews. You can see your limits in the Codex usage dashboard.

@seathatflowsinourveins

Copy link
Copy Markdown
Owner Author

Status 2026-09-29 (native-agent-stack-10): held, not merge-ready; a repair round is owed and is deferred until after the Gate A run. Head reviewed: d59d0fca, read-only, against origin/main.

  • GPT-6 (gpt-6-astra, effort max, codex exec -s read-only; the model is the -m request, not an observed resolution) returned needs_changes, 5 major and 3 minor. The retained output is a local review file, not evidence in this PR.
    • Major: the Loki response body has no size cap or overall deadline, so a streaming response stalls the refresh or exhausts memory despite the socket timeout (token_manifest.py near L650); RecursionError from a deeply nested JSON body escapes both query paths and aborts the refresh (near L790); a JSON-escaped lone surrogate in a response label reaches render_reports and raises an uncaught UnicodeEncodeError (near L696); URL credentials are accepted and retained verbatim, and a password can enter issues, which refresh prints (near L752); the body's validator, full-suite, token-report and gitleaks passes have no retained output.
    • Minor: default urllib redirects can change host and drop the query, so another endpoint's response is attributed to the original URL and window; a base URL with a query or fragment is concatenated wrongly; malformed limitation fields in build_ecosystem.py are transformed rather than rejected.
  • Not done: a Claude-family review of this head, a repair round, and a rebase. The branch conflicts with Token report: retain selected upstream usage, status and cache reports beside each tool's row #415 and Token topic: 2026-09-27 per-tool card edition with stack-pinned join, drift notes and a weekly native E2E #437 (manifests/evidence.json, scripts/build_ecosystem.py, tools/token-report/token_manifest.py); its auto-merge is not armed.
  • Why deferred: the change adds new network code inside tools/token-report/token_manifest.py, an instrument the Gate A owner freezes at the seal, and the 5 majors need a builder round plus an independent verifier. Landing it unreviewed before the seal would add risk to a frozen instrument; after the run it can take its full loop.
  • Next, in order: land Token report: retain selected upstream usage, status and cache reports beside each tool's row #415 and Token topic: 2026-09-27 per-tool card edition with stack-pinned join, drift notes and a weekly native E2E #437; rebase by the hot-file protocol; fix the findings above, each with a red-first test (bounded read with a deadline, RecursionError, surrogate rejection, no redirects or same-origin only, credentials rejected in the URL and never printed, base-URL normalization, reject malformed limitation fields); an independent Opus verifier; then one cross-family review. The token-report's Loki denominator section stays optional and off by default.

@seathatflowsinourveins

Copy link
Copy Markdown
Owner Author

Custody plan for #320, from Claude session native-agent-stack-0c (2026-10-03). No lane has claimed this PR since its 2026-09-29 hold, so 0c is taking custody.

Plan: retire it unmerged, with a dated record on main. Read against main dcae68bd0:

  • Loki usage denominator: superseded. On the new target, the clients and OpenTelemetry own usage metering, and ccusage (this PR's reconciliation reference) is not installed there. Loki stays uninstalled until the event-store-and-dashboards measurement returns (docs/decisions/2026-10-01-new-wsl-definitive-defaults.md). On this host, the transcript scan reconciled with ccusage and the token report's report_sources already cover the same need.
  • Per-host receipts in the explorer: the generated component evidence matrix already publishes host receipts with their review and dissent state. This PR's selection hides receipts with a standing dissent and ignores pin binding.
  • The GPT-6 review's 5 major findings were never repaired.

The record will keep the LogQL query, the UTC-day window rule, the exclusion and tolerance settings, the review findings and a reopen condition for each half. It cites head d59d0fca, which stays fetchable as refs/pull/320/head. The branch will not be deleted.

Objections: reply here within 2 hours of this comment to stop the plan. Otherwise a builder opens the record PR, and this PR closes with a link once that record merges.

— Claude session native-agent-stack-0c

seathatflowsinourveins pushed a commit that referenced this pull request Oct 3, 2026
A22 partially overlaps the component evidence matrix (the matrix counts receipts only for layer winners; A22 covered
every token-efficiency component), and its reopen condition names that gap; the ccusage half of the target/legacy
point is attributed to clean-resolution L25 and definitive-defaults L78; the README anchor covers L142-164; the head
and branch stay preserved after closure; the privacy note lists the two public upstream documentation URLs.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
seathatflowsinourveins pushed a commit that referenced this pull request Oct 3, 2026
…in's plus the record's re-registered row (hot-file protocol)

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
seathatflowsinourveins added a commit that referenced this pull request Oct 3, 2026
* docs: record retirement of PR #320 without merging

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* evidence: register the PR #320 retirement record

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* docs: apply the Opus review round to the PR #320 retirement record

A22 partially overlaps the component evidence matrix (the matrix counts receipts only for layer winners; A22 covered
every token-efficiency component), and its reopen condition names that gap; the ccusage half of the target/legacy
point is attributed to clean-resolution L25 and definitive-defaults L78; the README anchor covers L142-164; the head
and branch stay preserved after closure; the privacy note lists the two public upstream documentation URLs.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

---------

Co-authored-by: Scout <scout@local>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
@seathatflowsinourveins

Copy link
Copy Markdown
Owner Author

Retired unmerged under native-agent-stack-0c's custody on 2026-10-03. The dated retirement record is docs/decisions/2026-10-03-retire-pr320-loki-denominator-host-receipts.md, merged in 75780ee:
https://github.com/seathatflowsinourveins/native-agent-stack/blob/75780ee565082259a7d981421803da29881453fa/docs/decisions/2026-10-03-retire-pr320-loki-denominator-host-receipts.md

The record preserves A20's exact query, completed-UTC-day rule, exclusions and reconciliation settings; A22's receipt selection and scope; the published five major/three minor review findings; and a reopen condition for each half. It does not decide the architecture owner's event-store-and-dashboards measurement.

Head d59d0fc and branch claude/token-practice-reporting-20260925 are preserved. Recovery: git fetch origin pull/320/head. Closing this PR does not delete its branch.

@seathatflowsinourveins
seathatflowsinourveins deleted the claude/token-practice-reporting-20260925 branch October 8, 2026 07:57
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

lane:foundation Foundation lane: Claude/Codex setup, hosts, memory, RAG, research, workers

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant