Skip to content

feat(finops): add bounded usage evidence authority - #315

Draft
seonghobae wants to merge 3 commits into
mainfrom
feat/usage-evidence-authority-b84f0c9
Draft

feat(finops): add bounded usage evidence authority#315
seonghobae wants to merge 3 commits into
mainfrom
feat/usage-evidence-authority-b84f0c9

Conversation

@seonghobae

@seonghobae seonghobae commented Aug 27, 2026

Copy link
Copy Markdown
Contributor

Bounded acquisition slice for #312

This Draft started from exact protected main@b84f0c94154043a3473939c01bb6471de5a129ae on explicit non-default branch feat/usage-evidence-authority-b84f0c9. Protected main was not mutated. Current exact contributor head remains c990318345013ef24a619eff07f9f3bbb5698a2a (tree 4ffc56501d7bb6b4f010c752aaa93d7f70e84bf4); the PR changes only pg_llm_batch/usage_evidence.py and tests/test_usage_evidence.py. Fresh formal-review and inline-thread inventories remain empty.

Contract

The bounded first slice defines a closed usage-authority vocabulary (LOCAL_MEASURED, PROVIDER_REPORTED, HOST_RATE_ESTIMATE, RECONCILED), trusted caller-supplied tenant scope, bounded opaque identifiers and exact non-negative signed-bigint counts, deterministic canonical JSON plus SHA-256 evidence identity, explicit null optional dimensions, and provenance distinction across authority/source identity. It fails closed on open authority strings, booleans-as-integers, non-integer/negative/oversized counts, behavior-bearing objects, and unbounded/content-bearing identifier forms.

No pricing arithmetic, provider call, persistence, filesystem/environment read, logging, telemetry, schema, workflow, package metadata, canonical documentation, migration, or existing runtime module is changed.

Test-first / exact-head product evidence

  1. Hosted RED — test-only head 26b885bbec3aa64579f75ec2a3742540863f59c4 changed only tests/test_usage_evidence.py; production module was absent. CI 33066304934 failed because pg_llm_batch.usage_evidence did not exist.
  2. Narrow implementationfc80cd5f1095cb77788fc4da6c6fa9cc3339557c added only the production module; tests passed until the repository's 100% public-docstring contract exposed four undocumented private validators.
  3. Current repair headc990318345013ef24a619eff07f9f3bbb5698a2a adds only those helper docstrings. Exact-head CI 33066647287 proves lint success, 100% public docstrings, 1354 passed, 5 deselected, Python 3.12/3.14 unit success, container/PostgreSQL smoke success, and exact 100% statement/branch coverage for usage_evidence.py itself (41 statements / 14 branches, zero misses/partials).

Repository-wide coverage remains non-passing at 99.96% solely because protected main still carries the two weakref-registry arcs owned by dependency root #233. #315 must not duplicate #233's tests merely to manufacture aggregate 100%.

Release Acceptance 33066647102 is genuine exact-source evidence for current head c990318...: exact SHA checkout/verification, Python 3.14.7, uv 0.12.3, two clean builds, and reproducible wheel/sdist identity.

Security / SAST exact-source boundary

Historical #315 Security Scan 33066647057 and SAST Semgrep 33066647163 concluded green but their actual jobs checked out synthetic refs/pull/315/merge@61d6c1c0c68f784b0990508230be2c3468ecc304, not contributor head c990318.... They remain negative-control evidence only. Final acceptance must reacquire Security/SAST whose actual scanner workspace binds the final contributor head after legitimate dependency/base reconciliation.

Strix source-bearing RCA — attempts 3 and 4 are historical

Attempt 3 of workflow run 33066645588 / job 99167889678 bound the exact pg base/head, provisioned contextual-orchestrator, passed loopback gateway/provider-route/chat preflights, installed Strix 1.5.3 and materialized one source file, then failed before authoritative vulnerability analysis because the then-protected wrapper rejected the trusted process-local route:

ERROR: LLM_API_BASE must be an https URL when configured.

That transport-classification defect was repaired by protected central .github#1413 merge 9550c18b52468f4e886fdf6e4e1370ae23313229. Attempt-3 artifact strix-reports, ID 9722240104, SHA-256 04f14c12a6ea12d7a1dcab6f4d160202cc0e460995e0d928f60ff995fb32498d, remains RCA evidence only.

Attempt 4 of the same workflow run, exact job 99217157204, is terminal failure. Exact target materialization, trusted-source checkout, required-workflow self-test, credential gating, contextual-orchestrator provisioning, health/provider/chat preflight, Strix installation and one-file source scope all succeeded. The actual scan then failed before an authoritative vulnerability verdict because OpenAI Agents sent stream_options.include_usage=true to /v1/chat/completions and the vendored gateway returned deterministic 400 invalid_stream_options on all three bounded attempts. The required gate correctly failed closed as STRIX_PROVIDER_UNAVAILABLE; zero displayed findings and zero tokens are not vulnerability-clearance evidence. Attempt-4 artifact strix-reports, ID 9727781092, SHA-256 2ccabb851bffdb857a9f85af88f19793f9cdfd115169b46af185c687abf67c54, is RCA evidence only.

The decisive source-identity fact is that attempt 4 remained bound to trusted workflow source .github@9550c18b52468f4e886fdf6e4e1370ae23313229, whose sidecar vendored contextual-orchestrator b21645116b352967e50fc497b87eb745b9cc8c61. Re-running 33066645588 again would repeat that stale protocol boundary and is intentionally forbidden as a no-op.

Protected central authority has since moved. The protected transition through .github#1422 merge c48859ac3919f1e7d2f24e744e5c551b94e66ac2 advanced the vendored contextual-orchestrator lineage to one containing merged contextual-orchestrator #914 (e7618a3f144fcf63b4b9d84e0335ac85f6d24278), which implements the /v1/chat/completions compatibility contract accepting stream_options.include_usage=true while retaining unsupported-option fail-closed behavior. That historical transition does not by itself prove current end-to-end sidecar/provider availability. Exact current trusted .github source, vendored orchestrator pin, live dependency tip, provider discovery state, and review/scan runtime health are mutable control-plane identities and belong only in #244 plus fresh read-only central reads; do not freeze them into this Draft's product contract.

Accordingly, attempt 4 is superseded control-plane RCA, not a current pg product finding. A meaningful Strix re-test must be a new workflow invocation under then-current protected central source, not another rerun of the old run. Do not manufacture that event through source/head churn, Draft toggling, close/reopen, or duplicate control-plane activation.

Dependency root #233

#233 remains unchanged at exact 5951b7a4d779903b8924abaef2a387cae50b7f54 against protected main@b84f0c94154043a3473939c01bb6471de5a129ae. It owns the two aggregate-coverage arcs and has fresh exact-contributor-head CI, Release Acceptance, Trivy, Semgrep, and bounded Strix evidence.

It remains non-merge-ready because its active same-head OpenCode formal state is the legacy CHANGES_REQUESTED, that disposition has not been superseded by a genuinely fresh authenticated semantic review, and no qualifying independent approval exists. Fresh exact-head coverage-evidence success and a green verifier that confirms the current formal verdict exists do not retroactively turn that verdict into approval. Mutable reviewer/scheduler/control-plane truth belongs in pg #244.

Keep #315 Draft and source/head-stable while #233 is unintegrated. After #233 integrates or is genuinely superseded, reconcile #315 non-destructively to then-current protected main. That legitimate synchronization is the correct event boundary for a new Strix execution under current central authority and for reacquiring all invalidated exact-head/current-base evidence.

Writer boundary

Fresh refetch immediately before this metadata repair confirmed branch feat/usage-evidence-authority-b84f0c9 still resolves to exact c990318..., protected main remains exact b84f0c9..., formal reviews and inline threads are empty, and #316 still classifies this branch as the active writer for its two source/test paths. This metadata-only update changes no source, docs, branch ref, workflow, or protected-main tree.

Before any future source mutation, repeat the complete open-PR/no-PR path-overlap inventory and freeze if another writer moves onto the same surface.

Draft exit / acquisition acceptance

Do not mark Ready or merge until one unchanged final head freshly proves all of the following:

  1. test(recovery): cover stale evidence registry cleanup #233's aggregate-coverage boundary is protected authority or genuinely superseded without duplicated tests;
  2. repository-wide exact 100% owned production statement/branch coverage, 100% public docstrings, supported Python including 3.14, and focused usage-evidence tests pass;
  3. required Security and SAST actual scanner workspaces bind that final contributor head, not a synthetic merge;
  4. Strix is a genuine terminal-success authoritative result or an explicitly policy-valid bounded not-applicable outcome for the final diff, produced under then-current trusted central source;
  5. exact package/release/SBOM/provenance evidence is reacquired for the same final head;
  6. zero valid unresolved findings/threads remain; and
  7. then-live governance has its qualifying independent non-author approval(s), without self-approval or bypass.

The usage-evidence primitive is not billing settlement, provider truth, certification, or reconciliation-completeness authority. Mutable central state must be freshly read from #244 and the live read-only central repository. ContextualWisdomLab/.github#1222 is a closed/completed historical exact-source acceptance tracker, not an active prerequisite.

Refs #312, #244, #233, #311, #316; protected central transitions .github#1413, .github#1422; contextual-orchestrator #914.

@coderabbitai

coderabbitai Bot commented Aug 27, 2026

Copy link
Copy Markdown

Important

Draft PR not reviewed

Draft PRs are not automatically reviewed by default.

  • Trigger a manual review

To automatically review draft PRs, update your CodeRabbit configuration:

reviews:
  auto_review:
    drafts: true

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

Copy link
Copy Markdown
Contributor Author

Fresh gate refresh (2026-08-28): keep this PR Draft and unchanged at exact contributor head c990318345013ef24a619eff07f9f3bbb5698a2a against protected main@b84f0c94154043a3473939c01bb6471de5a129ae.

Two control-plane facts have materially changed since the body snapshot:

  1. Read-only central .github#941 is now merged into protected central history, so exact-submitted-head SAST checkout/identity semantics are implemented centrally. That does not retroactively repair this PR's existing SAST run 33066647163: its run belongs to the older workflow execution and remains non-transferable. A new unchanged-head downstream run under protected central authority must prove the actual scanner checkout equals c990318... before SAST can count as exact-head evidence here.
  2. Exact-submitted-head Security/Trivy repair remains a read-only central dependency: .github#897 is still an open candidate and .github#1222 remains open pending protected Security integration plus positive downstream runtime/SARIF/log/artifact identity proof.

Fresh pg-side state is otherwise unchanged: #233 remains the earliest repo-wide coverage dependency; this PR has no formal reviews or review threads; current CI is still non-passing at the repository-wide 100% gate while Release Acceptance is success. Do not churn source, duplicate #233, mark Ready, or rerun old evidence as if it used newer central workflow code. Re-evaluate only after material protected-central or #233 change.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

area: api API, protocol, event, or external contract priority: medium Normal-priority or P2 work status: blocked Blocked by conflict, dependency, or required prerequisite type: feature New or expanded product capability

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant