docs(#6735): define cost data contract for runtime-reported inference cost - #6736
Conversation
… cost Add a "Cost data contract" section to the tracing reference that documents how fullsend sources, aggregates, rounds, and displays runtime-reported inference cost. The new section covers: - Per-runtime cost extraction (Claude Code reads result.total_cost_usd, pi sums usage.cost.total, OpenCode sums step_finish.part.cost) - Cross-iteration aggregation (raw sum across completed iterations) - Rounding and precision by surface (full float in metrics.json, rounded to cents on OTel spans, four decimals on console, two decimals in status comments) - No pricing-table fallback when a runtime reports zero - Distinction from backend-derived estimates (e.g. MLflow) Expand the total_cost_usd description in the CLI reference (docs/cli/run.md) to note it is the runtime-reported raw aggregate with no fallback, and cross-reference the new section. Add a cross-reference from the Getting Started runtime-selection page so operators can discover the cost contract from the "Where to see what ran" list. Closes #6735
|
🤖 Finished Review · ✅ Success · Started 4:07 PM UTC · Completed 4:24 PM UTC Commit: Runtime: claude · Model: opus → claude-opus-4-6 · Effort: high · Cost: $5.32 |
Site previewPreview: https://d51403d5-site.fullsend-ai.workers.dev Commit: |
Codecov Report✅ All modified and coverable lines are covered by tests. 📢 Thoughts on this report? Let us know! |
|
Risk Assessment: low (1/5) DetailsDocumentation-only PR by bot addressing a well-scoped, triaged issue (#6735) with three files and 84 lines of additive changes defining the cost data contract — no protected paths, no security-sensitive files, no dependency changes, minimal churn, and all acceptance criteria met. |
|
Looks good to me |
![fullsend-ai-review[bot]](https://avatars.githubusercontent.com/in/3478501?s=60&v=4){kind=small}
fullsend-ai-review
Bot
added
the
ready-for-merge
|
🤖 Finished Retro · ✅ Success · Started 5:49 PM UTC · Completed 6:06 PM UTC Commit: Runtime: claude · Model: opus → claude-opus-4-6 · Effort: high · Cost: $4.26 |
Retro: PR #6736 — docs(#6735): define cost data contract for runtime-reported inference costWorkflow quality: Clean execution, zero reworkThis was a well-executed end-to-end automated workflow with no rework cycles. Timeline:
Total agent cost: $9.69 | Total automated pipeline time: ~35 minutes | Time to merge: ~26 hours Key observationsCode agent executed well on a detailed spec. The issue body contained the complete technical specification — runtime extraction behavior, aggregation formulas, rounding rules, precision by surface. The code agent translated this into well-structured documentation in one shot. This demonstrates the value of detailed issue specifications for zero-rework outcomes. Review agent produced no findings, consistent with human review. Both the agent (opus, high effort) and the human reviewer approved without requesting changes or posting inline comments. The review agent's risk assessment (low, 1/5) was appropriate for a docs-only PR. Code agent could not verify docs build locally. The PR description notes: "npm run docs:build could not run due to sandbox network restrictions on git submodules." The CI Build Site check passed, so no rework was needed, but local verification was skipped. Evidence for existing issues
No new proposalsAll identified improvement areas are covered by existing open issues. The workflow executed cleanly with zero rework, appropriate agent decisions, and correct final outcome. |
2 participants
Summary
Adds a "Cost data contract" section to the tracing reference documenting how fullsend sources, aggregates, rounds, and displays runtime-reported inference cost. Expands the
total_cost_usdfield description in the CLI reference and adds cross-references from the Getting Started runtime-selection guide.Related Issue
Addresses the documentation gap described in #6735 — operators had no public definition of how inference cost flows from each runtime through aggregation to the various output surfaces.
Changes
docs/guides/infrastructure/distributed-tracing.md): new "Cost data contract" section covering runtime cost extraction (per-runtime source fields), cross-iteration aggregation formula, rounding and precision by surface (metrics.json, OTel spans, console, status comments), no-fallback behavior, and distinction from backend-derived estimates (e.g. MLflow). Updatedfullsend.cost_usdattribute description to cross-reference the new section. Added CLI reference link to "See also."docs/cli/run.md): expandedtotal_cost_usddescription to note it is the runtime-reported raw aggregate with no pricing-table fallback, with a link to the cost data contract.docs/guides/getting-started/choosing-a-runtime.md): added a cross-reference from the status comment bullet in "Where to see what ran" to the cost data contract.Testing
scan-secretspassed on all changed files and staged contentlint-docs-linkspassed (no links escape the repo root)lycheeoffline link check passed (32 total, 0 errors)gitlintpassed on commit messageNote:
npm run docs:buildcould not run due to sandbox network restrictions on git submodules. Manual verification of the VitePress build is recommended.Checklist
!for breaking changes)Closes #6735
Post-script verification
agent/6735-cost-data-contract-docs)6bdc0fd748ee717986e37432a14018b6d661a8ec..HEAD)