docs(claude): correct the synthetic-placeholder guard's stated failure mode - #1057
Merged
Merged
Conversation
…e mode The comment said an unguarded `<synthetic>` carrier makes submission fail with "pricing is unavailable for submitted token usage". #1053 replaced that abort with per-model exclusion, so the fabricated row is now dropped with a warning while the rest submits. The consequence inverts: the guard matters more, because a regression in it no longer announces itself. Confidence: high Scope-risk: narrow
Contributor
|
The latest updates on your projects. Learn more about Vercel for GitHub. 1 Skipped Deployment
|
![cubic-dev-ai[bot]](https://avatars.githubusercontent.com/in/1082092?s=60&v=4){kind=small}
…ing it The first pass replaced one inaccurate claim with three more, all found in adversarial review: - "excluded with a warning and the rest is submitted" ignored #1055: with no pricing dataset loaded at all, require_trustworthy_exclusions fails the submission rather than excluding anything. - "carrying a char-based estimate" no longer describes production. #1037 set allow_char_estimate: false for Claude transcripts, so the extractor returns None and a text-only tool result is dropped for zero usage. Only explicit tool-result token metadata can reach the model-inheritance branch. - "invisible in normal use" overstated it. The CLI prints a named exclusion warning, and a batch left with nothing prints "No usage data found to submit" and performs no HTTP submission. Also fixes the opposite stale claim left in the same file, where a test still said submission "rejects" the escaped usage. Confidence: high Scope-risk: narrow Directive: this comment describes behavior owned by lib.rs, not this file — re-check it against the submission path when either changes
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Comment-only. The
<synthetic>guard inextract_claude_tool_result_message(added in #1045) documents its failure mode as:That was accurate when #1045 was written, but #1053 landed first and replaced the whole-batch abort with per-model exclusion. An unguarded
unknown/<synthetic>row is now dropped with a warning while the rest of the batch submits.The consequence inverts, which is the part worth recording: the guard matters more now, because a regression in it no longer announces itself — the fabricated usage just disappears from the leaderboard.
Tests
No behavior change.
cargo fmt --all --checkclean,cargo build -p tokscale-coreclean.Summary by cubic
Updated the
<synthetic>guard comment inextract_claude_tool_result_message. It now notes per-model exclusion with a warning when pricing is loaded, full submission failure when none is loaded, and thatallow_char_estimate: falsedrops text-only results; also fixed a stale test comment.Written for commit 137d06a. Summary will update on new commits.