Repository navigation
settings: say a Claude plan's 5-hour limit drains faster for runs like autofork's - #9
Conversation
…t drains at the interactive rate
|
The latest updates on your projects. Learn more about Vercel for GitHub.
|
AVGVSTVS96
changed the title
…ains faster for runs like autofork's
| </div> | ||
| <p className="fine">A rebuild starts only if its whole cap still fits under the month's, so none is cut off halfway. Costs are at API prices.</p> | ||
| <p className="fine"> | ||
| A rebuild starts only if its whole cap still fits under the month's, so none is cut off halfway. Costs are at API prices. On a Claude plan, runs like |
There was a problem hiding this comment.
🟢 Low
SettingsCard is rendered for every fork (Yours.tsx:19) and has no access to the fork's agent. So forks running on ChatGPT/Codex (CODEX_AUTH) also get this Claude-plan paragraph. "On a Claude plan" makes it conditional, so it's only noise for those forks. Still, the agent is already known (ForkPage reads fork.agent), and it could be passed in so the sentence only shows for Claude forks.
Evidence
src/Yours.tsx:19src/Spend.tsx:85src/ForkPage.tsx:109-111
microscope · correctness
microscope reviewed
|
| Check | Result |
|---|---|
| Correctness | 3 issues (3 low) new, 2 open, 1 code objects reviewed |
| Approvability | Needs human approval |
Approvability: The code is small and safe: it only changes one fine-print paragraph in SettingsCard. The problem is what the text says. It makes a factual, user-facing claim about how a third party's (Anthropic's) rate limits behave, and the open findings dispute that claim on several points. (1) The cited source's own numbers (about 1.9x controlled, about 2.3x via proxy, about 2.0x Pro corroboration) go beyond the stated "1.6x to 1.8x", which undercuts the PR's reason for leaving out the larger figures. Users budgeting their 5-hour window could hit the limit sooner than the note suggests. (2) The weekly claim ("count them the same") cites both links, but only T3's A/B measured it, and only coarsely. claude-code#97074 says it has no controlled weekly measurement, and its one anecdote leans the other way. (3) The paragraph shows on every fork, including Codex or custom-agent forks, where it isn't true, because SettingsCard doesn't receive the fork's agent. (4) There's newer evidence that setting CLAUDE_CODE_ENTRYPOINT removes the penalty, and t3code closed its issue as fixed today. So the note may describe as unavoidable a problem autofork could fix itself. Deciding the exact numbers, how to attribute the sources, whether to show the note only for Claude forks, and whether to apply the workaround instead are all editorial and product calls, so a person should read this before it merges.
Resolved since the last review:
src/Spend.tsx:158src/Spend.tsx:158
…terminal, with both sources linked
| A rebuild starts only if its whole cap still fits under the month's, so none is cut off halfway. Costs are at API prices. On a Claude plan with a 5-hour limit, | ||
| runs like autofork's use it about 1.6x to 1.8x faster per token than interactive Claude Code, while weekly limits count them the same{" "} |
There was a problem hiding this comment.
🟢 Low
The note documents a penalty that already has a known workaround, one autofork could apply itself. Commenters on #97074 say that setting CLAUDE_CODE_ENTRYPOINT to a different value avoids the extra drain. t3code closed #24649 as completed on Nov 2 after labelling its sessions (t3-code), and the Pro retest then matched the TUI rate (about 0.92–0.96x). The autofork action spawns claude -p with process.env and doesn't set that variable. This PR records the slowdown as unavoidable, and the earlier commit that said autofork labels its runs was dropped, but new evidence since then shows the label works. Consider setting the entrypoint in the action, at least as a follow-up. If not, the note could tell users they can set the variable themselves.
Evidence
src/Spend.tsx:157-158https://github.com/pingdotgg/t3code/issues/24649https://github.com/anthropics/claude-code/issues/97074AVGVSTVS96/autofork src/autofork.ts agent(): spawnSync("sh", ["-c", command], { env, input: prompt })
microscope · correctness
…lled A/Bs measured
| <p className="fine">A rebuild starts only if its whole cap still fits under the month's, so none is cut off halfway. Costs are at API prices.</p> | ||
| <p className="fine"> | ||
| A rebuild starts only if its whole cap still fits under the month's, so none is cut off halfway. Costs are at API prices. On a Claude plan with a 5-hour limit, runs | ||
| through <code>claude -p</code> like autofork's use it about 1.6x to 1.8x faster per token than interactive Claude Code, and weekly limits count them the |
There was a problem hiding this comment.
🟢 Low
"1.6x to 1.8x" understates what the linked claude-code#97074 measured against the same interactive cli baseline. Its controlled table gives ~$3.0 vs ~$1.6 per 1% of the window, which is ≈1.9x. Its proxy re-run of the same two arms gives $1.59 vs $0.69, ≈2.3x. A controlled Pro-plan corroboration (opus-5-5) found ~2.0x (1.8–2.2x), and it collapsed to 1x when both arms were accidentally labelled sdk-cli. So the larger figures aren't only derived numbers or other baselines, which is the reason the PR gives for leaving them out. This is new evidence against that decided trade-off, not a code bug. A user budgeting their 5-hour window from this sentence may hit the limit sooner than it suggests. Consider "about 1.6x to 2x" or "at least 1.6x".
| through <code>claude -p</code> like autofork's use it about 1.6x to 1.8x faster per token than interactive Claude Code, and weekly limits count them the | |
| through <code>claude -p</code> like autofork's use it about 1.6x to 2x faster per token than interactive Claude Code, and weekly limits count them the |
Evidence
src/Spend.tsx:157-159https://github.com/anthropics/claude-code/issues/97074https://github.com/pingdotgg/t3code/issues/24649
microscope · correctness
| A rebuild starts only if its whole cap still fits under the month's, so none is cut off halfway. Costs are at API prices. On a Claude plan with a 5-hour limit, runs | ||
| through <code>claude -p</code> like autofork's use it about 1.6x to 1.8x faster per token than interactive Claude Code, and weekly limits count them the | ||
| same ( |
There was a problem hiding this comment.
🟢 Low
The sentence says "weekly limits count them the same" and then cites both links in one parenthetical. That reads as if both sources back the weekly claim, but only T3 Code's A/B measures it, and only as "+2% in every arm" on a meter that shows whole percents. anthropics/claude-code#97074 says outright that it has no controlled weekly measurement. Its one weekly data point is DimaVIII's anecdote: about $120 of claude -p took about 10% of a Max 20x weekly limit. That doesn't support "the same" and arguably hints the other way. A reader who follows the second link to check the weekly part won't find it. Fix: put the weekly claim right next to the T3 link only, or soften it, e.g. "weekly limits looked unaffected in T3 Code's A/B". Also, the two "per token" ratios are measured differently: T3's is % of the window per million cache-write tokens, and #97074's is API-equivalent dollars per 1% of the window. Both are only good to about ±20% because the meter shows whole percents. The copy flattens all of that into one "1.6x to 1.8x per token" range.
Evidence
src/Spend.tsx:157-167https://github.com/anthropics/claude-code/issues/97074 (Weekly limit: 'no controlled weekly measurement'; DimaVIII anecdote ~10% weekly for ~$120 of claude -p)https://github.com/pingdotgg/t3code/issues/24649 (weekly +2% in every arm; meter only shows whole percents, ratios ±20%)
microscope · correctness
| A rebuild starts only if its whole cap still fits under the month's, so none is cut off halfway. Costs are at API prices. On a Claude plan with a 5-hour limit, runs | ||
| through <code>claude -p</code> like autofork's use it about 1.6x to 1.8x faster per token than interactive Claude Code, and weekly limits count them the |
There was a problem hiding this comment.
🟢 Low
SettingsCard shows this fine print on every fork, whatever agent it uses. SettingsCard doesn't receive the fork's agent (Settings has no agent field, and Yours.tsx passes only owner/name). So a fork set up with Codex or a custom agent also sees "runs through claude -p like autofork's …". For those forks that's untrue: autofork's runs don't go through claude -p, and no Claude 5-hour limit is involved. The "On a Claude plan" qualifier doesn't help much, because a user can have a Claude plan and still run the fork on Codex. Fix: pass the fork's agent (the agent already in the fork state at shared/api.ts:105) into SettingsCard and render this sentence only when the agent is claude.
Evidence
src/Spend.tsx:85src/Yours.tsx:19shared/api.ts:55shared/api.ts:105convex/http.ts:52
microscope · correctness
One sentence in the spend settings' fine print, where cost tracking lives: on a Claude plan with a 5-hour limit,
claude -pruns like autofork's use it about 1.6x to 1.8x faster per token than interactive Claude Code, and weekly limits count them the same. The 1.6x to 1.7x is Theo's controlled A/B on one Max account (pingdotgg/t3code#24649:claude -p1.83 vs TUI 1.11 to 1.14 % of the window per million cache-write tokens; weekly +2% in every arm). The ~1.8x is the controlled headline of anthropics/claude-code#97074. Larger figures (up to ~3x) exist but are derived or against other baselines, so they're left out. Plan with no 5-hour limit (verified Ultra) see no difference. No banner.Note
Note on plan usage for
claude -pruns in the spend settingssrc/Spend.tsx. On Claude plans with a 5-hour limit, autofork'sclaude -pruns use that limit about 1.6x to 1.8x faster per token than interactive Claude Code. Weekly limits count both the same.claude -p(entrypointsdk-cli) uses ~1.8× more of the 5-hour window per token than interactivecli, same workload anthropics/claude-code#97074.🔬 microscope summarized b8912e8. 1 file reviewed, 4 issues evaluated, 1 filtered, 3 comments posted
Filtered issues
src/Spend.tsx:168The bug is real, but it's already been reported. Yours.tsx:19 renders SettingsCard for every fork and passes it no agent info. So the Claude-plan note shows on ChatGPT/Codex and custom-agent forks too. The first already-reported item covers the same problem (unconditional render, no agent access, noise for non-Claude forks) and the same fix (pass fork.agent and show the note only for Claude forks). The custom-agent case is a small extension of that same issue, not a separate one.