Skip to content

fix(agent): halt on typed tool billing blockers - #21253

Open
dlkakbs wants to merge 1 commit into
NousResearch:mainfrom
dlkakbs:fix/typed-tool-billing-blockers-followup
Open

dlkakbs wants to merge 1 commit into
NousResearch:mainfrom
dlkakbs:fix/typed-tool-billing-blockers-followup

Conversation

@dlkakbs

@dlkakbs dlkakbs commented May 7, 2026

Copy link
Copy Markdown
Contributor

• Follow-up to #20691 and the review here:
#20691 (comment)

The previous version tried to halt on billing blockers by matching generic tool-result text like "Payment Required" / "Insufficient credits". That could false-positive on unrelated tool output.

This follow-up moves billing-blocker detection to the layers that actually see the provider response, and only halts on an explicit typed signal emitted from there.

What changed

  • Added a structured tool-result signal for terminal billing blockers.
  • Updated web_tools.py to emit that signal only when the backend exception carries an unambiguous HTTP 402.
  • Updated browser backend wrappers to do the same during session creation:
    • Firecrawl browser provider
    • Browser Use provider
    • Browserbase provider
  • Updated run_agent to halt only on the typed billing-blocker signal instead of substring-matching arbitrary tool output.

Behavioral notes

  • Plain tool output that merely contains 402 Payment Required text no longer triggers a halt.
  • Recoverable Browserbase 402s remain recoverable:
    • if dropping keepAlive / proxies lets session creation succeed, no blocker is emitted
    • only terminal 402s become typed billing blockers
  • Terminal browser-provider billing blockers are no longer hidden behind local fallback.

Targeted verification

  • typed 402 from web tools halts
  • plain error text mentioning 402 does not halt
  • terminal browser-provider 402 does not fall back to local
  • browser navigate surfaces the typed blocker result
  • recoverable Browserbase 402 still succeeds

@alt-glitch alt-glitch added type/bug Something isn't working P2 Medium — degraded but workaround exists comp/agent Core agent runtime: loop, agent_init, prompt builder, context-compression, responses endpoint tool/browser Browser automation (CDP, Playwright) tool/web Web search and extraction labels May 7, 2026
@teknium1

Copy link
Copy Markdown
Collaborator

Thanks for narrowing the proposal to typed provider signals. The underlying issue still exists on current main: tools/browser_tool.py:2050-2079 catches provider failures and falls back to local Chromium, while default guardrails do not hard-stop (agent/tool_guardrails.py:73, :241-244).

Problems

  • The changed execution bodies no longer exist on current main: commit 79559214a moved them into agent/tool_executor.py; current post-result handling is at agent/tool_executor.py:852-857 and :1517-1522.
  • The changed tools/browser_providers/* modules were removed by the browser-provider plugin migration. Current implementations are plugins/browser/*/provider.py; e.g. terminal Browserbase failure remains a generic RuntimeError at plugins/browser/browserbase/provider.py:187-191.
  • Current web providers may return failure envelopes rather than raise. plugins/web/tavily/provider.py:172-176 catches HTTP errors into {success: false, error: ...}, so an outer exception handler in tools/web_tools.py would not classify that case.

Suggested changes

  • Port the signal consumer to both current executor paths and define/employ the signal through the current browser/web provider contracts. Preserve Browserbase's paid-feature retry sequence before terminal classification, and cover envelope-returning web failures.

Automated hermes-sweeper review.

@teknium1 teknium1 added sweeper:risk-compatibility Sweeper risk: may break existing users, config, migrations, defaults, or upgrades sweeper:blast-broad Sweeper blast radius: broad — a core path most sessions hit area/usage-cost Token accounting, usage reporting, billing, cost tracking labels Jul 13, 2026

@GottZ GottZ left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This was generated by AI during triage.

Summary

Two PRs address tools wasting iterations after terminal provider billing failures. #20691 detects billing phrases in generic results, while #21253 replaces that broad matching with explicit provider-originated HTTP 402 signals consumed by the runtime.

Related pull requests

  • #20691 [closed] related — (+144/-0) — closed as a rejected precursor, still relevant because #21253 supersedes its approach: its generic result-string matching can halt on logs, fixtures, search snippets, or quoted errors and introduces an always-on stop that bypasses the opt-in hard-stop policy, as documented in the blocking contributor review.
  • #21253 related — (+484/-5) — keep open with a salvage path, consistent with the hermes-sweeper keep_open review: the typed non-retryable HTTP 402 signal, plain-text false-positive test, and Browserbase retry-before-terminal-classification directly address #20691's defects. The patch must be rebased and ported from removed browser-provider modules and pre-79559214a runtime bodies to both current agent/tool_executor.py result paths and the plugins/browser and plugins/web provider contracts, including failure-envelope handling.

Duplicates

#21253 supersedes #20691 for the same billing-blocker issue, but they are not implementation duplicates: #20691 uses unsafe generic substring matching, whereas #21253 uses typed provider signals.

Suggested consolidation

Keep #21253 open with a salvage path rather than reviving #20691. Building on the automated keep-open verdict, preserve #21253's structured billing signal, false-positive protection, and terminal-only Browserbase classification, but require the author to rebase and move the producer and consumer changes onto the current executor and plugin contracts; #20691 should remain closed as the superseded, contributor-blocked implementation.

Cross-PR triage: Reviewed 2 pull requests and 0 issues in this complex. Each diff was read against this issue; Assessment working set: 38 kB of PR diffs, 4 kB of issue/PR text, 2 kB of discussion (1 comments), 0 verify verdicts. verdicts reflect diff content, not PR titles. Part of an automated triage batch.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

area/usage-cost Token accounting, usage reporting, billing, cost tracking comp/agent Core agent runtime: loop, agent_init, prompt builder, context-compression, responses endpoint P2 Medium — degraded but workaround exists sweeper:blast-broad Sweeper blast radius: broad — a core path most sessions hit sweeper:risk-compatibility Sweeper risk: may break existing users, config, migrations, defaults, or upgrades tool/browser Browser automation (CDP, Playwright) tool/web Web search and extraction type/bug Something isn't working

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants