Skip to content

fix(sse): probe one other proxy after an OpenCode FreeTierError before giving up on the egress - #18423

Merged
diegosouzapw merged 45 commits into
release/v3.8.53from
fix/opencode-free-tier-403-egress-rotation
Nov 5, 2026
Merged

diegosouzapw merged 45 commits into
release/v3.8.53from
fix/opencode-free-tier-403-egress-rotation

Conversation

@diegosouzapw

Copy link
Copy Markdown
Collaborator

Summary

  • OpenCode's free tier sends back the same 403 FreeTierError body whether it refused the request (client identity or tool shape) or the egress IP the request came from (a flagged proxy). The executor used to treat every one of these as request-scoped: it did one observed-tools retry, returned the 403, and kept using the flagged egress.
  • A new opt-in flag, OPENCODE_FREE_TIER_EGRESS_PROBE (off by default), tells the two cases apart by evidence instead of text. Once the observed-tools retry fails or does not apply, the request is sent once more through a different proxy: another account's own proxy, or failing that another member of the ambient connection pool. If the probe answers 200, the first egress is set aside in the proxy refusal memory and the probe response is served. If both refuse, the original refusal is returned and nothing is recorded.
  • The probe never opens a direct connection. It only dispatches through a concrete proxy config, never picks a proxy-less sibling, and never probes a refusal that already went out direct. With the flag off, or with no account proxies and no pool, behavior is identical to today.

Attribution

Thanks to @yxxrn for the original diagnosis and the live measurement showing a flagged pool egress gets 403 while a healthy egress gets 200 for the same request. Their fix replays directly, which would bypass the required-proxy guard here, so this PR uses the existing proxy rotation and refusal memory instead.

Changes

  • open-sse/executors/opencodeFreeTierEgressProbe.ts (new, 144 lines): the probe. It only acts on a FreeTierError, skips a direct first egress, picks another account proxy (cooled-down or set-aside members are skipped) and otherwise asks the pool resolver for another member. It runs at most one probe per request through a request-context marker, because the request-shape replay can run the executor twice. A non-200 probe body is discarded, and only a 200 probe writes to the refusal memory.
  • open-sse/executors/opencodeFreeTierRetry.ts: handleLoopFreeTierRefusal gets an optional probe hook that runs only when the observed-tools retry failed or did not apply. When the hook returns null, the existing path runs unchanged.
  • open-sse/executors/opencode.ts (+53): the flag is read once per request. In the rotation loop, the free-tier branch passes the probe hook, and the refusal is recorded with the loop's own noteRefused. In the single-connection fast path (keyed connection behind a pool member), the probe runs after the retry. Its applied-egress tracker is only created when the flag is on.
  • open-sse/executors/opencodeRequestContext.ts: adds an optional freeTierEgressProbed field.
  • src/shared/constants/featureFlagDefinitions.ts, src/shared/utils/featureFlags.ts: the flag (network, boolean, default "false") and its getter, placed next to OPENCODE_POOL_RESELECT.
  • docs/reference/FEATURE_FLAGS.md: adds the flag row in the Network section and updates the counts (111 → 112, Network 29 → 30).
  • The refusal is recorded with the existing geo_blocked kind (short 60 s → 10 min curve, already used for 403-class egress refusals in this same branch of the executor), so no new kind, UI label or locale strings are needed.

Test plan

TDD: tests/unit/opencode-free-tier-egress-probe.test.ts (11 cases, mocked fetch, local TCP listeners as proxies, no real network).

  • Before the fix (base opencode.ts restored, same test file): ℹ tests 11 ℹ pass 6 ℹ fail 5. The failing cases were: flagged + healthy egress (account proxies), the set-aside egress is skipped by the next request, both egresses refuse, pool flagged + healthy, pool both refuse. The 6 that passed are the regression guards (flag off ×2, no direct probe, direct first egress not probed, non-FreeTierError 403, pool with nothing else to offer).
  • After the fix: ℹ tests 11 ℹ pass 11 ℹ fail 0.
  • What the cases cover: no request ever leaves without a proxy (a proxy-less sibling is never used, and observed never contains direct); with the flag off the refusal is returned after one dispatch and nothing is recorded (account proxies and pool); flagged + healthy egress rotates, serves 200 and sets only the flagged egress aside (kind: geo_blocked), and the next request skips it; both refusing returns the original body with nothing recorded.
  • Related suites (every test file that cites the touched modules, the flag catalog, or the opencode executor, 134 files, run with the test:unit:ci preamble): ℹ tests 1235 ℹ pass 1235 ℹ fail 0.
  • npm run typecheck:core: exit 0.
  • npx eslint --suppressions-location config/quality/eslint-suppressions.json <changed files>: clean. npx prettier --check <changed files>: clean.
  • npm run check:file-size: OK. npm run check:cycles:ratchet: 14/14. npm run check:complexity-ratchets: OK (3064/3218, 1408/1437). npm run check:dead-code: OK (371/377). npm run check:docs-all: exit 0.
  • GITHUB_BASE_REF=release/v3.8.53 npm run check:test-masking: OK, 2 modified, no weakening.

live_check

Not validated live yet (hold-vps). Steps on the VPS:

  1. Turn on OPENCODE_FREE_TIER_EGRESS_PROBE. Use an OpenCode free-tier connection behind a proxy pool with one member the upstream currently flags (it answers 403 FreeTierError through that member) and one healthy member (it answers 200 for the same request).
  2. Send an oc/* free-model request that the pool routes through the flagged member. Expected: the client gets 200. The log shows free-tier refusal 403 (proxy <flagged>), probing once (proxy <healthy>)… and then answered 200: (proxy <flagged>) set aside, and the pool visibility shows the flagged member set aside.
  3. Send the same request with a request shape that is refused on every egress (for example, the gated tool set removed). Expected: the original 403 FreeTierError, exactly one probe, request-scoped refusal, nothing recorded, and no member set aside.
  4. In both runs, confirm in the proxy logs that no upstream connection left without a proxy.

⚠️ base-red inherited: #17219 — docs-sync-strict (README stale version), unit shards, integration, package artifact / boot-smoke, build:cli timeout

…e giving up on the egress

The upstream answers the same 403 FreeTierError body for a refused request
shape and for a flagged egress IP, so the text alone cannot say whether
another egress would be served. Behind the new opt-in flag
OPENCODE_FREE_TIER_EGRESS_PROBE (default off), once the observed-tools retry
failed or did not apply, the executor replays the request once through a
different proxy: another account's own proxy, else another member of the
ambient connection pool. Only a 200 on that probe sets the first egress aside
in the proxy refusal memory and serves the probe; when both refuse, the
original refusal is returned and nothing is recorded.

The probe never uses a direct connection: it only dispatches through a
concrete proxy config, a proxy-less sibling is never a probe egress, and a
refusal that went out direct is never probed. One probe per request. With the
flag off, or without account proxies and pool, behavior is unchanged.

Inspired-by: decolua/9router#4988
@diegosouzapw diegosouzapw added the hold-vps PR verde, merge aguardando validação live (release-drain) label Oct 28, 2026
@github-actions

Copy link
Copy Markdown
Contributor

CI Coverage Report

  • Coverage job: skipped
  • PR test policy: success

Coverage artifact was not available for this run.

diegosouzapw and others added 6 commits November 2, 2026 21:47
# Conflicts:
#	docs/reference/FEATURE_FLAGS.md
#	tests/unit/feature-flags-settings.test.ts
#	tests/unit/server-owned-tool-loop-flag.test.ts
Recount the feature flags (116) after the tip's API_KEY_AUDIT_CLIENT_DETAILS_ENABLED.
… complexity under the ratchet

No behavior change: eligibility checks and the probe dispatch move into helpers.
@coderabbitai

coderabbitai Bot commented Nov 3, 2026 •

Copy link
Copy Markdown

Review in Change Stack →

Important

Review skipped

Auto incremental reviews are disabled on this repository.

Please check the settings in the CodeRabbit UI or the .coderabbit.yaml file in this repository. To trigger a single review, invoke the @coderabbitai review command.

Run configuration

You can disable this status message by setting the reviews.review_status to false in the CodeRabbit configuration file.

Use the checkbox below for a quick retry:

  • Trigger review

No actionable comments were generated in the recent review.

Summary

Summary by CodeRabbit

  • New Features
    • Added an opt-in, off-by-default retry for qualifying OpenCode free-tier refusals. When available, the request is tried once through a different proxy; a successful response is served and the original proxy is set aside.
  • Documentation
    • Updated the feature-flag catalog with the new network option and its behavior.

The change adds a default-off probe that retries qualifying proxied OpenCode free-tier refusals through another proxy. It also moves Responses API field normalizers into a separate module.

Changes: Free-tier egress probe

Layer / File(s) Summary
Flag definition and request state
src/shared/constants/featureFlagDefinitions.ts, src/shared/utils/featureFlags.ts, open-sse/executors/opencodeRequestContext.ts, src/i18n/messages/en.json, docs/reference/FEATURE_FLAGS.md, tests/unit/feature-flags-settings.test.ts, tests/unit/server-owned-tool-loop-flag.test.ts
Adds the default-off network flag and a request field that tracks whether a probe ran. Updates the flag catalog and registry assertions.
Alternate-egress probe flow
open-sse/executors/opencodeFreeTierEgressProbe.ts
Checks refusal eligibility, then selects a different account proxy or pool member. It dispatches once. A 200 response records the refused egress; other outcomes do not record it.
Retry and executor integration
open-sse/executors/opencodeFreeTierRetry.ts, open-sse/executors/opencode.ts, tests/unit/opencode-free-tier-egress-probe.test.ts, changelog.d/fixes/18423-opencode-free-tier-egress-probe.md
Calls the probe after an unsuccessful or inapplicable observed-tools retry. Wires the single-account and account-rotation paths. Adds probe tests and a changelog entry.

Responses field normalizers

Layer / File(s) Summary
Normalizer extraction
open-sse/executors/opencodeResponsesFields.ts, open-sse/executors/opencode.ts
Moves call-ID, argument, output, token, and reasoning normalizers into a separate module and imports them into the executor.

Estimated code review effort: 3 (Moderate) | ~25 minutes

Change: Feature

Possibly related PRs

Merge Risk: Minimal · previous review 8c499

The opt-in probe has no established merge-blocking issue. It is ready for normal checks; live validation remains outstanding.

Security architecture risk: Low · up to 8c499

The default-off probe is bounded to one alternate proxy and preserves the request’s upstream credentials. No proxy-boundary bypass was established, but failure handling and deployment coverage remain incomplete.

Retained concerns
No architecture-level concerns identified.

Security review details

Security Blast Radius

  • inferred — When enabled, an eligible request can cause one additional upstream dispatch through a configured alternate proxy. A resulting refusal-memory write can influence other pool or account selectors sharing that proxy entry in the same process, not only the initiating request. Keys include scheme, username, host and port rather than tenant or destination. Existing geo-blocked policy bounds avoidance from one minute up to ten minutes, and the store is nonpersistent and capped at 1,000 entries.

Trust Boundaries and Controls

  • observed — The probe rejects direct initial egress and excludes missing, repeated, cooled-down or avoided account proxies. Pool output must satisfy the concrete-proxy guard. Dispatch retains the original ExecuteInput credentials and signal, uses an explicit selected proxy and disables upstream retries. It does not request the transport’s separately opt-in direct fallback.

Resilience and Maintainability Implications

  • inferred — The probe catches every dispatch exception, including caller cancellation, and returns the saved refusal instead of preserving the thrown cancellation. This changes error semantics, but the caller’s signal still reaches transport, no refusal entry is written on this failure path, and the examined refusal handler returns without another probe. The inspected evidence does not establish a security-boundary escape from this behavior.
Pre-merge checks: Passed 5 · Failed 1 · Inconclusive 1

Failed Failed checks: 1 warning · 1 inconclusive

Check name Status Explanation Resolution
Docstring Coverage Warning Docstring coverage is 61.54% which is insufficient. The required threshold is 80.00%. Docstring coverage is scoped to functions touched by this diff with cyclomatic complexity of at least 2. Documenta… Write docstrings for the functions missing them to satisfy the coverage threshold.
No Ai Attribution Inconclusive All 8 commits in the reviewed range were inspected. They contain no AI/bot Co-Authored-By trailer or Generated with footer; the only trailer is an Inspired-by link. The available PR body excerpt conta… Provide the complete PR body, including its final footer, so the No AI attribution check can be verified.
Passed Passed checks: 5 passed

Check name Status Explanation
Title check Passed The title follows Conventional Commits with type "fix" and scope "sse". It names the main change: after an OpenCode FreeTierError, the executor probes one other proxy before giving up on the egress. T…
Description check Passed The description follows the repository template. It has a Summary, a Changes list, and a Test plan. It also covers Tests Added Or Updated and Coverage in substance, and it names the flag and its revie…
Linked Issues check Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check Passed Check skipped because no linked issues were found for this pull request.
Tests For Production Code Passed The pull request changes production code under open-sse/ and src/. It adds tests/unit/opencode-free-tier-egress-probe.test.ts with coverage for the new probe behavior and updates feature-flag te…
Full details: Title check

Explanation

The title follows Conventional Commits with type "fix" and scope "sse". It names the main change: after an OpenCode FreeTierError, the executor probes one other proxy before giving up on the egress. The raw summary and objectives confirm this matches the changeset, which adds the opt-in OPENCODE_FREE_TIER_EGRESS_PROBE flag and the probe logic.

Full details: Description check

Explanation

The description follows the repository template. It has a Summary, a Changes list, and a Test plan. It also covers Tests Added Or Updated and Coverage in substance, and it names the flag and its reviewer risks. It lists the live validation steps under live_check. Missing or weak parts: the Related Issues section has no issue reference (it is not clearly filled in), and the Target Branch checkboxes are not marked. The Validation checklist items are not ticked. The author also says live validation has not been done (hold-vps).

Full details: Tests For Production Code

Explanation

The pull request changes production code under open-sse/ and src/. It adds tests/unit/opencode-free-tier-egress-probe.test.ts with coverage for the new probe behavior and updates feature-flag tests in two existing test files. The required tests are present in the reviewed diff.

Full details: Docstring Coverage

Explanation

Docstring coverage is 61.54% which is insufficient. The required threshold is 80.00%. Docstring coverage is scoped to functions touched by this diff with cyclomatic complexity of at least 2. Documentation-only edits and deletions within surviving functions are included; unchanged diff context and fully deleted functions are excluded. Analyzed 13 functions across 10 files. (3 skipped: 3 unsupported.)

Resolution

Write docstrings for the functions missing them to satisfy the coverage threshold.

Full details: No Ai Attribution

Explanation

All 8 commits in the reviewed range were inspected. They contain no AI/bot Co-Authored-By trailer or Generated with footer; the only trailer is an Inspired-by link. The available PR body excerpt contains no prohibited attribution, but the body is explicitly truncated and no complete PR description is available in the checkout or review refs.

Resolution

Provide the complete PR body, including its final footer, so the No AI attribution check can be verified.

Finishing Touches

Generate docstrings

  • Commit to this branch
  • Create a new PR
Generate unit tests (beta)

  • Commit to this branch
  • Create a new PR
  • Autofix · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts
Recent review info

Run configuration

Commits

Reviewing files that changed from the base of the PR and between f000c11 and 8c49949.

Files ignored due to path filters: 1

  • src/i18n/messages/pt-BR.json is excluded by !src/i18n/messages/[a-df-z]*.json
Files selected for processing: 13

  • changelog.d/fixes/18423-opencode-free-tier-egress-probe.md
  • docs/reference/FEATURE_FLAGS.md
  • open-sse/executors/opencode.ts
  • open-sse/executors/opencodeFreeTierEgressProbe.ts
  • open-sse/executors/opencodeFreeTierRetry.ts
  • open-sse/executors/opencodeRequestContext.ts
  • open-sse/executors/opencodeResponsesFields.ts
  • src/i18n/messages/en.json
  • src/shared/constants/featureFlagDefinitions.ts
  • src/shared/utils/featureFlags.ts
  • tests/unit/feature-flags-settings.test.ts
  • tests/unit/opencode-free-tier-egress-probe.test.ts
  • tests/unit/server-owned-tool-loop-flag.test.ts

Limit details: You’ve used all 10 included reviews currently available.

# Conflicts:
#	tests/unit/feature-flags-settings.test.ts
#	tests/unit/server-owned-tool-loop-flag.test.ts
… and the upscale helper (boundary 774 -> 759) (#18648)
@diegosouzapw
diegosouzapw merged commit 9d9c3e8 into release/v3.8.53 Nov 5, 2026
1 of 12 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

hold-vps PR verde, merge aguardando validação live (release-drain)

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant