Conversation
… model endpoint
_gateway_provider_error_reply() answered every connection-shaped provider failure with
"the configured model endpoint is not running or is unreachable". The marker tuple behind
that single row mixes three different failures, and only one of them supports the sentence:
* interrupted — an ESTABLISHED connection died mid-transfer (ReadError / ECONNRESET /
RemoteProtocolError). Something accepted and answered, so whether the endpoint is up is
unknown from this alone; an earlier call in the same turn may have succeeded against it.
* unreachable — nothing accepted the connection (ECONNREFUSED, WinError 10061, no route).
Here "not running or unreachable" IS the diagnosis, and it is the case the wording was
written for (NousResearch#86570, merged in NousResearch#86729). Unchanged.
* ambiguous — connection-shaped with the cause flattened away by the SDK
(openai.APIConnectionError: Connection error.). Neither diagnosis is supported.
Split the one connection row into three ordered rows so the first and third stop asserting
that the model server stopped. Neither new reply claims a retry count, which this layer
cannot know.
_CONNECTION_ERROR_MARKERS is deliberately untouched: its first 8 entries are sliced into
_GATEWAY_PROVIDER_ERROR_SHAPE_RE, so editing it would change WHICH texts are treated as
provider envelopes rather than what they are called. Auth/policy/rate-limit precedence,
secret redaction and raw passthrough for programmatic surfaces are unchanged.
Refs NousResearch#26339 (a real errno-104 reset against a live endpoint; its payload-side cause is not
fixed here, only the way it is described).
Co-authored-by: Lei-k <11388531+Lei-k@users.noreply.github.com>
alt-glitch
added
type/bug
Lei-k
marked this pull request as ready for review
September 14, 2026 16:37
teknium1
pushed a commit
that referenced
this pull request
Sep 20, 2026
Split the single connection row of the gateway's shaped provider-error reply into three: an interrupted established connection (reset/EOF/RemoteProtocolError), a refused/unroutable endpoint (the case #86570 wrote the "not running or is unreachable" wording for), and a cause-free SDK ``APIConnectionError`` that supports neither diagnosis. A reset says nothing about whether the endpoint is up, so telling the user to restart a server that just answered sends them to debug the wrong thing (#116323). Selectively adapted from Lei-k#16 (497cc47) via PR #109701, rebased onto the current reply contract (rate-limit > auth > policy > connection, every reply names a slash command, no operator jargon).
Collaborator
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What does this PR do?
Distinguishes three connection failure causes in the gateway's user-facing provider-error reply:
This changes wording/classification only. Authentication, policy and rate-limit precedence, secret redaction, provider-envelope detection and raw programmatic-surface passthrough remain unchanged. The original
_CONNECTION_ERROR_MARKERStuple is deliberately untouched: its first eight entries feed the provider-envelope gate, so changing it would alter which messages are filtered, not just their wording. No retry policy, model configuration or delivery lifecycle changes.Related issue and provenance
Refs #26339: a live local endpoint can reset a connection; this PR does not fix that issue's payload-side root cause.
Preserves the refused-endpoint guidance introduced by #86729 for #86570.
Selectively adapted from Lei-k#16, original commit
497cc47556b10eba94c65147cceedeba622aa087, fork merge5cd65c0c21003ee022bc5119c66f87fac78085e3. Source credit is retained in the commit history.The fork owner reports a week of successful operation. This is user-reported fork experience, not an upstream E2E validation claim.
Scope and deduplication
Based on upstream main
b6b53c69a6ed49cb099cf1bfe76b5e6edd718e5a, not fork main. Onlygateway/run.pyand its existing topical connection-reply test file change.The other fork #16 changes are intentionally omitted:
No duplicate recovery/delivery implementation is submitted here. Delegation route ownership from fork #22 is an independent logical change and is not bundled, following the contribution guide.
Verification
git diff --check: passed.ab7b0f54bbc5466a064f7f885ac3c13590a0f261: PASS, no blocking regression found; focused connection tests and real sanitizer probes reproduced independently. Existing WindowsWinError 10053/10054wordings can still fall through to the generic reply (also true on base); broader Windows marker coverage is intentionally deferred, not claimed fixed. This remains a Draft for user/maintainer review, not authorization to merge.Checklist