Skip to content

fix(combo): hop Astra quality-empty 200 and Gemini 3.8 empty-content to family siblings - #13603

Merged
diegosouzapw merged 7 commits into
cheaper-inference:release/v3.8.52from
HouMinXi:feat/astra-quality-family-fallback
Oct 8, 2026
Merged

diegosouzapw merged 7 commits into
cheaper-inference:release/v3.8.52from
HouMinXi:feat/astra-quality-family-fallback

Conversation

@HouMinXi

@HouMinXi HouMinXi commented Sep 13, 2026 •

Copy link
Copy Markdown
Contributor

openai-gpt-sol pins codex/gpt-6-astra-high. Live 2026-09-13 the Codex leg returned HTTP 200 and combo quality validation marked the stream streaming upstream error, so the combo step failed 502. Family fallback never ran: quality 502 returned from executeTargetAttempt without calling getNextFamilyFallback, and FAMILY_FALLBACK_TEMPLATES had no Astra rows.

onmi-gemini3.6 pins agy/gemini-3.8-flash-high. Empty-content already hops in nonStreamingProviderLeg, but the family table had no 3.8 rows. parseModel("agy/...") canonicalizes provider to antigravity, whose static catalog listed 3.7 only, so hop candidates were filtered out.

What

  • Add GPT-6 Astra effort-alias family (high -> max -> ultra).
  • On quality-invalid 200, hop the same connection to the next sibling instead of failing the combo step. Do not consume combo maxRetries. Guard hop when provider is missing.
  • Add Gemini 3.8 Flash family (high -> medium -> low). Prefer combo alias / providerHint for registry lookup so agy/ hops against the catalog the combo named.

Not subsumed by later merges

  • fix(sse): classify a 2xx body as a disguised upstream failure (#13461) #13910 classifies a 2xx body whose assistant text is the provider's own error prose (credits exhausted, and similar). This hop is for quality-empty 200: the stream is structurally a completion, combo validateQuality rejects it, and we want the next effort alias on the same connection. Tip executeTargetAttempt still returns qualityValidationFailure() immediately after handlePreContentStreamRetry; there is still no family hop.
  • fix(combo): retry pre-content streaming failures #13630 retries pre-content streaming failures. The hop here is after that helper returns false. Same function, different branch.

Catalog files were rebased onto buildSurfaceCatalog / ANTIGRAVITY_SHARED_MODELS. Unique vs current release/v3.8.51: family templates, the hop in executeTargetAttempt, providerHint lookup, tests.

Test

tests/unit/combo/execute-target-attempt.test.ts and tests/unit/model-family-fallback-notation.test.ts: 21/21. Inject: drop the Astra high template -> 3 fail; drop the hop continue -> 1 fail; drop Gemini 3.8 high template -> 1 fail. Restore: 21/21.

This is submitted for review and merge. The earlier hold wording was a mistake, not a request to defer.

⚠️ base-red inherited: #13866

@diegosouzapw

Copy link
Copy Markdown
Collaborator

Solid root-cause work on both the Astra quality-empty-200 case and the Gemini 3.8
family-table gap — ran your two test files at head, 21/21 pass. I noticed the PR body
ends with "Contributor only. Do not merge." — flagging that I'm respecting that and
holding this as defer rather than routing it toward merge. Whenever you're ready to
have it merged, just let us know and we'll also need to reconcile it with #13630, which
touches the exact same return branch in executeTargetAttempt.ts (different mechanism,
same hunk) — happy to help coordinate the rebase order at that point.

@diegosouzapw diegosouzapw changed the title fix(combo): hop Astra quality-empty 200 and Gemini 3.8 empty-content to family siblings [defer] fix(combo): hop Astra quality-empty 200 and Gemini 3.8 empty-content to family siblings Sep 15, 2026
@diegosouzapw

Copy link
Copy Markdown
Collaborator

Leaving this one as you asked — the title still carries [defer] and the body says contributor-only, do not merge. Noting for whenever you pick it up again:

  • open-sse/config/agyModels.ts and open-sse/config/antigravityModelAliases.ts now conflict with the tip: the catalog there was refactored to buildSurfaceCatalog / ANTIGRAVITY_SHARED_MODELS, while this branch still carries the literal AGY_PUBLIC_MODELS array shape. A rebase will need the new builder.
  • The idea itself (hopping a quality-invalid 200 to the next family sibling without consuming combo maxRetries) is sound and we would review it properly once you lift the defer.

Nothing to do on your side right now; just keeping the record straight.

@HouMinXi HouMinXi changed the title [defer] fix(combo): hop Astra quality-empty 200 and Gemini 3.8 empty-content to family siblings fix(combo): hop Astra quality-empty 200 and Gemini 3.8 empty-content to family siblings Sep 16, 2026
@HouMinXi

Copy link
Copy Markdown
Contributor Author

@diegosouzapw I do want this contribution merged. The earlier "Contributor only / Do not merge" wording was my mistake; it described who would perform the merge, not a request to hold the PR. Sorry for the confusion. I have removed that wording and the [defer] title prefix.

Your technical concerns still stand: the catalog changes need to use the current buildSurfaceCatalog / ANTIGRAVITY_SHARED_MODELS structure, and the quality-invalid return path must be reconciled with #13630. Removing the accidental hold is not a claim that those conflicts or checks are already resolved.

@diegosouzapw

Copy link
Copy Markdown
Collaborator

Segurando conforme o veredito defer do plan-file, e o conflito com a base reforça.

A PR está conflitante com release/v3.8.51 e o plan-file já a marcava como defer (⭐4) — ou seja, o adiamento não é pela qualidade, é de sequenciamento.

Ela mexe na mesma área que várias PRs mergeadas hoje (o caminho de combo/fallback recebeu #12956, #12957, #13630 e outras), então o conflito tende a crescer, não encolher, enquanto ela espera.

Sugestão: rebase sobre o tip atual antes de retomar a avaliação — o veredito ⭐4 foi medido contra uma base bem anterior e precisa ser reconfirmado contra o que existe hoje. Vale especialmente verificar se o hop de quality-empty 200 do Astra ainda é necessário depois da #13910 (classify a 2xx body as a disguised upstream failure), que ataca uma classe parecida de problema.

@HouMinXi
HouMinXi force-pushed the feat/astra-quality-family-fallback branch from a3270c7 to 38516bf Compare September 18, 2026 07:28
@HouMinXi

Copy link
Copy Markdown
Contributor Author

Rebased onto current release/v3.8.51.

The quality-empty hop is still needed. #13910 (classifyFakeSuccessBody) is allowlisted to pollinations / perplexity-web and looks for error prose in a 200 body. Astra quality-empty and Gemini 3.8 empty-content are a different class: HTTP 200 with no usable assistant text, not a disguised failure sentence.

Conflict resolution kept the tip's pre-content streaming retry (handlePreContentStreamRetry) and runs the family hop after it. Unknown providers do not hop.

Tests: 21/21 on the hop + family tables. Injection: forcing familyNext = null turns the Astra hop red; restoring it is green.

Head 38516bf198.

@HouMinXi

Copy link
Copy Markdown
Contributor Author

Rebased onto current release/v3.8.51 (1603c86e06). GitHub now reports MERGEABLE / 0 behind.

Checked #13910 against this hop: it classifies a 2xx body whose assistant text is the provider's own error prose. The Astra case is a quality-empty 200 (validateQuality → streaming upstream error) on a structurally normal completion. Tip executeTargetAttempt still returns qualityValidationFailure() right after handlePreContentStreamRetry; there is no family hop on that path. #13630 covers the pre-content retry helper above this branch.

Family templates for Astra and Gemini 3.8, plus the hop, are still unique vs tip. Tests 21/21 on this tree.

Please re-evaluate when you have a slot. The earlier defer / contributor-only wording is withdrawn.

@HouMinXi
HouMinXi force-pushed the feat/astra-quality-family-fallback branch from 38516bf to 7635948 Compare September 18, 2026 14:41
@HouMinXi

Copy link
Copy Markdown
Contributor Author

Rebased onto current release/v3.8.51 (36493a6270). Unique commits unchanged.

@HouMinXi

Copy link
Copy Markdown
Contributor Author

Live combos this hop is for were not Gemini failing over to Claude.

onmi-gemini3.6 has 10 steps. Every step is agy/gemini-3.8-flash-high on a different Antigravity account. Quality-empty 200 fails that account and tries the next Gemini account, or returns 502.

openai-gpt-sol has two steps, both codex/gpt-6-astra-high (live is -xhigh) on two Codex accounts. Quality-empty 200 does the same thing: next account, or 502.

On 2026-09-13 Codex returned HTTP 200 with no visible text. validateQuality marked streaming upstream error. executeTargetAttempt returned 502 without calling getNextFamilyFallback, and the family table had no Astra rows. Gemini 3.8 already hops empty content in nonStreamingProviderLeg, but agy/ canonicalizes to antigravity and that catalog only listed 3.7, so the hop candidates were filtered out.

This PR hops the same connection to the next effort alias: Astra high -> max -> ultra, Gemini 3.8 high -> medium -> low. It does not spend combo maxRetries. After the family is exhausted, the existing step failover still runs.

A combo that pins *-high can still return medium/low (Gemini) or max/ultra (Astra) after a quality-empty 200. If that is unacceptable, keep those ids out of FAMILY_FALLBACK_TEMPLATES.

diegosouzapw pushed a commit that referenced this pull request Sep 22, 2026
…#14213)

Merged. Thank you, @maxmad64bis.

Four distinct shapes of "the turn came back empty" — reasoning-only 200, zero-chunk 502, pre-first-byte drop, header-then-quiet stall — collapsed into one retry through the normal credential path, with a bounded `EMPTY_TURN_RETRY_MAX` budget and the stall gated on `STREAM_READINESS_TIMEOUT_MS`. Retrying server-side *before* anything reaches the client is the right layer for this; the related issues you listed (#11902 client-side signalling, #12656 first-byte watchdog, #13603 combo-level hop) each sit one stage away, and you were explicit about which.

The thing that made this safe to take: `FLUSH_EMPTY_RETRY_ENABLED` defaults to `"false"` and the flag-off path is byte-identical. Verified on the final tree, not taken on trust.

Validation before merge: reconciled twice as the tip moved under it. The only conflict each time was the additive `_rebaseline_*` key at the top of `config/quality/file-size-baseline.json` — resolved keep-both, JSON re-parsed, 523 keys, every tip entry preserved. `flush-empty-retry` + `flush-empty-retry-hook` + `feature-flags-settings` + `server-owned-tool-loop-flag` **100 pass / 0 fail** across 12 suites on the final tree. `check:file-size` OK, `check:changelog-integrity` OK, `typecheck:core` clean.

Two numbers I re-measured rather than trusting:

- The feature-flag count. Both count assertions said 76; the tip has 75 and this PR adds exactly one, so 76 is right and needed no change. `docs/reference/FEATURE_FLAGS.md` totals and the `### Network (18)` heading agree.
- `open-sse/handlers/chatCore.ts` — your ceiling of 6392 was exact on your fork base but stale; the reconciled tree measures **6400**, which is your `+113` own growth on top of the tip's 6287. Set to 6400. Note that leaves the file sitting exactly at its ceiling with zero headroom, so the next PR to touch `chatCore.ts` will need a rebaseline of its own.

Note: the release tip is currently base-red on `open-sse/executors/auggie.ts` (#14496). Inherited, unrelated to this diff.
@HouMinXi
HouMinXi force-pushed the feat/astra-quality-family-fallback branch 2 times, most recently from 59281d3 to 68e3e2e Compare September 24, 2026 17:21
@diegosouzapw diegosouzapw changed the title fix(combo): hop Astra quality-empty 200 and Gemini 3.8 empty-content to family siblings [defer] fix(combo): hop Astra quality-empty 200 and Gemini 3.8 empty-content to family siblings Sep 28, 2026
@diegosouzapw diegosouzapw added the deferred-v3.8.52 Grande demais / suspeito para o lote atual; precisa de sessão dedicada no ciclo v3.8.52 label Sep 28, 2026
@diegosouzapw
diegosouzapw changed the base branch from release/v3.8.51 to release/v3.8.52 September 29, 2026 11:25
@diegosouzapw

Copy link
Copy Markdown
Collaborator

Re-homed to release/v3.8.52: v3.8.51 entered its release freeze, so the branch now belongs to the release captain and development continues on the next cycle. Nothing is wrong with this PR — it just needed a live base. No action needed from you; CI will re-run against the new base.

openai-gpt-sol pins two Codex Astra connections. A 200 stream with
no visible text failed the combo quality gate and skipped family
fallback, so the client saw 502 empty output. Astra high/max/ultra
are the same catalog family; hop on the same connection before
leaving the step.

Signed-off-by: Minxi Hou <houminxi@gmail.com>
agy/gemini-3.8-flash-high parses as antigravity, so hop candidates
were filtered by a catalog that only listed 3.7. Prefer the combo
alias and pin 3.8 ids on both agy and antigravity static catalogs.

Signed-off-by: Minxi Hou <houminxi@gmail.com>
Rebase onto release/v3.8.51. Quality-empty 200 still hops Astra and
Gemini 3.8 siblings. Tip added handlePreContentStreamRetry; the hop
runs after that retry, and unknown providers do not hop.

pollinations/perplexity-web error prose, not empty-content quality fails.

Related to cheaper-inference#13603.

Signed-off-by: Minxi Hou <houminxi@gmail.com>
@HouMinXi
HouMinXi force-pushed the feat/astra-quality-family-fallback branch from 68e3e2e to e8ba143 Compare September 29, 2026 16:12
@diegosouzapw diegosouzapw removed the deferred-v3.8.52 Grande demais / suspeito para o lote atual; precisa de sessão dedicada no ciclo v3.8.52 label Oct 1, 2026
@diegosouzapw diegosouzapw changed the title [defer] fix(combo): hop Astra quality-empty 200 and Gemini 3.8 empty-content to family siblings fix(combo): hop Astra quality-empty 200 and Gemini 3.8 empty-content to family siblings Oct 1, 2026
diegosouzapw and others added 3 commits October 7, 2026 22:31
… tip (cheaper-inference#13603)

The tip already serves gemini-3.8-flash-high/medium/low in
antigravitySharedModels.ts; the PR's copy merged textually clean but left
duplicate ids, failing tests/unit/agy-antigravity-shared-catalog-12724.test.ts.

Co-authored-by: diegosouzapw <8016841+diegosouzapw@users.noreply.github.com>
…int is given (cheaper-inference#13603)

Folding providerHint into explicitProvider made getNextFamilyFallback
prefix a bare id with the hint provider (an unprefixed opus-5 id plus a
provider hint came back as <provider>/<sibling> instead of the bare
sibling), failing tests/unit/t30-kiro-400-model-unavailable.test.ts.
Registry lookup keeps the PR's hint/alias-first order; the output prefix
comes only from a provider written in the model id. Also records the
executeTargetAttempt.ts ceiling for the family hop wiring (1280 -> 1294).

Co-authored-by: diegosouzapw <8016841+diegosouzapw@users.noreply.github.com>
@diegosouzapw
diegosouzapw merged commit 2a5aae2 into cheaper-inference:release/v3.8.52 Oct 8, 2026
21 of 41 checks passed
diegosouzapw added a commit that referenced this pull request Oct 8, 2026
Drains 9 v3.8.52 tip unit reds: 1 production fix (combo family-fallback skipped for request-scoped refusals, regression of #13603) + 8 test alignments to intentional contract changes (#15301 #15132 #15035 #15478 #15310 #15710 #15487), each annotated. Validated on a combined board with #15902 after clean npm ci: 32 previously-red files 327/329 (remaining 2 = quality-rail-gate-membership owned by #15682, and route-namespaces fixed in #15902), typecheck/open-sse/api typecheck, lint, 22 gates.
@HouMinXi
HouMinXi deleted the feat/astra-quality-family-fallback branch October 10, 2026 11:44
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants