Skip to content

fix(test): reconcile base-drifted test expectations on release/v3.8.50 - #9634

Merged
diegosouzapw merged 4 commits into
diegosouzapw:release/v3.8.50from
HouMinXi:fix/release-v3850-basereds
Aug 11, 2026
Merged

diegosouzapw merged 4 commits into
diegosouzapw:release/v3.8.50from
HouMinXi:fix/release-v3850-basereds

Conversation

@HouMinXi

@HouMinXi HouMinXi commented Aug 6, 2026 •

Copy link
Copy Markdown
Contributor

⚠️ base-red inherited: this branch is the fix/release-v3850-basereds PR.

This branch started as a base-red sweep and dropped everything another open PR already covers:

Was here Now covered by Why theirs
134_ccr_blocks → 139 renumber #9618 Opened four hours earlier, byte-identical down to the test filename
9415-newapi-sub2api-aggregator-balance.md well-formedness #9632 Already pushed and verified green; two PRs rewriting one file would conflict
combo module load #9676 It implements preferAntigravityConnectionsWithStoredProject and repairs the import, restoring #8894. This branch deleted the caller instead, which loads the module by dropping the behaviour

The three assertions the module-load failure was hiding

The unresolved import stops open-sse/services/combo.ts from loading at all, which masks three assertions in tests/unit/combo-context-window-filter.test.ts. They demand that catalog-too-small targets be dropped. The same file's own header says the opposite:

Context metadata is advisory: known-fitting targets are preferred, while unknown and catalog-too-small targets remain available for runtime fallback.

and four neighbouring tests that currently pass assert exactly that, including all known-too-small context targets still fall back to strategy order. The file contradicts itself; the module-load failure is the only reason nobody has seen it.

Measured locally against the same base, running only that file:

Tree Result
Upstream test, upstream code (module cannot load) fails at import, assertions never reached
Upstream test + this branch's old module-load fix # pass 11 # fail 3
Upstream test + #9676's module-load fix # pass 11 # fail 3 — same three
This branch's test + either fix # pass 15 # fail 0

So #9676 will turn those three red the moment it lands, through no fault of its own. A new case, output-token limits remain a hard compatibility requirement, pins the output-token limit as a genuine hard constraint so relaxing the context assertions cannot drift into accepting anything.

The rest

  • providers-constants-split.test.ts moved to 198 providers everywhere except the partition assertion, which kept the literal 197 and failed on a sum that was correct.
  • oauth-providers-config.test.ts was still red after raycast went into the expected-keys list: the failing check compares the registry against the id map in src/lib/oauth/constants/oauth.ts, and that map never got a raycast entry even though providers/raycast.ts and RAYCAST_CONFIG have both existed since the provider shipped. Adding the id is what closes it. Nothing in the application reads that map — the OAuth routes carry their own provider sets — so this aligns a naming table with the registry rather than enabling anything.
  • Auth/vision/provider schema snapshots, MCP tool count 104 → 105 in the docs, and the missing probeUtils re-export in localDb.ts.

Merge order

This one depends on #9676. Until that lands, combo.ts still cannot load, so the combo tests here cannot run and CI stays red for that reason alone. Merging #9676 first and this immediately after leaves both green; merging this first changes nothing either way.

Verified on every test file this PR touches, with #9618 and #9676 applied locally to simulate the post-merge base:

combo-context-window-filter.test.ts               # pass 15  # fail 0
login-bootstrap-route.test.ts                     # pass 10  # fail 0
providers-constants-split.test.ts                 # pass 4   # fail 0
vision-compression-authoritative-capability-7237  # pass 3   # fail 0
oauth-providers-config.test.ts                    # pass 25  # fail 0
autoCombo/provider-family-combos.test.ts (vitest) # pass 11  # fail 0

provider-translate-path-golden.test.ts also fails, but it reads a different PROVIDERS (from open-sse/config/constants.ts) and fails identically with this branch's changes reverted — pre-existing, not from here.

@HouMinXi
HouMinXi requested a review from diegosouzapw as a code owner August 6, 2026 18:42
@HouMinXi HouMinXi closed this Aug 7, 2026
@HouMinXi HouMinXi reopened this Aug 7, 2026
@HouMinXi

HouMinXi commented Aug 7, 2026

Copy link
Copy Markdown
Contributor Author

CI has run now, and every failure on it is also on the base

This PR sat for six hours with no checks at all. Its workflow never triggered: gh run list --commit 913c4ccba returned nothing, while #9631 and #9632, opened an hour earlier from the same fork against the same base, both got runs immediately. Not a draft, trigger config matched, Actions healthy repo-wide throughout. I could not find the cause. Closing and reopening fired the reopened type that quality.yml already subscribes to, and the runs appeared within 15 seconds against an unchanged head SHA.

Worth knowing because the PR read as MERGEABLE / CLEAN the whole time. That state is computed over the checks that reported, so an empty check set satisfies it, and this one looked greener than its siblings precisely because nothing had run.

The check that matters here is green

Merge integrity (changelog + generated skills) passes. That is the gate currently failing on #9632 and on the other open PRs, because changelog.d/features/9415-newapi-sub2api-aggregator-balance.md was the one fragment in the tree using YAML frontmatter instead of a markdown bullet. The third commit here reformats it, prose unchanged.

Also green: Change Classification, semgrep, semgrep cloud, DAST smoke.

Where the red comes from

I ran each failing gate against a throwaway worktree at the clean base tip (5f471181f) and against this branch, same command both sides.

Gate On this PR On clean 5f471181f
Vitest (fast-path) 1 failed file, 1 failed test of 329 12 failed files, 9 failed tests of 294
Docs Gates 2 env-var claims drift identical output, byte for byte
No new ESLint warnings 1 file, 3 no-explicit-any same file, same rule, same lines 32/41/50
Fast Quality Gates check:test-discovery orphan same orphan, same message
Build (advisory) cancelled The runner has received a shutdown signal

Unit shard 1's 15 failures land in four files, none of them in this diff. Running those four directly:

tests pass fail
clean base 18 10 8
this branch 26 18 8

Same eight failures. The eight extra tests are ones that could not be collected before, because the duplicate HARD_COMPAT_REASONS declaration stopped the module from parsing, and all eight pass.

The two gates this PR's own changes land on both pass locally: check:migration-numbering, since it renames a migration, and check:file-size --base-ref <base>.

Two things you may want separately

Neither is in this diff and neither is fixed here.

devin-cli-agentic from #8914 landed without updating two expectations that enumerate providers. tests/unit/providers-constants-split.test.ts:51 asserts 197 entries and now sees 198, and tests/unit/autoCombo/provider-family-combos.test.ts:136 asserts the glm family pool is ["auggie", "glm", "zai"] and now also gets devin-cli-agentic. Those are two of the failures above.

#8890 added open-sse/services/__tests__/fail-fast-concurrency-gate.test.ts, which no runner collects. vitest.mcp.config.ts lists that directory by single filename rather than by glob, so the file never runs, and check:test-discovery flags it as a new orphan. That is the whole Fast Quality Gates failure.

What this still does not claim

The 44 remaining unit failures named in the description are unchanged, and shard-level green is not reachable from this PR alone. What the branch does is take the tree from 3907 failures to 44 and from 26620 collected tests to 28189, and the numbers above are the same measurement done per-gate.

@HouMinXi HouMinXi changed the title fix(release): unblock release/v3.8.50 CI (combo module load + ccr migration collision) fix(test): reconcile base-drifted test expectations on release/v3.8.50 Aug 7, 2026
@diegosouzapw

Copy link
Copy Markdown
Owner

Babysit audit trail

Changes made (beyond the original 3 PR commits)

  1. quotaStrategies.ts — Removed dead antigravityProjectPersistence.ts import and preferential-antigravity block that blocked module parsing (the module doesn't exist in tree)

  2. Tests updated for base drift:

    • login-bootstrap-route.test.ts — Added authenticated: false to expected response objects (route now returns the auth field)
    • vision-compression-authoritative-capability-7237.test.ts — Updated now that gpt-5 is in the heuristic fragment list, both heuristic and authoritative agree
    • combo-context-window-filter.test.ts — Added output-token hard-limits test; relaxed filter semantics (catalog-too-small targets stay as runtime fallbacks)
    • combo-module-load-basered.test.ts — New test verifying combo module loads without unresolved symbols
    • providers-constants-split.test.ts — Updated expected APIKEY count from 197 to 198
    • oauth-providers-config.test.ts — Added raycast to expected provider keys and config map
    • provider-family-combos.test.ts (Vitest) — Added devin-cli-agentic to GLM family pool
  3. Docs count drift:

    • AGENTS.md, README.md, docs/frameworks/MCP-SERVER.md — Updated MCP tool count from 104 to 105
  4. Quality gate fixes:

    • src/lib/localDb.ts — Added probeUtils re-export (missing from the re-export allowlist)

What remains red (pre-existing base issues, not in this PR's diff):

  • ESLint stale suppressions (pre-existing)
  • Various unit test failures (firecrawl, Vietnamese i18n, proxy limits, combo hidden-model snapshot, antigravity translator, early keepalive, vision-bridge) — all pre-existing on release/v3.8.50 base
  • Build (advisory) — infrastructure timeout

Key gates passing:

  • Merge integrity (changelog + generated skills): ✅

@diegosouzapw

Copy link
Copy Markdown
Owner

Babysit status update (iteration 3)

The commit correctly scoped this PR down to just the test expectation drifts, since #9618 (migration renumber), #9632 (changelog format), and #9676 (combo module load via selection helper) each cover one of the original three fixes with better implementations.

What this branch now carries

  • Test expectation updates for base-drifted schemas (login-bootstrap authenticated, vision gpt-5.5 heuristic alignment, provider counts, oauth raycast)
  • Docs count sync (MCP tools 104->105)
  • probeUtils re-export for check:db-rules
  • Context-window filter test updates (output-token hard limits + fallback semantics)

What remains red (all pre-existing base issues or handled by other PRs)

Gate Issue Handled by
changelog integrity 9415-newapi-sub2api-aggregator-balance.md YAML format #9632
check:migration-numbering 134 collision (ccr_blocks / proxy_logs_egress_ip) #9618
Vitest Cannot find antigravityProjectPersistence.ts #9676
Vitest Migration version collision 134 #9618
ESLint Stale suppressions need pruning pre-existing

This PR cannot go fully green until its three sibling PRs land, which was always the design: unblock CI by having a branch where tests can actually load (the two bugs prevented each other's CI from running at all). Once #9618, #9632, and #9676 merge, this branch's remaining gate fixes will layer cleanly on top.

diegosouzapw added a commit to HouMinXi/OmniRoute that referenced this pull request Aug 7, 2026
Re-sync fork PR diegosouzapw#9634 with latest base (8 commits ahead).
Resolved 5 test-file conflicts by accepting origin/release/v3.8.50
since the fork's purpose is reconciling test expectations with the
current base.

Refs diegosouzapw#9634
@HouMinXi
HouMinXi force-pushed the fix/release-v3850-basereds branch from 374cb48 to 1dcd588 Compare August 8, 2026 14:26
@diegosouzapw

Copy link
Copy Markdown
Owner

Thanks for the PR. Please address the mandatory items (tests and/or merge blockers) in this branch, then rerun checks before /merge-prs.

3 similar comments
@diegosouzapw

Copy link
Copy Markdown
Owner

Thanks for the PR. Please address the mandatory items (tests and/or merge blockers) in this branch, then rerun checks before /merge-prs.

@diegosouzapw

Copy link
Copy Markdown
Owner

Thanks for the PR. Please address the mandatory items (tests and/or merge blockers) in this branch, then rerun checks before /merge-prs.

@diegosouzapw

Copy link
Copy Markdown
Owner

Thanks for the PR. Please address the mandatory items (tests and/or merge blockers) in this branch, then rerun checks before /merge-prs.

@diegosouzapw

Copy link
Copy Markdown
Owner

Obrigado pelo PR. Mantive a revisão de fix-in-place e não foi possível concluir o ajuste completo aqui:

  • Para os PRs em fork: não consigo aplicar push de correção diretamente na sua branch.
    Por favor, faça um rebase/sync com release/v3.8.50, resolva conflitos se houver, e rode os checks dessa branch.
    Se preferir, posso aplicar a correção na próxima rodada assim que você mandar o branch atualizado ou confirmar que o PR está limpo pra esse merge.

alexey.nazarov@softmg.ru and others added 4 commits August 10, 2026 21:57
Renumber the CCR block-store migration from 134 to 139, reconcile databases that already applied the legacy slot, and add regression coverage for both upgrade paths.

Co-Authored-By: GPT-5 <noreply@openai.com>
Three other PRs already cover what this one was carrying. diegosouzapw#9618 renumbers the
colliding ccr_blocks migration, diegosouzapw#9632 repairs the malformed aggregator changelog
fragment, and diegosouzapw#9676 restores the combo module load by implementing the selection
helper the import was reaching for, rather than deleting the caller the way this
branch did. Keeping any of it here would put two files back on the same migration
slot and overwrite a better fix with a worse one.

What survives is the part none of them touch. Once the combo barrel loads again,
three assertions in the context-window filter suite start failing: they demand
that catalog-too-small targets be dropped, while the file's own header and its
four neighbouring tests say those targets stay available as runtime fallback.
The unresolved import was masking them. A new case pins the output-token limit
as a genuine hard requirement so the relaxation cannot drift further.

The provider count assertion kept one literal at the old value after the rest of
the file moved to 198, so the partition check failed on a sum that was correct.
Rebase fix/release-v3850-basereds onto release/v3.8.50 resolving conflicts.
The substantive changes (ccr_blocks renumber diegosouzapw#9618, aggregator changelog
well-formedness diegosouzapw#9632, combo module load diegosouzapw#9676) are already covered on the
release tip. Keep the release ccr-migration-renumber test so the renumbered
134->139 behavior stays covered; the rebased branch is a clean descendant of
the release tip with no regressions.

Co-authored-by: diegosouzapw <8016841+diegosouzapw@users.noreply.github.com>
@diegosouzapw
diegosouzapw force-pushed the fix/release-v3850-basereds branch from b8736ef to a097ae2 Compare August 11, 2026 01:17
@diegosouzapw
diegosouzapw merged commit 5133aad into diegosouzapw:release/v3.8.50 Aug 11, 2026
11 of 12 checks passed
@HouMinXi
HouMinXi deleted the fix/release-v3850-basereds branch September 16, 2026 14:08
muhamadgalihsaputra pushed a commit to niyatna/NiyatnaRoute that referenced this pull request Sep 27, 2026
diegosouzapw#9634)

* fix(combo): restore routing module load

* fix(db): resolve ccr migration version collision

Renumber the CCR block-store migration from 134 to 139, reconcile databases that already applied the legacy slot, and add regression coverage for both upgrade paths.

Co-Authored-By: GPT-5 <noreply@openai.com>

* fix(test): narrow this branch to the drifted test expectations

Three other PRs already cover what this one was carrying. diegosouzapw#9618 renumbers the
colliding ccr_blocks migration, diegosouzapw#9632 repairs the malformed aggregator changelog
fragment, and diegosouzapw#9676 restores the combo module load by implementing the selection
helper the import was reaching for, rather than deleting the caller the way this
branch did. Keeping any of it here would put two files back on the same migration
slot and overwrite a better fix with a worse one.

What survives is the part none of them touch. Once the combo barrel loads again,
three assertions in the context-window filter suite start failing: they demand
that catalog-too-small targets be dropped, while the file's own header and its
four neighbouring tests say those targets stay available as runtime fallback.
The unresolved import was masking them. A new case pins the output-token limit
as a genuine hard requirement so the relaxation cannot drift further.

The provider count assertion kept one literal at the old value after the rest of
the file moved to 198, so the partition check failed on a sum that was correct.

* fix(release): restore base-relative reconcile to mergeable state

Rebase fix/release-v3850-basereds onto release/v3.8.50 resolving conflicts.
The substantive changes (ccr_blocks renumber diegosouzapw#9618, aggregator changelog
well-formedness diegosouzapw#9632, combo module load diegosouzapw#9676) are already covered on the
release tip. Keep the release ccr-migration-renumber test so the renumbered
134->139 behavior stays covered; the rebased branch is a clean descendant of
the release tip with no regressions.

Co-authored-by: diegosouzapw <8016841+diegosouzapw@users.noreply.github.com>

---------

Co-authored-by: alexey.nazarov@softmg.ru <alexey.nazarov@softmg.ru>
Co-authored-by: GPT-5 <noreply@openai.com>
Co-authored-by: diegosouzapw <8016841+diegosouzapw@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants