Repository navigation
fix(tests): drain base-red cluster from 2026-08-23 merges (#9985) - #11280
Merged
diegosouzapw merged 7 commits intoAug 23, 2026
Merged
Conversation
Drain the unit-shard reds the 2026-08-23 merge wave left on release/v3.8.50, each discriminated as stale-test (contract moved intentionally, test aligned) vs real bug (fixed in code/messages): 1. check-deps 6A.8 allowlist — #11224 added @testing-library/dom and @testing-library/user-event to package.json without the gate allowlist. Both are legitimate: @testing-library/dom is a required peer of @testing-library/react v16, and user-event is the official companion for UI tests. Fix: allowlist entries with a justification note referencing #11224/#9985. 2. search-route 400-vs-fallback — #11097 intentionally changed the zero-credential /v1/search contract: instead of returning 400 it promotes the fallback-only duckduckgo-free provider so out-of-the-box search works. The test pinned the OLD 400 contract. Fix (contract alignment): the test now pins the new fallback contract — 200, provider duckduckgo-free, DuckDuckGo lite endpoint called, results parsed from lite HTML. 3. codex catalog token limits (4 named reds + 2 sibling sweeps) — #11179 raised GPT_5_6_CODEX_CAPABILITIES from the 272K pricing tier to the real usable 872K window (live evidence: 390K served past 272K with HTTP 200) and updated codex-gpt56-catalog.test.ts but missed the sibling pins. Stale tests aligned: models-catalog-combo- metadata (max_input_tokens now clamps to min(872000, 500000 override) = 500000), vscode-token-routes x3 (872000), and two more found in the sibling sweep: vscode-token-routes-gpt56 and provider-models-route- codex (conservative merge semantics unchanged: pinned 872000 < live 999999 still wins). 4. CLI catalog counts (3 reds) — #11166 added prime-agent (agent category) without the cardinality pins: EXPECTED_AGENT_COUNT 8->9, total 34->35, D15 agent list + prime-agent, cli-tools-schema id list. Also added the missing English/Vietnamese cliTools descriptions for prime-agent (cli-catalog-display-contract) and corrected the stale CLI-TOOLS.md agent count (8->9). 5. setup-qwen container guard — since #10057 the container guard exits 2 on ephemeral runtimes; the two tests exercising the merge/write path were environment-sensitive (red on container devboxes). Fix: pass allowContainerWrite so the tests are hermetic everywhere; the guard keeps its own dedicated coverage. 6. i18n health verdict namespace (real bug, fixed in messages) — #11224 added the verdict/diagnostics strings to the `sidebar` namespace but health/page.tsx reads them via useTranslations("health"), so the page rendered raw keys in every locale and the "direct translation calls have English messages" gate went red. Fix: keys moved sidebar->health in en.json + vi.json (the only locales that had them), plus health.healthSubtitle added. The sidebar never referenced them (verified: no usage), and sidebar.healthSubtitle (its real sidebar key) is untouched. 7. i18n pt-BR drift (22 keys) — the 08-23 wave (#11224/#11228/#11215/ #11204/#11195) added English keys never translated: common.batch*, endpoint.*, cliCommon.concept.acp.warning, resilienceConnections.*, plus the moved health.* keys and the prime-agent cliTools description. Fix: pt-BR translations added — 0 missing keys vs en; vi strict parity re-verified (0 missing, 0 extra). Validation: every touched test file RED->GREEN individually (node --import tsx/esm --test), typecheck:core clean, docs-counts-sync soft-pass, cli-i18n gate PASS, 8/8 unit shards re-run on the branch. Refs #9985
…ner guard (#10057) Same class as the setup-qwen drain in c2df757: since #10057 the config-write guard exits 2 on ephemeral container runtimes, so the four runSetupOpenCodeCommand tests and the /api/cli-tools/apply JSONC test were environment-sensitive (red on container devboxes, green on ubuntu-latest CI). They exercise the plugin install/merge path, not the guard — pass allowContainerWrite / set the env override so they run deterministically everywhere. The guard keeps its own dedicated coverage. Refs #9985
…e-v3.8.50-basereds-cluster
diegosouzapw
pushed a commit
that referenced
this pull request
Aug 23, 2026
…nto v3.8.51) (#10952) Validated on the resolved merge against the current tip (527da65 + the post-#11281 rebaseline): the single conflict was a comment-only collision in providers/[id]/models/route.ts (kept the tip's #10828-ordering note). Focused suites 125/125 across all 13 touched test files (build-sqlite-stub, cc-compatible, copilot-claude-messages, copilot-gemini-route, executor-github, ghe-copilot, github-copilot-discovery-token, github-copilot-model-discovery, noauth-sibling-7620, provider-header-profiles, provider-models-config, request-log-payloads, upstream-error-passthrough), typecheck:core clean, file-size/changelog-integrity OK. Merged --admin over the inherited 2026-08-23 base-red cluster (#9985) — the reds are proven tip failures (CLI catalog cluster + @testing-library allowlist, being drained by #11280), not from this diff. Note: the rebase means several items the body listed (relay x-relay-path SSRF, /v1/search blocked-providers, #10736 rotation fence, #10903, #10865, #10899, #10916) already landed upstream and are NOT in this delta — the delta is: better-sqlite3 build guard + build heap/worker caps + telemetry-off (#10060 re-derived), credential-echo passthrough refusal + OCR/moderation redaction + call-log key redaction, Copilot CLI 1.0.81-6 wire identity + Claude→/v1/messages name-matched routing + discovery token fix, CC model_not_found 400, compat overrides for no-auth aliases (#7620-pinned). The Copilot wire-identity change is the one to watch in production. Thank you @arminanton — and the ported-author credits in the commit history (@rqzbeh, yidecode, the #10899/#10916 authors) are preserved. Your config-posture finding (REQUIRE_API_KEY default vs 0.0.0.0) is noted for a maintainer decision, as you scoped it.
, #11267) Three reds the PR's CI surfaced after the base advanced past the branch cut — each discriminated with its origin PR: 1. chatcore-translation-paths 'Combo skip behavior' (shard 3/4) — REAL BUG in #11178: the incompatible-reasoning action derivation switched from the explicit fallback config to isComboStep = Boolean(comboStepId || comboExecutionKey). Combos whose records carry no explicit stepId/executionKey (plain model-list combos) had their explicit reasoningTransportFallback: 'skip' config silently degraded to 'drop', contradicting the PR's own stated intent ('combos keep their explicit strategy'). Fix: isComboStep now honors the isCombo marker (isCombo || step ids present). RED->GREEN on the exact CI failing test; the #10959 single-target drop defaults stay green. 2. check-db-rules-classification 'recovery zero importers' (shard 1/4) — STALE GATE, not dead code: #11238 converted bin/cli/runtime.mjs dynamic imports to the Windows-safe projectFileUrl('...') idiom, and the gate's importer regexes only recognized static/from/template import forms. recovery's only importer became invisible. Fix: gate pattern set extended to recognize import(projectFileUrl('…/db/<mod>.ts')). 3. mutation-test-coverage gate — #11267 added tests/unit/quota-exhaustion-cutoff-opencode.test.ts covering src/sse/services/auth.ts without registering it in stryker.conf.json tap.testFiles. Fix: register it (gate green locally). Refs #9985
…e-v3.8.50-basereds-cluster
…11205 follow-ups) The base advanced twice past the branch cut (15 + 2 merges); the PR's CI surfaced five more reds on the merged head, each discriminated: 1. check-env-doc-sync + Docs Gates: OMNIROUTE_BUILDING referenced in code (src/lib/buildPhase.ts, db/core, callLogArtifacts — build-phase stub signal from #10060 via #10952) but missing from .env.example / ENVIRONMENT.md. Fix: document it in both (never-set-for-server note). 2. db-core-init 'build phase uses an in-memory database' — STALE TEST: #10060 deliberately replaced the real in-memory build DB with a no-op stub (native better-sqlite3 aborts the Next.js build worker on exit, node::RemoveEnvironmentCleanupHook); the stub contract is pinned by tests/unit/build/10060-build-sqlite-stub.test.ts. The old test pinned the superseded contract (real memory DB with migrations). Aligned: stub driver, no-op queries, no file created. 3. 8134-github-t5-fallback-filter — STALE FIXTURE: #10952 added claude-opus-4.6 to the github registry, invalidating the fixture's 'no 4.6 tier' assumption. The provably-absent tier moved to claude-opus-4-6-thinking (still absent); test rewritten against the real ladder (4.8 -> 4.7 -> 4.6 -> skip 4-6-thinking -> 4.5), keeping the original #8134 regression assertion (absent tier never returned). 4. cli-tools Codex/Copilot fingerprint — STALE PIN: #10952 bumped GITHUB_COPILOT_CLI_VERSION 0.54.0 -> 1.0.81-6 without the sibling fingerprint pin. Assertion aligned to GitHubCopilotChat/1.0.81-6. 5. stryker.conf.json — drop a duplicate tap.testFiles entry for quota-exhaustion-cutoff-opencode.test.ts created by the merge (the base registered it too after my fix in 17f5e4e). Validation: env-doc-sync gate green; 8134 2/2, cli-tools 23/23, db-core-init build-phase test, check-env-doc-sync.test.ts all RED->GREEN locally (node --import tsx/esm --test). Refs #9985
#11249/#11251/#11252/#10952 follow-ups) The gate (eslint full-tree, baseline 0) failed on the merged head with 26 unsuppressed errors, all introduced by same-day base merges — none by this PR. Each fixed at the cause (no new suppressions): 1. tests/integration/qdrant-routes.test.ts (#11249/#11213): 20x no-explicit-any on route-invocation casts. Fix: typed asNextRequest() adapter (Request -> NextRequest) replacing every "as any". 2. tests/unit/effort-tiers-loop-catalog-e2e.test.ts (#11252): unused after/beforeEach imports — newly error-level under the no-unused-vars ratchet #11247 shipped the same day. Fix: trim the import (the file uses test.after/test.beforeEach method forms). 3. EditConnectionModal.tsx (#11251): react-hooks/set-state-in-effect on the modal-open setFormData init. The pattern (sync form state with the loaded connection when isOpen flips) is the codebase's sanctioned exception — same justified eslint-disable-next-line already used in FreePoolTab.tsx and BatchConceptCard.tsx; a key-remount of the 30+ field form would be a behavior-risking restructure for a basereds PR. 4. tests/unit/usage-service-hardening.test.ts (shard 2/4): last stale Copilot wire-identity pin — #10952 re-based it on the live-captured CLI 1.0.81-6 (copilot-developer-cli integration id, API version 2026-08-01). Assertions aligned (UA, Editor-Version, Editor-Plugin-Version, X-GitHub-Api-Version). Validation: all four files eslint-clean with the frozen suppressions; usage-service-hardening 23/23 and effort-tiers 2/2 RED->GREEN (node --import tsx/esm --test). Refs #9985
This was referenced Aug 24, 2026
Merged
diegosouzapw
added a commit
that referenced
this pull request
Aug 25, 2026
… a masked assert
Second base-red batch from the release pre-flight, measured on the .113 with a
clean npm ci (the devbox tree resolves eslint-plugin-react-hooks 7.1.1 from a
stray pnpm store instead of the lockfile 7.0.1 and reports 925 phantom errors).
Provider count 350 -> 352, one root cause behind three reds. Two providers
landed this cycle (volcengine-agent-plan, volcengine-coding-plan) without
regenerating the artifacts that quote the count:
- docs/reference/PROVIDER_REFERENCE.md regenerated (gen:provider-reference).
- README / AGENTS / llm.txt (+42 mirrors) / package.json description / 4 SVG
diagrams updated, including the section heading AND the anchor that links to
it, so the link does not break.
- tests/snapshots/provider/translate-path.json regenerated. The diff is purely
additive: 46 insertions, 0 deletions, exactly the two new providers.
GLM effort tiers. #11415 added the explicit glm-5.3-max tier and left two
sibling vitest specs pinning the old 16-model inventory and an empty tier list
for it. Aligned to the shipped contract (inventory order matches glmProvider.ts;
glm-5.3-max declares ["max"]).
Test-masking. Four assert reductions surfaced once the deleted-file signal was
resolved. Three are legitimate and are allowlisted with their reasoning:
#11355 inverted the startup-cooldown contract (preserve future quota cooldowns),
#11280 replaced two unrolled hops with a 3-hop loop that asserts MORE, and the
Gemini 3.5 Flash retirement removed the models those capability asserts described.
The fourth was real masking: #10960 rewrote the oneproxy status test to install a
stream mock, immediately overwrite it with a passthrough to the real fetch, and
assert `calls.length >= 0` — always true. Restored to assert what the test name
claims (the JSON-RPC tools/call carries omniroute_oneproxy_stats and its result
reaches the caller), with a scope note that it pins the MCP client contract
rather than the commander wiring.
Also allowlists the Gemini 3.5 Flash test deletion as _deletedWithReplacement
(the model was retired by 2764812; gemini-models-parser.test.ts pins the new
"excluded from the parsed list" contract), and rebaselines bundleSize
8045 -> 8461 with per-entry measurements — every entrypoint stays far below its
absolute budget.
muhamadgalihsaputra
pushed a commit
to niyatna/NiyatnaRoute
that referenced
this pull request
Sep 27, 2026
…nto v3.8.51) (cheaper-inference#10952) Validated on the resolved merge against the current tip (585c2b7 + the post-cheaper-inference#11281 rebaseline): the single conflict was a comment-only collision in providers/[id]/models/route.ts (kept the tip's cheaper-inference#10828-ordering note). Focused suites 125/125 across all 13 touched test files (build-sqlite-stub, cc-compatible, copilot-claude-messages, copilot-gemini-route, executor-github, ghe-copilot, github-copilot-discovery-token, github-copilot-model-discovery, noauth-sibling-7620, provider-header-profiles, provider-models-config, request-log-payloads, upstream-error-passthrough), typecheck:core clean, file-size/changelog-integrity OK. Merged --admin over the inherited 2026-08-23 base-red cluster (cheaper-inference#9985) — the reds are proven tip failures (CLI catalog cluster + @testing-library allowlist, being drained by cheaper-inference#11280), not from this diff. Note: the rebase means several items the body listed (relay x-relay-path SSRF, /v1/search blocked-providers, cheaper-inference#10736 rotation fence, cheaper-inference#10903, cheaper-inference#10865, cheaper-inference#10899, cheaper-inference#10916) already landed upstream and are NOT in this delta — the delta is: better-sqlite3 build guard + build heap/worker caps + telemetry-off (cheaper-inference#10060 re-derived), credential-echo passthrough refusal + OCR/moderation redaction + call-log key redaction, Copilot CLI 1.0.81-6 wire identity + Claude→/v1/messages name-matched routing + discovery token fix, CC model_not_found 400, compat overrides for no-auth aliases (cheaper-inference#7620-pinned). The Copilot wire-identity change is the one to watch in production. Thank you @arminanton — and the ported-author credits in the commit history (@rqzbeh, yidecode, the cheaper-inference#10899/cheaper-inference#10916 authors) are preserved. Your config-posture finding (REQUIRE_API_KEY default vs 0.0.0.0) is noted for a maintainer decision, as you scoped it.
muhamadgalihsaputra
pushed a commit
to niyatna/NiyatnaRoute
that referenced
this pull request
Sep 27, 2026
…ference#9985) (cheaper-inference#11280) Validated on a 2-PR combined board: 158/158 across the drain's 17 focused test files (incl. integration qdrant-routes), typecheck:core + dashboard-typecheck clean, env-doc-sync, mutation-test-coverage, cli-i18n, file-size, changelog-integrity all green. Every red discriminated per-item as stale-test (contract intentionally moved, citing the origin PR) or real bug (fixed, never masked) across 3 base-advance waves (cheaper-inference#9985): stryker registration, i18n gaps (health.* namespace mismatch + pt-BR parity), combo-token-window pins (872K), CLI catalog counts (prime-agent), the isComboStep single-target reasoning-transport-fallback regression, container-guard hermeticity, and 26 unsuppressed ESLint errors from the wave (typed qdrant adapter, import trim, justified disable with precedent). Drains the accumulated base-red cluster.
muhamadgalihsaputra
pushed a commit
to niyatna/NiyatnaRoute
that referenced
this pull request
Sep 27, 2026
… a masked assert
Second base-red batch from the release pre-flight, measured on the .113 with a
clean npm ci (the devbox tree resolves eslint-plugin-react-hooks 7.1.1 from a
stray pnpm store instead of the lockfile 7.0.1 and reports 925 phantom errors).
Provider count 350 -> 352, one root cause behind three reds. Two providers
landed this cycle (volcengine-agent-plan, volcengine-coding-plan) without
regenerating the artifacts that quote the count:
- docs/reference/PROVIDER_REFERENCE.md regenerated (gen:provider-reference).
- README / AGENTS / llm.txt (+42 mirrors) / package.json description / 4 SVG
diagrams updated, including the section heading AND the anchor that links to
it, so the link does not break.
- tests/snapshots/provider/translate-path.json regenerated. The diff is purely
additive: 46 insertions, 0 deletions, exactly the two new providers.
GLM effort tiers. cheaper-inference#11415 added the explicit glm-5.3-max tier and left two
sibling vitest specs pinning the old 16-model inventory and an empty tier list
for it. Aligned to the shipped contract (inventory order matches glmProvider.ts;
glm-5.3-max declares ["max"]).
Test-masking. Four assert reductions surfaced once the deleted-file signal was
resolved. Three are legitimate and are allowlisted with their reasoning:
cheaper-inference#11355 inverted the startup-cooldown contract (preserve future quota cooldowns),
cheaper-inference#11280 replaced two unrolled hops with a 3-hop loop that asserts MORE, and the
Gemini 3.5 Flash retirement removed the models those capability asserts described.
The fourth was real masking: cheaper-inference#10960 rewrote the oneproxy status test to install a
stream mock, immediately overwrite it with a passthrough to the real fetch, and
assert `calls.length >= 0` — always true. Restored to assert what the test name
claims (the JSON-RPC tools/call carries omniroute_oneproxy_stats and its result
reaches the caller), with a scope note that it pins the MCP client contract
rather than the commander wiring.
Also allowlists the Gemini 3.5 Flash test deletion as _deletedWithReplacement
(the model was retired by bc825a1; gemini-models-parser.test.ts pins the new
"excluded from the parsed list" contract), and rebaselines bundleSize
8045 -> 8461 with per-entry measurements — every entrypoint stays far below its
absolute budget.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What
Drains the unit-shard and gate base-reds the 2026-08-23 merge wave left on
release/v3.8.50(tracked in #9985). Every red was discriminated as stale test/gate (the contract moved intentionally — test aligned, citing the PR) or real bug (fixed in code/messages — never masked). The base tip moved twice while this PR was open (17 merges total); reds that arrived with each wave were drained on the same branch.Wave 1 — original cluster (branch cut at 855243a)
check-deps 6A.8:@testing-library/dom,user-eventnot approveddomis a required peer of@testing-library/reactv16)search-route: zero-credential search returned 200, not 400duckduckgo-freefallback instead of 400models-catalog-combo-metadata: codex combomax_input_tokens500000 ≠ 272000min(872000, override))vscode-token-routes×3 +vscode-token-routes-gpt56+provider-models-route-codex(sibling sweep)cli-catalog-counts×3 +cli-tools-schema: 35 ≠ 34 / 9 ≠ 8 agentsprime-agentwithout the cardinality pinscli-catalog-display-contract:prime-agentmissing EN/VI descriptionscliTools.toolDescriptions.prime-agent(en/vi/pt-BR)setup-qwen×2,cli-setup-opencode×4,cli-tools-apply-opencode-jsonc: exit 2 on containersallowContainerWrite— hermeticsettings-i18n-keys: 7health.*keys without English messagessidebarbut the page readsuseTranslations("health"); page rendered raw keys in every localesidebar→healthin en/vi (+health.healthSubtitle)i18n-pt-br: 22 missing keys vs en.jsonAlso:
docs/reference/CLI-TOOLS.mdagent count 8→9.Wave 2 — base advanced to 527da65 (15 merges)
chatcore-translation-paths"Combo skip behavior"isComboStepderived fromcomboStepId/comboExecutionKeydegraded explicitreasoningTransportFallback: "skip"to "drop" for plain model-list combos (no step ids), against the PR's own stated intentisComboStep: isCombo || step-idsin chatCore; #10959 single-target drop defaults stay greencheck-db-rules-classification:recoveryzero importersprojectFileUrl()idiom the gate's regexes didn't recognizemutation-test-coveragegatequota-exhaustion-cutoff-opencode.test.tsWave 3 — base advanced to 1027682 (+#10952, +#11281)
check-env-doc-sync+ Docs Gates:OMNIROUTE_BUILDINGundocumented.env.example+ENVIRONMENT.mddb-core-init"build phase uses in-memory database"10060-build-sqlite-stub.test.ts8134-github-t5-fallback-filterfixtureclaude-opus-4-6-thinkingcli-tools+usage-service-hardeningCopilot identity pinsNo new ESLint warningsgate: 26 unsuppressed errorsas any×20), #11252 (unused imports vs #11247's same-day ratchet), #11251 (react-hooks/set-state-in-effect)asNextRequest()adapter; effort-tiers: import trim; modal: repo-convention justifiedeslint-disable-next-line(precedents: FreePoolTab, BatchConceptCard) — no new suppressionsEnv-only (verified, NOT fixed)
provider-request-failure-pipeline×6 — deterministic on the loaded devbox (persistedpipelinePayloadsnull within the 1.5s poll window under load 25-61); passed clean CI, confirming the env-only prediction.resolveConversationIdbounded-time (feat(backend): full request-body clones amplify long-context peak memory #7847) — reproduced identically at 12d0acb (the commit that introduced the bound, green on 08-20) on the same loaded box. Budget assertion untouched.Advisory (not blocking)
dast-smoke(continue-on-error: true) — the CLI-bundle build logs TS errors (@/shared/constants/codexClientresolution,src/lib/db/*rootDir/extensions under the mitm tsconfig) from the base wave; advisory gate, reported for visibility.Validation (RED → GREEN per file,
node --import tsx/esm --test)Every touched test file verified individually;
typecheck:coreclean;check:cli-i18nPASS; UI i18n coverage PASS (42 locales, vi 100%); env-doc-sync, mutation-test-coverage, docs-counts gates green locally; targeted eslint clean with frozen suppressions.Refs #9985