Skip to content

fix(sse): limit exploitation to equivalent candidates - #11420

Closed
jacobsparts wants to merge 2 commits into
diegosouzapw:release/v3.8.50from
jacobsparts:fix/auto-equivalent-rotation
Closed

jacobsparts wants to merge 2 commits into
diegosouzapw:release/v3.8.50from
jacobsparts:fix/auto-equivalent-rotation

Conversation

@jacobsparts

Copy link
Copy Markdown
Contributor

Problem

With exploration disabled, Auto-Combo could still rotate from the highest-scored candidate into a materially lower-scored tier. In simple terms, the scoring engine identified the better target and ordinary exploitation then discarded that result.

This is a narrower replacement for #11406. It does not remove ScoreTierRotator or concentrate all traffic on one connection. Candidates whose normalized scores are within one point (0.01) of the winner remain in the rotation pool, preserving load balancing across equivalent providers and connection IDs.

Summary

  • restrict non-exploratory rotation to candidates within 0.01 of the best score
  • preserve round-robin across tied and near-equivalent connections
  • leave explicit exploration able to select lower-scored candidates
  • leave budget-cap fallback behavior unchanged

The boundary is evidence-based in the regression fixtures: the existing intended smart rotation spans a 0.005621 score gap, while the reproduced incorrect lower-tier choice has a 0.027147 gap.

Tests

  • Regression failed against unmodified release/v3.8.50
  • npx vitest run --config vitest.mcp.config.ts tests/unit/autoCombo/tieredRotation.test.ts — 12 passed
  • node --import tsx/esm --import ./open-sse/utils/setupPolyfill.ts --import ./tests/_setup/isolateDataDir.ts --test --test-force-exit tests/unit/auto-combo-engine.test.ts — 4 passed
  • focused suppression-aware ESLint — passed
  • git diff --check — passed
  • npm run check:known-symbols — passed

Repository-wide lint is currently blocked by inherited base-red issue #9985; the changed files pass the repository's suppression-aware focused lint.

Scope

Production code changes are limited to open-sse/services/autoCombo/engine.ts; the focused regression is in tests/unit/autoCombo/tieredRotation.test.ts.

⚠️ base-red inherited: #9985

Normal exploitation could rotate into a materially lower-scored tier even when exploration was disabled. Restrict its rotation pool to candidates within one normalized score point of the winner, preserving load balancing across equivalent connections while leaving explicit exploration and budget fallback unchanged.
@diegosouzapw

Copy link
Copy Markdown
Owner

Obrigado por revisar após o #11406 — esta versão é bem mais cuidadosa e resolve minha objeção principal: o round-robin por conexão (ScoreTierRotator.advance) é preservado, então não concentra tráfego numa única conexão. Ainda assim, vamos fechar este PR também.

O motivo: TIER_PREFERENCES (definido junto do ScoreTierRotator no #3078) faz combos nomeados smart/fast/cheap/coding ponderarem deliberadamente entre os tiers top/mid/rest — por exemplo cheap: { top: 0.2, mid: 0.3, rest: 0.5 } prefere pegar do tier mais barato mesmo com score mais baixo, mesmo fora de exploração. Restringir rotator.pick() a equivalentCandidates (score dentro de 0.01 do melhor) faz esse peso nunca se aplicar na prática, porque o gap de score entre tiers normalmente é bem maior que 0.01 — exatamente o que o teste removido ("cheap combo pulls from rest tier more often than smart") comprovava.

Se o objetivo é impedir que a exploração normal escolha um candidato "materialmente pior" quando os scores estão bem próximos, talvez o ajuste certo seja no CLEAR_WINNER_THRESHOLD/lógica de tiers em vez de bypassar o tierPreferencesForName inteiro. Fico à disposição para discutir uma versão que preserve os dois mecanismos.

@jacobsparts

Copy link
Copy Markdown
Contributor Author

Thanks for the feedback and for walking through the tier preference behavior.

The suggested direction sounds right, will give it another shot with that in mind.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants