Skip to content

feat(rankings): expose provider reliability alongside free provider rankings - #10909

Merged
diegosouzapw merged 1 commit into
diegosouzapw:release/v3.8.50from
maxmad64bis:feat/free-provider-rankings-reliability
Aug 21, 2026
Merged

diegosouzapw merged 1 commit into
diegosouzapw:release/v3.8.50from
maxmad64bis:feat/free-provider-rankings-reliability

Conversation

@maxmad64bis

@maxmad64bis maxmad64bis commented Aug 20, 2026 •

Copy link
Copy Markdown
Contributor

⚠️ base-red inherited: #9985

Summary

src/lib/freeProviderRankings.ts ranks free providers by ELO score only. It already loads connection state (getProviderConnections) but uses it to filter, never to qualify — a provider the operator cannot use right now still ranks first, with nothing in the payload saying so.

This PR exposes a second, additive dimension: each ranking gets a reliability field carrying the raw per-connection signals (testStatus, rateLimitedUntil) and the state they imply. The sort and the scores are untouched.

Two constraints shape the field.

It reuses the project's health vocabulary rather than inventing one. reliability.state is a ProviderHealthState — the healthy | degraded | down type already defined in src/lib/monitoring/providerHealthMatrix.ts — with the same split that module applies: a terminal testStatus is down, a live cooldown is degraded. The provider-level aggregate mirrors classifyProvider (all connections down ⇒ down, any non-healthy ⇒ degraded) minus its circuit-breaker input, which this code path does not load. The import is type-only, so nothing is pulled in at runtime. A boolean flag here would have meant the same word describing two different things in two APIs of the same product.

It exposes the raw signal next to the state, never instead of it. testStatus is written on failure paths at request time (open-sse/handlers/chatCore.ts marks expired, banned, deactivated, unavailable, credits_exhausted) and is reset to active only by an explicit connection test or a re-auth — so it can outlive the actual recovery. Consumers get the stored value verbatim plus the dated rateLimitedUntil, and can disagree with the state if they have better information.

No extra query: the field is attached to the connection snapshot the filters already load, so it exists when configuredOnly or availableOnly is set and is absent otherwise.

Related Issues

Validation

  • Change type: other (rankings API field)
  • Focused tests and category gates from the golden path
  • npm run lint
  • Reconciled with the current active release base; focused checks rerun afterward — the branch is cut from 3d7ed7aa8 and has not been merged forward. Six commits have landed on release/v3.8.50 since (142ae9349..eb6f31971); none touches src/lib/freeProviderRankings.ts, src/lib/monitoring/providerHealthMatrix.ts or this PR's changelog fragment, and the branch merges cleanly. git merge-tree reports no conflict against the current tip; the branch can be merged forward on request before review.
  • Production-code changes include a new or updated automated test in this PR

Tests Added Or Updated

  • tests/unit/freeProviderRankings-filters.test.ts — eight cases for attachProviderReliability: healthy connections, a terminal status classified down (not degraded), a future rateLimitedUntil degrading while a past one does not, a mixed provider (one down + one healthy) aggregating to degraded, an all-down provider aggregating to down, raw testStatus returned verbatim including its original casing, providers without a connection getting no field, and inputs never mutated.

Run: node --import tsx/esm --test tests/unit/freeProviderRankings-filters.test.ts

# pass 19
# fail 0

Also run on the branch: npm run typecheck:core (clean), npm run lint (clean), npm run check:cycles (no cycles).

Coverage Notes

The new pure functions are covered by the cases above, against the same FIXED_NOW snapshot the filter tests use. isProviderUsable is now expressed in terms of the same per-connection classifier, so the filter and the reported state cannot drift apart; its existing tests cover that refactor unchanged. The wiring in computeFreeProviderRankings is one line inside the existing filter block, on the already-loaded snapshot. No mocks, no DB.

Reviewer Notes

  • Additive only: the field is optional and absent when filters are off. The API route and the dashboard page compile and behave unchanged.
  • The ranking order is byte-identical to today's for every input: no line of the sort or the score math is touched.
  • Interaction with availableOnly, worth knowing when reading the payload: that filter already drops every provider without a healthy connection, so under it state can only be healthy or degraded (a partially rate-limited multi-connection provider) — never down. down is reachable with configuredOnly alone.
  • Consumer-side rendering is intentionally out of scope: this PR ships the data, and the rankings page is left untouched. Surfacing it in the UI would add i18n work across the message catalogs and is better decided by the maintainers.
  • No migration, no feature flag.

@maxmad64bis
maxmad64bis force-pushed the feat/free-provider-rankings-reliability branch 2 times, most recently from f0d1277 to a56b9ff Compare August 20, 2026 23:44
…ankings

`computeFreeProviderRankings` already loads connection state for the
configured/available filters (diegosouzapw#6150) but only used it to drop rows. Attach that
same snapshot to each surviving ranking as an additive `reliability` field: raw
per-connection `testStatus`/`rateLimitedUntil` plus the state they imply. No
extra query, no change to the sort or the scores.

States reuse `ProviderHealthState` from providerHealthMatrix (type-only import)
with the same split `classifyAccount` applies, and `isProviderUsable` now sits on
the shared classifier so the filter and the reported state cannot drift.

Validation: 19/19 in freeProviderRankings-filters.test.ts, lint, typecheck:core,
check:cycles, changelog-integrity.
@maxmad64bis
maxmad64bis force-pushed the feat/free-provider-rankings-reliability branch from a56b9ff to 2f25e0f Compare August 20, 2026 23:49
@diegosouzapw
diegosouzapw merged commit 4c0b54a into diegosouzapw:release/v3.8.50 Aug 21, 2026
5 checks passed
diegosouzapw pushed a commit that referenced this pull request Aug 21, 2026
Validado no worktree combinado: mesmos gates + testes focados verdes. Extensão opt-in bem desenhada sobre #10909 (dimensão de uso real via call_logs). CI vermelho é o base-red já rastreado em #9985.
@maxmad64bis
maxmad64bis deleted the feat/free-provider-rankings-reliability branch September 24, 2026 21:11
muhamadgalihsaputra pushed a commit to niyatna/NiyatnaRoute that referenced this pull request Sep 27, 2026
…ankings (diegosouzapw#10909)

Validado no worktree combinado: typecheck:core, changelog-integrity, complexity, cognitive-complexity, file-size, lint e testes focados (freeProviderRankings-filters) todos verdes. Feature aditiva bem documentada (campo reliability nos rankings). CI vermelho é o base-red já rastreado em diegosouzapw#9985. Obrigado!
muhamadgalihsaputra pushed a commit to niyatna/NiyatnaRoute that referenced this pull request Sep 27, 2026
…zapw#10926)

Validado no worktree combinado: mesmos gates + testes focados verdes. Extensão opt-in bem desenhada sobre diegosouzapw#10909 (dimensão de uso real via call_logs). CI vermelho é o base-red já rastreado em diegosouzapw#9985.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants