Skip to content

feat(combos): opt-in auto-prune of stale combo steps after a model sync - #15925

Merged
diegosouzapw merged 5 commits into
cheaper-inference:release/v3.8.52from
tiagovilasboas:feat/13505-combo-stale-auto-prune
Oct 10, 2026
Merged

diegosouzapw merged 5 commits into
cheaper-inference:release/v3.8.52from
tiagovilasboas:feat/13505-combo-stale-auto-prune

Conversation

@tiagovilasboas

Copy link
Copy Markdown
Contributor

Summary

  • Adds an opt-in auto-prune for the stale combo steps that feat(combos): flag combo steps pinned to models missing from synced catalog #15800 flags. It is controlled by a new feature flag, COMBO_AUTO_PRUNE_STALE_STEPS (runtime, boolean, default false).
  • OFF (default): nothing changes from feat(combos): flag combo steps pinned to models missing from synced catalog #15800. The sync only flags.
  • ON: after a successful model sync, the steps findStaleComboModelRefs flagged are removed from their combos. Each removed step writes one combo.stale_model_ref.pruned audit entry, with the combo as target and { connectionId, provider, comboId, comboName, stepId, model } as details. The sync response adds prunedComboRefs, and staleComboRefs keeps only what is still flagged.
  • Never prunes when:
    • the sync failed or was degraded (those paths return before the check);
    • the provider has no authoritative catalog (the helper returns nothing);
    • the import was free-only (importFreeModelsOnly / freeFilterEmpty), because that catalog is a deliberate subset and doesn't reflect what the provider actually serves.
  • A combo is never emptied. If every step is stale, the combo is left as is and its steps stay flagged.

Related Issues

Validation

  • Change type: routing (combo maintenance on model sync)
  • Focused tests and category gates: tests/unit/combo-stale-auto-prune-13505.test.ts (4/4), plus the impacted set from the test-impact map and every feature-flag test: 42 files, 267/267
  • npm run lint scope: eslint + prettier clean on all touched files. check-api-typecheck OK (within the frozen baseline), and no new type errors in the touched files
  • Docs gates: check-env-doc-sync, check-fabricated-docs, check-docs-sync and feature-flags-doc-sync-static pass
  • Reconciled with the current active release base (release/v3.8.52 @ f9989a14)
  • Production-code changes include new automated tests in this PR

Tests Added Or Updated

  • tests/unit/combo-stale-auto-prune-13505.test.ts (new). It uses the real route, DB and audit log, with a mocked upstream:
    • OFF: the stale steps are flagged, prunedComboRefs is empty, combos are unchanged and there is no prune audit entry;
    • ON: the stale steps are removed from two combos while their valid steps are kept, there is one audit entry per removed step, and a combo whose only step is stale is left as is and stays in staleComboRefs;
    • ON with a failed (503) or degraded (local_catalog) sync: nothing is removed and nothing is audited.
  • tests/unit/feature-flags-settings.test.ts, tests/unit/server-owned-tool-loop-flag.test.ts: flag count pins 84 to 85.

Coverage Notes

  • src/lib/combos/staleModelPrune.ts (new) and the sync route hook are covered by the new test file end to end.
  • test:scoped: featureFlagDefinitions.ts maps to about 4.2k test files and the new staleModelPrune.ts is unmapped (__RUN_ALL__). Locally I ran the impacted set for the other changed files plus all *flag* unit tests (42 files). The full suite runs in CI.

Reviewer Notes

  • Where the setting lives: this is a feature flag, like the other opt-in side effects of a model sync (OMNIROUTE_AUTO_SYNC_CODEX_PROFILES / ..._CLAUDE_PROFILES). It shows up on the existing Feature Flags page, toggles without a restart, and can be set from env. So there is no new settings UI.
  • i18n: none. The Feature Flags grid renders featureFlags.definitions.<KEY> when it exists and otherwise falls back to the English definition text, as it does for about 39 other flags. The root featureFlag*Description keys are not rendered by the dashboard (see the note in feature-flag-descriptions-claims.test.ts), so adding one would only add untranslated weight to 66 locale files.
  • Independent of the editor badge PR (feat(combos): badge combo editor steps whose model left the live catalog #15923): the two PRs touch different code paths, with no shared new files and no overlapping hunks (this PR only exports isExplicitModelStep and updates the staleModelRefs.ts header comment). Either can merge first.
  • LOC: about 269 hand-written lines added (169 of them tests) and 12 removed. No locale or generated files.

Opt-in runtime flag, default false, for removing the stale combo steps a
model sync flags. Documented in the feature flag catalog; the flag count
pins move from 84 to 85.
…abled

With COMBO_AUTO_PRUNE_STALE_STEPS on, a successful sync removes the steps
findStaleComboModelRefs flagged and writes one combo.stale_model_ref.pruned
audit entry per removed step. The response adds prunedComboRefs, and
staleComboRefs keeps only what is still flagged.

Failed and degraded syncs return before the check, a provider without an
authoritative catalog flags nothing, and a free-only import never prunes
because its catalog is a deliberate subset. A combo is never emptied: if
every step is stale it stays as is and stays flagged. With the flag off
the sync only flags, as before.
tiagovilasboas and others added 2 commits October 8, 2026 19:18
The feature-flag page matches the hash of the text this PR adds.

Co-authored-by: diegosouzapw <8016841+diegosouzapw@users.noreply.github.com>
@diegosouzapw
diegosouzapw merged commit b65b298 into cheaper-inference:release/v3.8.52 Oct 10, 2026
19 of 41 checks passed
@diegosouzapw

Copy link
Copy Markdown
Collaborator

Thanks @tiagovilasboas — this is in the upcoming release.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants