Skip to content

feat(compression): per-model/endpoint compression exclusion filter (#8034) - #8064

Merged
diegosouzapw merged 3 commits into
release/v3.8.49from
feat/8034-compression-exclusions
Jul 22, 2026
Merged

diegosouzapw merged 3 commits into
release/v3.8.49from
feat/8034-compression-exclusions

Conversation

@diegosouzapw

Copy link
Copy Markdown
Collaborator

Summary

  • Adds an operator-configurable exclusion list (model ids / provider/model patterns, * wildcard) that bypasses the whole compression pipeline for matching chat-completion requests — the body passes through byte-identical, no engine runs.
  • open-sse/services/compression/exclusions.ts (new leaf module): normalizeCompressionExclusions() + isCompressionExcluded(). Patterns are matched case-insensitively against both the bare model id and the provider/model composite; every regex metacharacter except * is escaped before compiling (ReDoS-safe, bounded, ≤200 patterns).
  • CompressionConfig.exclusions?: string[] (new optional field, absent = no exclusions = zero behavior change for existing installs).
  • src/lib/db/compression.ts reads/persists the exclusions key through the existing key_value compression namespace — no new table/migration.
  • chatCore.ts computes the exclusion right after resolveCompressionSettings() and treats a match exactly like compression being globally disabled: the modular pipeline is skipped (config.enabled = false for this request) and the legacy auto-trigger path is skipped too (promptCompressionEnabled = false), recording the skip via the existing writeCompressionSkip(..., "excluded") analytics hook. ~10 lines of real logic in the frozen chatCore.ts (4728→4761 lines, well under the 5125 baseline); all matching logic lives in the new leaf module.
  • New dashboard tab: Dashboard → Compression → Exclusions (/dashboard/compression/exclusions), a simple textarea-backed panel reading/writing the exclusions field via the existing GET/PUT /api/settings/compression endpoint. Added to the sidebar nav and to the SidebarItemId union.
  • Zod schema: exclusions added to compressionSettingsUpdateSchema (bounded array/string length).
  • Docs: new "Scope & exclusions" section in docs/compression/COMPRESSION_ENGINES.md, also documenting that embeddings are structurally never compressed (unrelated to this change, but the guardrail's stated rationale).
  • i18n: new keys added to en.json, mirrored/translated into pt-BR.json and vi.json (sidebar nav + settings-panel copy).

Test plan

  • node --import tsx/esm --test tests/unit/compression-exclusions.test.ts — 8/8 passing: excluded/non-excluded matching, provider/model composite + bare-model patterns, provider/* and bare * wildcards, case-insensitivity, empty/absent/malformed exclusions default to false (regression guard for zero behavior change), regex-metacharacter escaping (gpt-5.6 must not match gpt-5x6), normalizeCompressionExclusions dedup/non-string filtering, and a behavior test proving an excluded target's body stays byte-identical while a non-excluded target still compresses via applyStackedCompression.
  • npx vitest run tests/unit/ui/compression-exclusions-page.test.tsx — 3/3 passing: mounts and loads existing exclusions, saves edited patterns via PUT /api/settings/compression, fails soft when fetch rejects.
  • node scripts/check/check-file-size.mjs — clean (added one justified frozen-baseline entry for sidebarVisibility/sections.ts, 796→806, one new nav item pushed it over the new-file 800 cap).
  • npm run check:complexity-ratchets — clean (both complexity and cognitive-complexity under baseline).
  • node scripts/check/check-test-discovery.mjs — clean.
  • npm run typecheck:core — clean.
  • npm run typecheck:noimplicit:core — clean (pre-existing unrelated errors in combo.ts/usageTracking.ts/cliRuntime.ts, untouched by this PR, confirmed via git diff against the base branch).
  • node scripts/check/check-dashboard-typecheck.mjs — clean for all files touched by this PR; the gate also flags one pre-existing src/lib/services/installers/utils.ts TS2339 error unrelated to compression (confirmed zero diff against origin/release/v3.8.49 for that file).
  • npx eslint --suppressions-location config/quality/eslint-suppressions.json <changed files> — clean.
  • node scripts/check/check-mutation-test-coverage.mjs --strict — no drift.
  • npm run check:cycles — no cycles.
  • node scripts/check/check-docs-sync.mjs / check:any-budget:t11 / check:tracked-artifacts — all clean (pre-commit hook passed).

Closes #8034

@gemini-code-assist

Copy link
Copy Markdown
Contributor

Caution

The consumer version of Gemini Code Assist on GitHub has been sunset. All code review activity has officially ceased.

@diegosouzapw

Copy link
Copy Markdown
Collaborator Author

Reviewed this against the current release/v3.8.49 tip in an isolated probe worktree.

Content is solid: the exclusion filter is scoped tightly (new leaf module, ReDoS-safe pattern escaping, bounded to 200 patterns), the chatCore.ts wiring is minimal and mirrors the existing compression-disabled path, settings persist through the existing key_value namespace (no new table), and the dashboard tab reuses the existing authenticated GET/PUT /api/settings/compression endpoint. I ran both test suites the PR ships — tests/unit/compression-exclusions.test.ts (8/8) and tests/unit/ui/compression-exclusions-page.test.tsx (3/3) — both green, plus typecheck:core and eslint on every touched file, both clean.

One thing to fix before merge: GitHub shows mergeable: CONFLICTING. I confirmed it's a single trivial conflict in src/i18n/messages/pt-BR.json — this PR's compressionExclusions/compressionExclusionsSubtitle keys landed at the same insertion point as another already-merged PR's *Subtitle keys. Everything else (chatCore.ts, types.ts, file-size-baseline.json, compression.ts, compressionConfigSchemas.ts, en.json, vi.json) auto-merges cleanly. I'll rebase and resolve it (keep both key blocks) as part of the merge pass.

The two other local gate flags I saw (test-file-size on providers-page-utils.test.ts, dashboard-typecheck TS2339 on installers/utils.ts) are pre-existing tip drift already fixed by #8055 after this branch was cut — this PR doesn't touch either file, so both resolve themselves on rebase, matching what the PR body already notes.

Probe Test and others added 2 commits July 22, 2026 16:09
@diegosouzapw
diegosouzapw merged commit dbdc7da into release/v3.8.49 Jul 22, 2026
6 of 9 checks passed
@diegosouzapw

Copy link
Copy Markdown
Collaborator Author

Obrigado, @diegosouzapw — mergeado na release/v3.8.49 via admin-merge. Resolvi o conflito trivial em pt-BR.json (mantidos ambos os blocos de chaves), rebaseline own-growth de compression.ts 845→850 (com marcador justificado). Validação no tip combinado: 8/8 node:test + vitest 271/271; typecheck/file-size/changelog verdes. Único red de CI era o base-red compartilhado de complexity (2152, idêntico à tip pura).

@diegosouzapw
diegosouzapw deleted the feat/8034-compression-exclusions branch July 23, 2026 00:24
fenix007 pushed a commit to fenix007/OmniRoute that referenced this pull request Jul 23, 2026
…e was added

Base-red slice 5. cheaper-inference#8034/cheaper-inference#8064 (per-model/endpoint compression exclusion filter)
appended the new `compression-exclusions` sidebar item to the END of the compression
group, after `compression-studio`. That broke the deliberate "Compression Studio is
the last item of the compression group" invariant asserted by two tests
(sidebar-engine-items "Studio must be last" and sidebar-visibility's exact-order array,
whose expected list places compression-studio immediately before cli-code).

Rather than weaken the invariant, moved `compression-exclusions` ABOVE
`compression-studio` in sections.ts (new pages go above the Studio showcase) and added
the new item to sidebar-visibility's expected order. Both tests, plus the other four
sidebar suites (reorg / back-compat / tools / engine-items), stay green.
diegosouzapw added a commit that referenced this pull request Jul 24, 2026
…bar order snapshots

Two independent "code is right, bookkeeping lagged" base-reds:

1. #8233 made open-sse/executors/muse-spark-web.ts import
   sanitizeErrorMessage from utils/error.ts (a real Rule #12 fix), but
   left its KNOWN_MISSING_ERROR_HELPER allowlist entry in
   scripts/check/check-error-helper.mjs in place. The gate's own
   stale-allowlist enforcement (assertNoStale) correctly flagged the now
   -obsolete entry: `npm run check:error-helper` failed with "1 entrada(s)
   obsoleta(s)", and tests/unit/check-error-helper.test.ts's "the shipped
   allowlist freezes exactly the known current violators" test expected
   an empty Set. Removed the entry (kept the assertNoStale machinery and
   the general scope-header comments untouched).

2. #8064 added the "compression-exclusions" sidebar item right after
   "compression-studio" in COMPRESSION_CONTEXT_GROUP (deliberate,
   complete feature) but didn't update two order-snapshot tests written
   before that item existed:
   - tests/unit/sidebar-visibility.test.ts expected the "omni-proxy"
     section's flattened id list to end the compression block at
     "compression-studio".
   - tests/unit/ui/sidebar-engine-items.test.ts asserted "Studio must be
     last" in COMPRESSION_CONTEXT_GROUP.
   Updated both to the real, intentional order: Settings -> Combos ->
   engines -> Studio -> Exclusions (Studio now second-to-last,
   Exclusions last).

Validation (red -> green):
- check:error-helper gate: red ("1 entrada(s) obsoleta(s)") -> green
  ("OK (898 files scanned, 0 known-missing frozen)")
- tests/unit/check-error-helper.test.ts: 31/32 -> 32/32
- tests/unit/sidebar-visibility.test.ts: 6/7 -> 7/7
- tests/unit/ui/sidebar-engine-items.test.ts: 13/14 -> 14/14

Refs #8233
Refs #8064
diegosouzapw added a commit that referenced this pull request Jul 24, 2026
…bar order snapshots (#8383)

Two independent "code is right, bookkeeping lagged" base-reds:

1. #8233 made open-sse/executors/muse-spark-web.ts import
   sanitizeErrorMessage from utils/error.ts (a real Rule #12 fix), but
   left its KNOWN_MISSING_ERROR_HELPER allowlist entry in
   scripts/check/check-error-helper.mjs in place. The gate's own
   stale-allowlist enforcement (assertNoStale) correctly flagged the now
   -obsolete entry: `npm run check:error-helper` failed with "1 entrada(s)
   obsoleta(s)", and tests/unit/check-error-helper.test.ts's "the shipped
   allowlist freezes exactly the known current violators" test expected
   an empty Set. Removed the entry (kept the assertNoStale machinery and
   the general scope-header comments untouched).

2. #8064 added the "compression-exclusions" sidebar item right after
   "compression-studio" in COMPRESSION_CONTEXT_GROUP (deliberate,
   complete feature) but didn't update two order-snapshot tests written
   before that item existed:
   - tests/unit/sidebar-visibility.test.ts expected the "omni-proxy"
     section's flattened id list to end the compression block at
     "compression-studio".
   - tests/unit/ui/sidebar-engine-items.test.ts asserted "Studio must be
     last" in COMPRESSION_CONTEXT_GROUP.
   Updated both to the real, intentional order: Settings -> Combos ->
   engines -> Studio -> Exclusions (Studio now second-to-last,
   Exclusions last).

Validation (red -> green):
- check:error-helper gate: red ("1 entrada(s) obsoleta(s)") -> green
  ("OK (898 files scanned, 0 known-missing frozen)")
- tests/unit/check-error-helper.test.ts: 31/32 -> 32/32
- tests/unit/sidebar-visibility.test.ts: 6/7 -> 7/7
- tests/unit/ui/sidebar-engine-items.test.ts: 13/14 -> 14/14

Refs #8233
Refs #8064
HouMinXi pushed a commit to HouMinXi/OmniRoute that referenced this pull request Aug 2, 2026
…heaper-inference#8034) (cheaper-inference#8064)

* feat(compression): per-model/endpoint compression exclusion filter (cheaper-inference#8034)

* chore(quality): rebaseline compression.ts own-growth 845->850 (cheaper-inference#8034 exclusions persistence)

---------

Co-authored-by: Probe Test <probe@example.com>
HouMinXi pushed a commit to HouMinXi/OmniRoute that referenced this pull request Aug 2, 2026
…bar order snapshots (cheaper-inference#8383)

Two independent "code is right, bookkeeping lagged" base-reds:

1. cheaper-inference#8233 made open-sse/executors/muse-spark-web.ts import
   sanitizeErrorMessage from utils/error.ts (a real Rule cheaper-inference#12 fix), but
   left its KNOWN_MISSING_ERROR_HELPER allowlist entry in
   scripts/check/check-error-helper.mjs in place. The gate's own
   stale-allowlist enforcement (assertNoStale) correctly flagged the now
   -obsolete entry: `npm run check:error-helper` failed with "1 entrada(s)
   obsoleta(s)", and tests/unit/check-error-helper.test.ts's "the shipped
   allowlist freezes exactly the known current violators" test expected
   an empty Set. Removed the entry (kept the assertNoStale machinery and
   the general scope-header comments untouched).

2. cheaper-inference#8064 added the "compression-exclusions" sidebar item right after
   "compression-studio" in COMPRESSION_CONTEXT_GROUP (deliberate,
   complete feature) but didn't update two order-snapshot tests written
   before that item existed:
   - tests/unit/sidebar-visibility.test.ts expected the "omni-proxy"
     section's flattened id list to end the compression block at
     "compression-studio".
   - tests/unit/ui/sidebar-engine-items.test.ts asserted "Studio must be
     last" in COMPRESSION_CONTEXT_GROUP.
   Updated both to the real, intentional order: Settings -> Combos ->
   engines -> Studio -> Exclusions (Studio now second-to-last,
   Exclusions last).

Validation (red -> green):
- check:error-helper gate: red ("1 entrada(s) obsoleta(s)") -> green
  ("OK (898 files scanned, 0 known-missing frozen)")
- tests/unit/check-error-helper.test.ts: 31/32 -> 32/32
- tests/unit/sidebar-visibility.test.ts: 6/7 -> 7/7
- tests/unit/ui/sidebar-engine-items.test.ts: 13/14 -> 14/14

Refs cheaper-inference#8233
Refs cheaper-inference#8064
muhamadgalihsaputra pushed a commit to niyatna/NiyatnaRoute that referenced this pull request Sep 27, 2026
…heaper-inference#8034) (cheaper-inference#8064)

* feat(compression): per-model/endpoint compression exclusion filter (cheaper-inference#8034)

* chore(quality): rebaseline compression.ts own-growth 845->850 (cheaper-inference#8034 exclusions persistence)

---------

Co-authored-by: Probe Test <probe@example.com>
muhamadgalihsaputra pushed a commit to niyatna/NiyatnaRoute that referenced this pull request Sep 27, 2026
…bar order snapshots (cheaper-inference#8383)

Two independent "code is right, bookkeeping lagged" base-reds:

1. cheaper-inference#8233 made open-sse/executors/muse-spark-web.ts import
   sanitizeErrorMessage from utils/error.ts (a real Rule cheaper-inference#12 fix), but
   left its KNOWN_MISSING_ERROR_HELPER allowlist entry in
   scripts/check/check-error-helper.mjs in place. The gate's own
   stale-allowlist enforcement (assertNoStale) correctly flagged the now
   -obsolete entry: `npm run check:error-helper` failed with "1 entrada(s)
   obsoleta(s)", and tests/unit/check-error-helper.test.ts's "the shipped
   allowlist freezes exactly the known current violators" test expected
   an empty Set. Removed the entry (kept the assertNoStale machinery and
   the general scope-header comments untouched).

2. cheaper-inference#8064 added the "compression-exclusions" sidebar item right after
   "compression-studio" in COMPRESSION_CONTEXT_GROUP (deliberate,
   complete feature) but didn't update two order-snapshot tests written
   before that item existed:
   - tests/unit/sidebar-visibility.test.ts expected the "omni-proxy"
     section's flattened id list to end the compression block at
     "compression-studio".
   - tests/unit/ui/sidebar-engine-items.test.ts asserted "Studio must be
     last" in COMPRESSION_CONTEXT_GROUP.
   Updated both to the real, intentional order: Settings -> Combos ->
   engines -> Studio -> Exclusions (Studio now second-to-last,
   Exclusions last).

Validation (red -> green):
- check:error-helper gate: red ("1 entrada(s) obsoleta(s)") -> green
  ("OK (898 files scanned, 0 known-missing frozen)")
- tests/unit/check-error-helper.test.ts: 31/32 -> 32/32
- tests/unit/sidebar-visibility.test.ts: 6/7 -> 7/7
- tests/unit/ui/sidebar-engine-items.test.ts: 13/14 -> 14/14

Refs cheaper-inference#8233
Refs cheaper-inference#8064
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

feat(compression): per-model/endpoint compression exclusion filter (embeddings already excluded structurally)

1 participant