Repository navigation
fix(compression): Unicode-aware word boundaries in every rule pack + user overlay dir (#15677) - #15812
Merged
diegosouzapw merged 4 commits intoOct 10, 2026
Conversation
…ru-compression-boundary # Conflicts: # open-sse/services/compression/languageDetector.ts
…user overlay dir (#15677)
…in every mirror (#15677) Mechanical, language-neutral edit (one code-block line removed in the source and in all 66 mirrors); .i18n-state.json re-adopted for this doc only so the drift gate stays truthful.
Owner
Author
|
Merge integrity red is |
Contributor
CI Coverage Report
Coverage artifact was not available for this run. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Refs #15677 — follow-up to #15719 (depends on #15719, which fixed the ru packs; this branch is stacked on it).
Root cause
JS
\bis ASCII-only withoutu+\p{L}, so rules wrapped in\b...\bnever match when the keyword starts/ends with an accented or non-Latin letter. #15719 fixed ru; the same dead-boundary class existed in the de/es/fr/it/pt-BR/hu packs (120 rules with non-ASCII keywords). Also,docs/compression/EXTENDING_COMPRESSION.mddocumented a user pack dir~/.omniroute/compression/rulesthat the loader never read.Changes
\b->(?<![\p{L}\p{N}_])/(?![\p{L}\p{N}_])anduflag added. Bounded lookarounds, no new quantifiers (ReDoS-safe). ASCII-only rules untouched.languageDetector.tsuntouched (done in fix: use Unicode-aware word boundaries for non-ASCII language detection #15708).ruleLoader.ts: user overlay dir$DATA_DIR/compression/rules(default~/.omniroute): same-name rules override bundled ones, new names are appended, invalid user files are ignored with a warning. Docs corrected (dropped the never-implemented project-level path).Tests
tests/unit/compression/rule-packs-unicode-boundary-15677.test.ts(new): invariant that no non-ASCII rule uses\b, that every literal keyword of every pack matches, plus overlay tests using a temp DATA_DIR.RED on the unfixed packs:
de/de_question_directive still uses \band keyword-match invariant failing. GREEN after: 5/5. Existing compression/caveman/language-pack tests: 121 pass, 0 fail.Gates
file-size, complexity, cognitive-complexity, eslint (changed files), prettier, typecheck:core, check:compression-budget all green; pre-commit hooks ran.
check:docs-allreports only pre-existing unrelated drift (migration counts in README/AGENTS/llm.txt, readCodeFacts); the edited doc is not flagged. i18n mirrors of EXTENDING_COMPRESSION.md not regenerated.