Skip to content

fix(compression): avoid regex for repeated literal blocks - #14633

Merged
diegosouzapw merged 2 commits into
diegosouzapw:release/v3.8.51from
Jehu:fix/session-dedup-literal-scan
Sep 24, 2026
Merged

diegosouzapw merged 2 commits into
diegosouzapw:release/v3.8.51from
Jehu:fix/session-dedup-literal-scan

Conversation

@Jehu

@Jehu Jehu commented Sep 23, 2026

Copy link
Copy Markdown
Contributor

Summary

The session-dedup compressor now counts repeated text blocks with literal string scanning instead of compiling each block as a regular expression. Large memory or JSON-like blocks can no longer trip V8's Regular expression too large limit while the existing dedup behavior stays intact.

Test plan

  • DISABLE_SQLITE_AUTO_BACKUP=true node --max-old-space-size=8192 --import tsx/esm --import ./open-sse/utils/setupPolyfill.ts --import ./tests/_setup/isolateDataDir.ts --test --test-force-exit tests/unit/compression/session-dedup.test.ts tests/unit/compression/session-dedup-memory-7849.test.ts tests/unit/compression/session-dedup-intra-message-6467.test.ts

@Jehu
Jehu requested a review from diegosouzapw as a code owner September 23, 2026 12:08
@diegosouzapw

Copy link
Copy Markdown
Owner

Nice fix — this lines up with the project's existing regex-size guardrails, and the guarded-RegExp test (asserting no RegExp is ever compiled from the block) is a clean way to prove it structurally. Before merging, could you add a changelog fragment under changelog.d/fixes/ referencing #6467? Everything else looks solid — the literal-scan replacement functions correctly preserve the "keep the first occurrence" behavior of the original regex-based code.

@Jehu

Jehu commented Sep 24, 2026

Copy link
Copy Markdown
Contributor Author

could you add a changelog fragment under changelog.d/fixes/ referencing #6467?

Added in changelog.d/fixes/14633-session-dedup-literal-scan.md and pushed in b9e5dda.

@diegosouzapw
diegosouzapw merged commit 8220434 into diegosouzapw:release/v3.8.51 Sep 24, 2026
3 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants