feat(compression): compression engines + async pipeline + Combo/Compression Studios

[diegosouzapw]

Compression engines + Studios — engine layer, async pipeline, two ReactFlow dashboards

Implements the coherent "compression engines (stackable into combos) + the two real-time
Studios" track, end to end, in 20 TDD commits. Every item was built RED→GREEN, audited
(diff + scope + re-run + anti-masking) before acceptance, and closed with a 2-round deep
code-review cycle that converged clean.

What's here

Foundation

• applyAsync? on CompressionEngine + applyStackedCompressionAsync/applyCompressionAsync
  (H10) — async-capable stacked pipeline; chatCore awaits it. Sync-only pipelines stay
  byte-identical.
• Shared FlowCanvas/edgeStyles/StatusDot extracted from ProviderTopology (behaviour-
  preserving), reused by both Studios.
• API-free eval/benchmark harness: token ratio + semantic retention (reuses preservation.ts),
  tokens-per-task gate (N4), transcript replay.

Deterministic engines (opt-in, stackable)

• Real JS/TS comment removal via the TypeScript parser (string/template/regex safe, bails on JSX).
• Matched-filter preserve.errorPatterns/summaryPatterns propagated into smartTruncate.
• RTK grouping strategy. session-dedup (content-addressed cross-turn dedup, SHA-256 keyed,
  reversible). headroom/SmartCrusher (lossless columnar compaction of homogeneous JSON arrays,
  ≥30%, dependency-free).

Advanced

• ccr retrieve engine + omniroute_ccr_retrieve MCP tool (scope read:compression).
• llmlingua async engine — pluggable backend, fail-open in 3 layers, never touches code
  (real ONNX backend is a documented VPS follow-up).
• Reproducible A/B benchmark over the engines + N4 gate.

Studios

• Compression Studio (Tela A): compression WS channel + fire-and-forget emit at the
  chokepoint; pure compressionFlowModel + replay; EngineNode/IoNode/CompressionCockpit/
  WaterfallInspector + useLiveCompression/useCompressionReplay.
• Combo/Routing Studio (Tela B) (backend-zero — events already exist): comboFlowModel +
  fleetAggregation; Request/Strategy/ProviderCascade/Response nodes + ComboLiveStudio
  (Single⇄Fleet, graceful disconnect banner).

Gates

• Opt-in bail-out discipline (failure-aware + skip-if-gain<10%, default-off byte-identical).
• RTK filter learn/discover (mine history → suggest filters). MCP tool-manifest cardinality.

Deep review (F5) — converged in 2 rounds

A 3-agent battery (correctness + silent-failure + type-design) over the full diff found and the
fixes resolved (all with tests): HIGH ReDoS in discover.ts (~54s → ~150ms), HIGH djb2
32-bit→SHA-256 hash with an equality guard in session-dedup, MEDIUM bail-out telemetry that
silently dropped a crashing engine, plus clamps/logs/docstring/fleet-clock fixes. Round 2 verified
all fixes correct with no new issues. (Earlier, per-item trust-but-verify also caught a headroom
mixed-column data-loss bug and a build-breaking NodeTypes-from-react import.)

Test gate (20 commits)

compression unit suite 577/577 · test:vitest (MCP) 171/171 · new UI render tests
50/50 · typecheck:core clean · check:cycles 0 · check:any-budget OK · no new npm deps.

Deliberately deferred (env-gated)

• App/e2e: dashboard tab/nav wiring for the two Studios (left untouched on purpose),
  U6 step-streaming, Playwright e2e of the live WS flow.
• VPS: Anthropic context-management delegated header, the real LLMLingua-2 ONNX backend,
  Qdrant int8 recall, real A/B benchmark numbers.

Acknowledged by-design (not bugs)

session-dedup/headroom are opt-in lossy engines (NOT in the default pipeline), like
caveman/ultra — their markers/tables are the compressed payload; reconstruct* is for
replay/verification, not the wire. Combo "exhausted" needs a producer terminal event. CCR's
in-memory store has a documented LRU/TTL follow-up.

Note: if release/v3.8.25 advanced chatCore.ts/events/types.ts/strategySelector.ts since
this branch was cut, a rebase may be needed before merge.

[chatgpt-codex-connector]

You have reached your Codex usage limits for code reviews. You can see your limits in the Codex usage dashboard.

[gemini-code-assist]

Code Review

This pull request introduces a modular compression pipeline featuring several new engines: CCR (Content-Compression-Retrieve), Headroom SmartCrusher (tabular compaction of homogeneous JSON arrays), session-dedup (cross-turn deduplication), and LLMLingua (async semantic pruning). It also adds a comprehensive offline evaluation and benchmarking harness, integrates these engines into the async stacked pipeline with TV1 bailout capabilities, and implements live visualization dashboards for both routing cascades and compression steps. The review comments highlight several valuable improvement opportunities, including addressing a potential O(N^2) complexity issue in suffix block extraction, improving deduplication efficiency by removing a premature loop break, handling out-of-order or missed WebSocket events defensively in the cascade flow model, and adding defensive null/undefined checks across various engine entry points.

Important

The consumer version of Gemini Code Assist on GitHub is being sunset. Starting June 18, 2026, new organization installations will be blocked, and all code review activity will officially cease on July 17, 2026.
For more details on the timeline and next steps, please review the Help Documentation.

[gemini-code-assist]

The findSuffixBlocks function has $O(N^2)$ complexity on the number of lines because it slices and joins the remaining lines in a loop. For extremely large inputs (e.g., large log files), this can block the event loop or cause out-of-memory crashes. Consider adding a defensive limit to the maximum number of lines processed to guard against this.

Suggested change

	for (let start = 0; start < n; start++) {
	const block = lines.slice(start).join("\n");
	const limit = Math.min(n, 1000); // Guard against O(N^2) complexity on huge inputs
	for (let start = 0; start < limit; start++) {
	const block = lines.slice(start).join("\n");

[gemini-code-assist]

The break; statement here stops the loop after the first duplicate block replacement in a message. This prevents other non-overlapping duplicate blocks in the same message from being replaced, which unnecessarily reduces compression efficiency. Since there is already an overlap guard ([...replaced].some(...)), this break; can be safely removed to allow multiple distinct duplicate blocks to be replaced.

Suggested change

	break;
	// Continue to allow replacing other non-overlapping duplicate blocks

[gemini-code-assist]

If the attempt event for a target was missed or arrived out of order, the target will be silently ignored because run.targets.map only updates existing targets. To make the cascade flow robust against out-of-order or missed WebSocket events, handle this defensively by appending the target if it does not exist in run.targets.

      const targetExists = run.targets.some((t) => t.targetIndex === ev.targetIndex);
      const updatedTargets = targetExists
        ? run.targets.map((t) =>
            t.targetIndex === ev.targetIndex
              ? {
                  ...t,
                  state: "failed" as TargetState,
                  latencyMs: ev.latencyMs,
                  error: ev.error,
                  failKind: classifyFailKind(ev.error),
                }
              : t
          )
        : [
            ...run.targets,
            {
              targetIndex: ev.targetIndex,
              provider: ev.provider,
              model: ev.model,
              state: "failed" as TargetState,
              latencyMs: ev.latencyMs,
              error: ev.error,
              failKind: classifyFailKind(ev.error),
            },
          ].sort((a, b) => a.targetIndex - b.targetIndex);

[gemini-code-assist]

Similarly to the failed case, if the attempt event was missed or arrived out of order, the succeeded target will be silently ignored. Handle this defensively by appending the target if it does not exist in run.targets.

      const targetExists = run.targets.some((t) => t.targetIndex === ev.targetIndex);
      const updatedTargets = targetExists
        ? run.targets.map((t) =>
            t.targetIndex === ev.targetIndex
              ? {
                  ...t,
                  state: "succeeded" as TargetState,
                  latencyMs: ev.latencyMs,
                }
              : t
          )
        : [
            ...run.targets,
            {
              targetIndex: ev.targetIndex,
              provider: ev.provider,
              model: ev.model,
              state: "succeeded" as TargetState,
              latencyMs: ev.latencyMs,
            },
          ].sort((a, b) => a.targetIndex - b.targetIndex);

[gemini-code-assist]

To prevent potential runtime crashes if args is null or undefined, add a defensive check for args itself before accessing args.hash.

Suggested change

	export function handleCcrRetrieve(args: { hash: string }): { content: string } | { error: string } {
	if (!args.hash || typeof args.hash !== "string") {
	export function handleCcrRetrieve(args: { hash: string }): { content: string } | { error: string } {
	if (!args || !args.hash || typeof args.hash !== "string") {

[gemini-code-assist]

Add a defensive check to ensure samples is a valid array before accessing samples.length to prevent potential runtime errors if the function is called with null or undefined.

Suggested change

	export function discoverRepeatedNoise(samples: CommandSample[]): NoiseCandidate[] {
	if (samples.length === 0) return [];
	export function discoverRepeatedNoise(samples: CommandSample[]): NoiseCandidate[] {
	if (!samples || !Array.isArray(samples) || samples.length === 0) return [];

[gemini-code-assist]

Add a defensive check to ensure body is not null or undefined before attempting to read its property descriptors, preventing potential runtime crashes.

Suggested change

	export function reconstructSessionDedup(body: Record<string, unknown>): Record<string, unknown> {
	// The reverse map is stored as a non-enumerable property so it doesn't appear
	export function reconstructSessionDedup(body: Record<string, unknown>): Record<string, unknown> {
	if (!body) return body;
	// The reverse map is stored as a non-enumerable property so it doesn't appear

[gemini-code-assist]

Add a defensive check to ensure config is not null or undefined before accessing its properties, preventing potential runtime crashes.

function validateLlmlinguaConfig(config: Record<string, unknown>): EngineValidationResult {
  const errors: string[] = [];
  if (!config) {
    return { valid: false, errors: ["config is required"] };
  }
  if (config["enabled"] !== undefined && typeof config["enabled"] !== "boolean") {

[diegosouzapw]

Studio routes are now wired (URL-reachable) for the VPS/Chrome validation:

• Compression Studio (Tela A): /dashboard/compression/studio
• Combo/Routing Studio (Tela B): /dashboard/combos/live

Both are new non-invasive route pages (no existing route touched). They degrade gracefully (empty state / 'Live disabled' banner) when the WS feed is off, so to see live data the instance must run with OMNIROUTE_ENABLE_LIVE_WS=1 and process a request that triggers compression / a combo.

Discoverable Sidebar nav links are deferred (each item needs an i18nKey across 42 locales + a HideableSidebarItemId union entry under the strict count-guard — a separate i18n chore). Also added README Acknowledgments for TOON, LLMLingua/llmlingua-2-js, ts-morph, React Flow/xyflow and LangGraph.