Skip to content

feat(chaos+ponytail): parallel chaos-mode dispatch + ponytail output … - #7781

Merged
diegosouzapw merged 5 commits into
cheaper-inference:release/v3.8.49from
Moseyuh333:pr/chaos-ponytail-v349
Jul 20, 2026
Merged

diegosouzapw merged 5 commits into
cheaper-inference:release/v3.8.49from
Moseyuh333:pr/chaos-ponytail-v349

Conversation

@Moseyuh333

Copy link
Copy Markdown
Contributor

…style (rebased on v3.8.49)

  • Chaos mode: new auto/chaos variant fans the prompt out to the top-N stable models in parallel and returns a single merged SSE stream.
    • Progressive streaming: each panel model's answer is enqueued as it lands (omni-chaos-part event), instead of awaiting the whole panel.
    • withTimeout now aborts the underlying request (modelAbortSignal) on timeout so the connection is released, not leaked.
    • concatSseText parses both OpenAI and Anthropic SSE wire formats.
    • autoPrefix/modePacks add the chaos-mode weight pack; virtualFactory materializes auto/chaos with fusion strategy + chaos config flag.
  • Ponytail (lazy-senior-dev mode) integrated into the existing OUTPUT_STYLE_CATALOG registry (id 'ponytail') so it rides the production output-style injector, instead of a bespoke duplicate module. Dev-only scripts and the duplicate ponytail/ module are removed.
  • Tests: chaosEngine/chaosVirtualCombo cover panel dispatch, progressive broadcast, timeout abort, and Anthropic parsing; autoCombo pack count updated to 6.

Rebased onto release/v3.8.49 (no provider-registry or validation changes — those are split out per review).

Summary

  • Describe the user-facing or operational change.

Related Issues

  • Closes #
  • Related to #

Validation

  • npm run lint
  • npm run test:unit
  • npm run test:coverage
  • Coverage is still >= 60% for statements, lines, functions, and branches
  • SonarQube PR analysis is green or any remaining issues are explicitly documented below

Tests Added Or Updated

  • List every changed or added automated test file.
  • If no production code changed, state that here.

Coverage Notes

  • If this PR changes src/, open-sse/, electron/, or bin/, explain which tests cover the change.
  • If coverage moved down in any touched file, explain why and what follow-up task will recover it.

Reviewer Notes

  • Call out any risky areas, migrations, feature flags, or manual validation that reviewers should know about.

…style (rebased on v3.8.49)

- Chaos mode: new auto/chaos variant fans the prompt out to the top-N
  stable models in parallel and returns a single merged SSE stream.
  - Progressive streaming: each panel model's answer is enqueued as it
    lands (omni-chaos-part event), instead of awaiting the whole panel.
  - withTimeout now aborts the underlying request (modelAbortSignal) on
    timeout so the connection is released, not leaked.
  - concatSseText parses both OpenAI and Anthropic SSE wire formats.
  - autoPrefix/modePacks add the chaos-mode weight pack; virtualFactory
    materializes auto/chaos with fusion strategy + chaos config flag.
- Ponytail (lazy-senior-dev mode) integrated into the existing
  OUTPUT_STYLE_CATALOG registry (id 'ponytail') so it rides the production
  output-style injector, instead of a bespoke duplicate module. Dev-only
  scripts and the duplicate ponytail/ module are removed.
- Tests: chaosEngine/chaosVirtualCombo cover panel dispatch, progressive
  broadcast, timeout abort, and Anthropic parsing; autoCombo pack count
  updated to 6.

Rebased onto release/v3.8.49 (no provider-registry or validation changes —
those are split out per review).
Copilot AI review requested due to automatic review settings July 19, 2026 15:47
@Moseyuh333
Moseyuh333 requested a review from diegosouzapw as a code owner July 19, 2026 15:47

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot was unable to review this pull request because the user who requested the review has reached their quota limit.

@gemini-code-assist gemini-code-assist Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This pull request introduces 'chaos' mode, which implements parallel multi-model dispatch with progressive SSE streaming of individual model responses, and adds a new 'ponytail' output style to the compression catalog. Feedback on these changes highlights a potential race condition in the timeout handling of the chaos engine that could lead to lost progressive failure events, and suggests a simplification to the primary model selection logic.

Important

The consumer version of Gemini Code Assist on GitHub is being sunset. Starting June 18, 2026, new organization installations will be blocked, and all code review activity will officially cease on July 17, 2026.
For more details on the timeline and next steps, please review the Help Documentation.

Comment on lines +82 to +107
function withTimeout<T>(
p: Promise<T>,
ms: number,
fallback: T,
onTimeout?: () => void
): Promise<T> {
return new Promise((resolve) => {
const t = setTimeout(() => {
try {
onTimeout?.();
} catch {
/* ignore abort errors */
}
resolve(fallback);
}, ms);
Promise.resolve(p)
.then((v) => {
clearTimeout(t);
resolve(v);
})
.catch(() => {
clearTimeout(t);
resolve(fallback);
});
});
}

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

high

There is a potential race condition in withTimeout when a timeout occurs. When the timeout fires, withTimeout immediately resolves to the fallback value. This causes Promise.all(modelPromises) in handleChaosChat to resolve and proceed to enqueue the final chunk and close the stream. However, the aborted promise p (the IIFE) only rejects asynchronously after the abort signal propagates. Its catch block (which enqueues the progressive omni-chaos-part failure event) runs in a future microtask turn. By that time, the stream may have already been closed or the [DONE] message enqueued, causing the progressive failure event to be lost or sent out of order.

To fix this, withTimeout should give the aborted promise a short grace period (e.g., 500ms) to settle and run its catch block before force-resolving with the fallback.

function withTimeout<T>(
  p: Promise<T>,
  ms: number,
  fallback: T,
  onTimeout?: () => void
): Promise<T> {
  return new Promise((resolve) => {
    let timedOut = false;
    const t = setTimeout(() => {
      timedOut = true;
      try {
        onTimeout?.();
      } catch {
        /* ignore abort errors */
      }
      const grace = setTimeout(() => {
        resolve(fallback);
      }, 500);

      Promise.resolve(p)
        .then((v) => {
          clearTimeout(grace);
          resolve(v);
        })
        .catch(() => {
          clearTimeout(grace);
          resolve(fallback);
        });
    }, ms);

    Promise.resolve(p)
      .then((v) => {
        if (!timedOut) {
          clearTimeout(t);
          resolve(v);
        }
      })
      .catch(() => {
        if (!timedOut) {
          clearTimeout(t);
          resolve(fallback);
        }
      });
  });
}

Comment on lines +382 to +385
const primaryPart =
(primaryModel && allParts.find((p) => p.model === primaryModel && p.ok)) ||
allParts.filter((p) => p.ok).slice(-1)[0] ||
successes[0];

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

medium

The primary part selection logic can be simplified. Since successes is already defined as allParts.filter((p) => p.ok) and guaranteed to have at least one element (otherwise the function would have returned early), we can avoid the redundant filtering and slicing by directly accessing the last element of successes.

Suggested change
const primaryPart =
(primaryModel && allParts.find((p) => p.model === primaryModel && p.ok)) ||
allParts.filter((p) => p.ok).slice(-1)[0] ||
successes[0];
const primaryPart =
(primaryModel && allParts.find((p) => p.model === primaryModel && p.ok)) ||
successes[successes.length - 1];

@diegosouzapw

Copy link
Copy Markdown
Collaborator

Thanks for this — the progressive SSE broadcast design for auto/chaos (serialized enqueue chain, per-model abort on timeout) is a genuinely nice pattern, and folding ponytail into the existing OUTPUT_STYLE_CATALOG instead of a bespoke module is exactly the right call.

Ran it through CI-repro + a couple of targeted checks before merging, and found a few things that need addressing first — happy to help land these in your branch:

  1. File-size gate (confirmed failing in CI today). open-sse/services/combo.ts is frozen at 3387 lines and this PR pushes it to 3404. The fix is mechanical: move the model/judge-array extraction currently inlined at the config.chaos.enabled branch into chaosEngine.ts as an exported helper, so combo.ts's own dispatch site shrinks back to a thin wiring call (same pattern already used for the quota-share/task-aware branches nearby).

  2. A real race drops progressive failure events on panel timeout. I reproduced this directly against your handleChaosChat: with a 2-model panel where one model times out, the client only received 1 of the 2 expected omni-chaos-part events — the timed-out model's fail event never reaches the stream, because it's enqueued from the original call's catch handler, which resolves after withTimeout's fallback has already closed the stream. This matches what Gemini's review flagged. The existing timeout test only covers runChaosPanel, not this SSE path in handleChaosChat, so it wasn't caught. Worth a regression test alongside the fix.

  3. Naming/permission collision with the existing "Chaos Mode" feature. OmniRoute already ships a chaosConfig.ts/chaosExecutor.ts + /api/chaos/run "Chaos Mode" — parallel dispatch to multiple providers, gated behind a per-API-key chaosModeEnabled permission specifically because fanning one request out to N models multiplies cost/quota. auto/chaos does conceptually the same fan-out but through the model-string, with no permission gate at all. Could you either rename it to avoid the clash, or wire it behind the same chaosModeEnabled flag so operators keep control over who can trigger the cost multiplier?

  4. Minor: a changelog.d/features/ fragment is missing (see CONTRIBUTING.md), and X-OmniRoute-Chaos-Primary is always an empty string since headers are fixed before the real primary model is known — either populate it via a trailer/final event or drop it.

None of this is a rejection of the idea — it's good work and we'd like to ship it. Once the file-size + race items are fixed and the chaos-mode/permission question is settled, this is mergeable.

diegosouzapw and others added 4 commits July 19, 2026 17:12
Co-authored-by: diegosouzapw <8016841+diegosouzapw@users.noreply.github.com>
…file-size gate

combo.ts's chaos-detection block pushed the frozen file-size gate over its
cap (3387 lines) after merging release/v3.8.49. Extracted the config
detection + model-list building into dispatchChaosFromCombo() in
chaosEngine.ts (the module this PR already introduces), mirroring the
existing fusion-strategy short-circuit pattern. combo.ts now does a 9-line
early-return; no behavior change.

combo.ts: 3406 -> 3386 lines (frozen cap 3387).
handleComboChat complexity/cognitive/lines all decrease as a side effect
(120->118 / 153->152 / 1541->1524) since the block moved out.

Co-authored-by: diegosouzapw <8016841+diegosouzapw@users.noreply.github.com>
@diegosouzapw

Copy link
Copy Markdown
Collaborator

Validated in local merge-train (fast mode) on 192.168.0.113 @ 809513aabe7c4a657710cc97c7450a42a3cf22ae — 5-PR wave: static gates + changed tests + vitest green (log: /tmp/mt-full.log + suite log on the box). Full-suite parity for today already ran on this branch (runs 4+5).

@diegosouzapw
diegosouzapw merged commit 0b2c46a into cheaper-inference:release/v3.8.49 Jul 20, 2026
15 checks passed
@diegosouzapw diegosouzapw mentioned this pull request Jul 23, 2026
HouMinXi pushed a commit to HouMinXi/OmniRoute that referenced this pull request Aug 2, 2026
cheaper-inference#7781)

* feat(chaos+ponytail): parallel chaos-mode dispatch + ponytail output style (rebased on v3.8.49)

- Chaos mode: new auto/chaos variant fans the prompt out to the top-N
  stable models in parallel and returns a single merged SSE stream.
  - Progressive streaming: each panel model's answer is enqueued as it
    lands (omni-chaos-part event), instead of awaiting the whole panel.
  - withTimeout now aborts the underlying request (modelAbortSignal) on
    timeout so the connection is released, not leaked.
  - concatSseText parses both OpenAI and Anthropic SSE wire formats.
  - autoPrefix/modePacks add the chaos-mode weight pack; virtualFactory
    materializes auto/chaos with fusion strategy + chaos config flag.
- Ponytail (lazy-senior-dev mode) integrated into the existing
  OUTPUT_STYLE_CATALOG registry (id 'ponytail') so it rides the production
  output-style injector, instead of a bespoke duplicate module. Dev-only
  scripts and the duplicate ponytail/ module are removed.
- Tests: chaosEngine/chaosVirtualCombo cover panel dispatch, progressive
  broadcast, timeout abort, and Anthropic parsing; autoCombo pack count
  updated to 6.

Rebased onto release/v3.8.49 (no provider-registry or validation changes —
those are split out per review).

* fix(combo): align chaos dispatch callback with ChaosTarget signature

Co-authored-by: diegosouzapw <8016841+diegosouzapw@users.noreply.github.com>

* fix(combo): extract chaos dispatch to chaosEngine.ts to fix combo.ts file-size gate

combo.ts's chaos-detection block pushed the frozen file-size gate over its
cap (3387 lines) after merging release/v3.8.49. Extracted the config
detection + model-list building into dispatchChaosFromCombo() in
chaosEngine.ts (the module this PR already introduces), mirroring the
existing fusion-strategy short-circuit pattern. combo.ts now does a 9-line
early-return; no behavior change.

combo.ts: 3406 -> 3386 lines (frozen cap 3387).
handleComboChat complexity/cognitive/lines all decrease as a side effect
(120->118 / 153->152 / 1541->1524) since the block moved out.

Co-authored-by: diegosouzapw <8016841+diegosouzapw@users.noreply.github.com>

---------

Co-authored-by: Diego Rodrigues de Sa e Souza <diegosouza.pw@gmail.com>
Co-authored-by: Moseyuh333 <Moseyuh333@users.noreply.github.com>
Co-authored-by: diegosouzapw <8016841+diegosouzapw@users.noreply.github.com>
woodsonl added a commit to woodsonl/OmniRoute that referenced this pull request Sep 22, 2026
…ety carve-out

The original port (cheaper-inference#7781) omitted the ponytail safety carve-out that
upstream ships ('Never simplify away: input validation at trust
boundaries, error handling that prevents data loss, security measures,
or accessibility basics'). It is not a removal — the clause never existed
in the compression catalog. Port it as an exported SAFETY_BOUNDARIES
constant on ponytail and less-code, with translations for the 10 style
languages.

OutputStyle.boundaries was declared but never read (dead field). Resolve
it in buildStyleBoundaries(): SHARED_BOUNDARIES is always the base clause
and a style that declares boundaries contributes its own clause on top,
deduped, in catalog order. The two ADD rather than replace so a
code-shaping style never loses the shared guarantee (e.g. 'keep code
blocks, file paths, commands, errors, URLs exact'). A style that declares
nothing (terse-prose) stays byte-identical to the legacy
instructions + SHARED_BOUNDARIES injection (D-A5); per-(selection,
language) output is static and prompt-cache stable (D-A4).
muhamadgalihsaputra pushed a commit to niyatna/NiyatnaRoute that referenced this pull request Sep 27, 2026
cheaper-inference#7781)

* feat(chaos+ponytail): parallel chaos-mode dispatch + ponytail output style (rebased on v3.8.49)

- Chaos mode: new auto/chaos variant fans the prompt out to the top-N
  stable models in parallel and returns a single merged SSE stream.
  - Progressive streaming: each panel model's answer is enqueued as it
    lands (omni-chaos-part event), instead of awaiting the whole panel.
  - withTimeout now aborts the underlying request (modelAbortSignal) on
    timeout so the connection is released, not leaked.
  - concatSseText parses both OpenAI and Anthropic SSE wire formats.
  - autoPrefix/modePacks add the chaos-mode weight pack; virtualFactory
    materializes auto/chaos with fusion strategy + chaos config flag.
- Ponytail (lazy-senior-dev mode) integrated into the existing
  OUTPUT_STYLE_CATALOG registry (id 'ponytail') so it rides the production
  output-style injector, instead of a bespoke duplicate module. Dev-only
  scripts and the duplicate ponytail/ module are removed.
- Tests: chaosEngine/chaosVirtualCombo cover panel dispatch, progressive
  broadcast, timeout abort, and Anthropic parsing; autoCombo pack count
  updated to 6.

Rebased onto release/v3.8.49 (no provider-registry or validation changes —
those are split out per review).

* fix(combo): align chaos dispatch callback with ChaosTarget signature

Co-authored-by: diegosouzapw <8016841+diegosouzapw@users.noreply.github.com>

* fix(combo): extract chaos dispatch to chaosEngine.ts to fix combo.ts file-size gate

combo.ts's chaos-detection block pushed the frozen file-size gate over its
cap (3387 lines) after merging release/v3.8.49. Extracted the config
detection + model-list building into dispatchChaosFromCombo() in
chaosEngine.ts (the module this PR already introduces), mirroring the
existing fusion-strategy short-circuit pattern. combo.ts now does a 9-line
early-return; no behavior change.

combo.ts: 3406 -> 3386 lines (frozen cap 3387).
handleComboChat complexity/cognitive/lines all decrease as a side effect
(120->118 / 153->152 / 1541->1524) since the block moved out.

Co-authored-by: diegosouzapw <8016841+diegosouzapw@users.noreply.github.com>

---------

Co-authored-by: Diego Rodrigues de Sa e Souza <diegosouza.pw@gmail.com>
Co-authored-by: Moseyuh333 <Moseyuh333@users.noreply.github.com>
Co-authored-by: diegosouzapw <8016841+diegosouzapw@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants