Repository navigation
feat(chaos+ponytail): parallel chaos-mode dispatch + ponytail output … - #7781
diegosouzapw merged 5 commits into
Conversation
…style (rebased on v3.8.49)
- Chaos mode: new auto/chaos variant fans the prompt out to the top-N
stable models in parallel and returns a single merged SSE stream.
- Progressive streaming: each panel model's answer is enqueued as it
lands (omni-chaos-part event), instead of awaiting the whole panel.
- withTimeout now aborts the underlying request (modelAbortSignal) on
timeout so the connection is released, not leaked.
- concatSseText parses both OpenAI and Anthropic SSE wire formats.
- autoPrefix/modePacks add the chaos-mode weight pack; virtualFactory
materializes auto/chaos with fusion strategy + chaos config flag.
- Ponytail (lazy-senior-dev mode) integrated into the existing
OUTPUT_STYLE_CATALOG registry (id 'ponytail') so it rides the production
output-style injector, instead of a bespoke duplicate module. Dev-only
scripts and the duplicate ponytail/ module are removed.
- Tests: chaosEngine/chaosVirtualCombo cover panel dispatch, progressive
broadcast, timeout abort, and Anthropic parsing; autoCombo pack count
updated to 6.
Rebased onto release/v3.8.49 (no provider-registry or validation changes —
those are split out per review).
There was a problem hiding this comment.
Code Review
This pull request introduces 'chaos' mode, which implements parallel multi-model dispatch with progressive SSE streaming of individual model responses, and adds a new 'ponytail' output style to the compression catalog. Feedback on these changes highlights a potential race condition in the timeout handling of the chaos engine that could lead to lost progressive failure events, and suggests a simplification to the primary model selection logic.
Important
The consumer version of Gemini Code Assist on GitHub is being sunset. Starting June 18, 2026, new organization installations will be blocked, and all code review activity will officially cease on July 17, 2026.
For more details on the timeline and next steps, please review the Help Documentation.
| function withTimeout<T>( | ||
| p: Promise<T>, | ||
| ms: number, | ||
| fallback: T, | ||
| onTimeout?: () => void | ||
| ): Promise<T> { | ||
| return new Promise((resolve) => { | ||
| const t = setTimeout(() => { | ||
| try { | ||
| onTimeout?.(); | ||
| } catch { | ||
| /* ignore abort errors */ | ||
| } | ||
| resolve(fallback); | ||
| }, ms); | ||
| Promise.resolve(p) | ||
| .then((v) => { | ||
| clearTimeout(t); | ||
| resolve(v); | ||
| }) | ||
| .catch(() => { | ||
| clearTimeout(t); | ||
| resolve(fallback); | ||
| }); | ||
| }); | ||
| } |
There was a problem hiding this comment.
There is a potential race condition in withTimeout when a timeout occurs. When the timeout fires, withTimeout immediately resolves to the fallback value. This causes Promise.all(modelPromises) in handleChaosChat to resolve and proceed to enqueue the final chunk and close the stream. However, the aborted promise p (the IIFE) only rejects asynchronously after the abort signal propagates. Its catch block (which enqueues the progressive omni-chaos-part failure event) runs in a future microtask turn. By that time, the stream may have already been closed or the [DONE] message enqueued, causing the progressive failure event to be lost or sent out of order.
To fix this, withTimeout should give the aborted promise a short grace period (e.g., 500ms) to settle and run its catch block before force-resolving with the fallback.
function withTimeout<T>(
p: Promise<T>,
ms: number,
fallback: T,
onTimeout?: () => void
): Promise<T> {
return new Promise((resolve) => {
let timedOut = false;
const t = setTimeout(() => {
timedOut = true;
try {
onTimeout?.();
} catch {
/* ignore abort errors */
}
const grace = setTimeout(() => {
resolve(fallback);
}, 500);
Promise.resolve(p)
.then((v) => {
clearTimeout(grace);
resolve(v);
})
.catch(() => {
clearTimeout(grace);
resolve(fallback);
});
}, ms);
Promise.resolve(p)
.then((v) => {
if (!timedOut) {
clearTimeout(t);
resolve(v);
}
})
.catch(() => {
if (!timedOut) {
clearTimeout(t);
resolve(fallback);
}
});
});
}| const primaryPart = | ||
| (primaryModel && allParts.find((p) => p.model === primaryModel && p.ok)) || | ||
| allParts.filter((p) => p.ok).slice(-1)[0] || | ||
| successes[0]; |
There was a problem hiding this comment.
The primary part selection logic can be simplified. Since successes is already defined as allParts.filter((p) => p.ok) and guaranteed to have at least one element (otherwise the function would have returned early), we can avoid the redundant filtering and slicing by directly accessing the last element of successes.
| const primaryPart = | |
| (primaryModel && allParts.find((p) => p.model === primaryModel && p.ok)) || | |
| allParts.filter((p) => p.ok).slice(-1)[0] || | |
| successes[0]; | |
| const primaryPart = | |
| (primaryModel && allParts.find((p) => p.model === primaryModel && p.ok)) || | |
| successes[successes.length - 1]; |
|
Thanks for this — the progressive SSE broadcast design for Ran it through CI-repro + a couple of targeted checks before merging, and found a few things that need addressing first — happy to help land these in your branch:
None of this is a rejection of the idea — it's good work and we'd like to ship it. Once the file-size + race items are fixed and the chaos-mode/permission question is settled, this is mergeable. |
Co-authored-by: diegosouzapw <8016841+diegosouzapw@users.noreply.github.com>
…file-size gate combo.ts's chaos-detection block pushed the frozen file-size gate over its cap (3387 lines) after merging release/v3.8.49. Extracted the config detection + model-list building into dispatchChaosFromCombo() in chaosEngine.ts (the module this PR already introduces), mirroring the existing fusion-strategy short-circuit pattern. combo.ts now does a 9-line early-return; no behavior change. combo.ts: 3406 -> 3386 lines (frozen cap 3387). handleComboChat complexity/cognitive/lines all decrease as a side effect (120->118 / 153->152 / 1541->1524) since the block moved out. Co-authored-by: diegosouzapw <8016841+diegosouzapw@users.noreply.github.com>
|
Validated in local merge-train (fast mode) on 192.168.0.113 @ 809513aabe7c4a657710cc97c7450a42a3cf22ae — 5-PR wave: static gates + changed tests + vitest green (log: /tmp/mt-full.log + suite log on the box). Full-suite parity for today already ran on this branch (runs 4+5). |
0b2c46a
into
cheaper-inference:release/v3.8.49
cheaper-inference#7781) * feat(chaos+ponytail): parallel chaos-mode dispatch + ponytail output style (rebased on v3.8.49) - Chaos mode: new auto/chaos variant fans the prompt out to the top-N stable models in parallel and returns a single merged SSE stream. - Progressive streaming: each panel model's answer is enqueued as it lands (omni-chaos-part event), instead of awaiting the whole panel. - withTimeout now aborts the underlying request (modelAbortSignal) on timeout so the connection is released, not leaked. - concatSseText parses both OpenAI and Anthropic SSE wire formats. - autoPrefix/modePacks add the chaos-mode weight pack; virtualFactory materializes auto/chaos with fusion strategy + chaos config flag. - Ponytail (lazy-senior-dev mode) integrated into the existing OUTPUT_STYLE_CATALOG registry (id 'ponytail') so it rides the production output-style injector, instead of a bespoke duplicate module. Dev-only scripts and the duplicate ponytail/ module are removed. - Tests: chaosEngine/chaosVirtualCombo cover panel dispatch, progressive broadcast, timeout abort, and Anthropic parsing; autoCombo pack count updated to 6. Rebased onto release/v3.8.49 (no provider-registry or validation changes — those are split out per review). * fix(combo): align chaos dispatch callback with ChaosTarget signature Co-authored-by: diegosouzapw <8016841+diegosouzapw@users.noreply.github.com> * fix(combo): extract chaos dispatch to chaosEngine.ts to fix combo.ts file-size gate combo.ts's chaos-detection block pushed the frozen file-size gate over its cap (3387 lines) after merging release/v3.8.49. Extracted the config detection + model-list building into dispatchChaosFromCombo() in chaosEngine.ts (the module this PR already introduces), mirroring the existing fusion-strategy short-circuit pattern. combo.ts now does a 9-line early-return; no behavior change. combo.ts: 3406 -> 3386 lines (frozen cap 3387). handleComboChat complexity/cognitive/lines all decrease as a side effect (120->118 / 153->152 / 1541->1524) since the block moved out. Co-authored-by: diegosouzapw <8016841+diegosouzapw@users.noreply.github.com> --------- Co-authored-by: Diego Rodrigues de Sa e Souza <diegosouza.pw@gmail.com> Co-authored-by: Moseyuh333 <Moseyuh333@users.noreply.github.com> Co-authored-by: diegosouzapw <8016841+diegosouzapw@users.noreply.github.com>
…ety carve-out The original port (cheaper-inference#7781) omitted the ponytail safety carve-out that upstream ships ('Never simplify away: input validation at trust boundaries, error handling that prevents data loss, security measures, or accessibility basics'). It is not a removal — the clause never existed in the compression catalog. Port it as an exported SAFETY_BOUNDARIES constant on ponytail and less-code, with translations for the 10 style languages. OutputStyle.boundaries was declared but never read (dead field). Resolve it in buildStyleBoundaries(): SHARED_BOUNDARIES is always the base clause and a style that declares boundaries contributes its own clause on top, deduped, in catalog order. The two ADD rather than replace so a code-shaping style never loses the shared guarantee (e.g. 'keep code blocks, file paths, commands, errors, URLs exact'). A style that declares nothing (terse-prose) stays byte-identical to the legacy instructions + SHARED_BOUNDARIES injection (D-A5); per-(selection, language) output is static and prompt-cache stable (D-A4).
cheaper-inference#7781) * feat(chaos+ponytail): parallel chaos-mode dispatch + ponytail output style (rebased on v3.8.49) - Chaos mode: new auto/chaos variant fans the prompt out to the top-N stable models in parallel and returns a single merged SSE stream. - Progressive streaming: each panel model's answer is enqueued as it lands (omni-chaos-part event), instead of awaiting the whole panel. - withTimeout now aborts the underlying request (modelAbortSignal) on timeout so the connection is released, not leaked. - concatSseText parses both OpenAI and Anthropic SSE wire formats. - autoPrefix/modePacks add the chaos-mode weight pack; virtualFactory materializes auto/chaos with fusion strategy + chaos config flag. - Ponytail (lazy-senior-dev mode) integrated into the existing OUTPUT_STYLE_CATALOG registry (id 'ponytail') so it rides the production output-style injector, instead of a bespoke duplicate module. Dev-only scripts and the duplicate ponytail/ module are removed. - Tests: chaosEngine/chaosVirtualCombo cover panel dispatch, progressive broadcast, timeout abort, and Anthropic parsing; autoCombo pack count updated to 6. Rebased onto release/v3.8.49 (no provider-registry or validation changes — those are split out per review). * fix(combo): align chaos dispatch callback with ChaosTarget signature Co-authored-by: diegosouzapw <8016841+diegosouzapw@users.noreply.github.com> * fix(combo): extract chaos dispatch to chaosEngine.ts to fix combo.ts file-size gate combo.ts's chaos-detection block pushed the frozen file-size gate over its cap (3387 lines) after merging release/v3.8.49. Extracted the config detection + model-list building into dispatchChaosFromCombo() in chaosEngine.ts (the module this PR already introduces), mirroring the existing fusion-strategy short-circuit pattern. combo.ts now does a 9-line early-return; no behavior change. combo.ts: 3406 -> 3386 lines (frozen cap 3387). handleComboChat complexity/cognitive/lines all decrease as a side effect (120->118 / 153->152 / 1541->1524) since the block moved out. Co-authored-by: diegosouzapw <8016841+diegosouzapw@users.noreply.github.com> --------- Co-authored-by: Diego Rodrigues de Sa e Souza <diegosouza.pw@gmail.com> Co-authored-by: Moseyuh333 <Moseyuh333@users.noreply.github.com> Co-authored-by: diegosouzapw <8016841+diegosouzapw@users.noreply.github.com>
…style (rebased on v3.8.49)
Rebased onto release/v3.8.49 (no provider-registry or validation changes — those are split out per review).
Summary
Related Issues
Validation
npm run lintnpm run test:unitnpm run test:coverage>= 60%for statements, lines, functions, and branchesTests Added Or Updated
Coverage Notes
src/,open-sse/,electron/, orbin/, explain which tests cover the change.Reviewer Notes