Skip to content

fix(sse): scan Devin and PromptQL tool markup in linear time - #15099

Merged
diegosouzapw merged 14 commits into
cheaper-inference:release/v3.8.52from
HouMinXi:fix/devin-promptql-tag-scan
Sep 29, 2026
Merged

diegosouzapw merged 14 commits into
cheaper-inference:release/v3.8.52from
HouMinXi:fix/devin-promptql-tag-scan

Conversation

@HouMinXi

Copy link
Copy Markdown
Contributor

Summary

The Devin tool-request parser (<tool>), the Devin summary-envelope check (<summary>) and the PromptQL <final_response> extractor match model output with patterns of the shape <tag>\s*([\s\S]*?)\s*</tag>. The two whitespace classes overlap, so an opening tag followed by a long run of spaces and no closing tag makes each pattern take cubic time: 2,000 spaces cost about 3 s and 4,000 about 22 s, which blocks the event loop and stalls every other request. The text comes from an upstream model, so a prompt-injected or misbehaving model, or a compromised upstream, can trigger it.

Find the blocks with findTagBlocks (added for the Grok Web and web-tools scanners) instead. The summary envelope, which must span the whole trimmed string, is checked with an anchored prefix and suffix test. The result for well-formed input is unchanged; the tests compare the summary and final_response cases against the old regexes.

Related Issues

  • None. This is the follow-up noted in the reviewer notes of the web-tool markup PR, which left these two files for a change that could reuse findTagBlocks.

Validation

  • Change type: other (availability hardening)
  • Focused tests: tests/unit/devin-promptql-tag-scan-linear.test.ts, plus the 15 existing test files that use the Devin and PromptQL executors (167 tests)
  • npm run typecheck:core
  • Prettier and ESLint clean on the changed files
  • Reconciled with the current active release base (release/v3.8.51 at 9bbf6eff8)
  • Production-code changes include a new or updated automated test in this PR

Each new timing test was run against the change reverted (it does not finish; the run was stopped after 250 s) and restored (it passes).

Tests Added Or Updated

  • tests/unit/devin-promptql-tag-scan-linear.test.ts

Coverage Notes

The test exercises every changed production path: extractBareSummaryEnvelope (ordinary input compared with the old regex, unclosed and closed inputs with a long whitespace run), parseDevinToolRequest (a well-formed envelope, mixed narrative, multiple tool requests, unclosed tags, long whitespace and newline runs) and extractFinalResponseMessage (ordinary input compared with the old regex, unclosed tag, long whitespace run).

Reviewer Notes

No behaviour change for well-formed input. An opening tag with no closing tag now ends the scan instead of restarting at each later opening tag; no later tag can be closed either, so the result is the same.

diegosouzapw and others added 13 commits August 29, 2026 03:15
…ead to the npm leg (cheaper-inference#11973)

v3.8.50 shipped with zero desktop assets. The tag push did trigger electron-release.yml
(run 33005490476) but GitHub refused the run at startup:

  Error calling workflow 'npm-publish.yml@5458026'. The nested job 'publish' is
  requesting 'actions: read', but is only allowed 'actions: none'.

npm-publish.yml's `publish` job gained `actions: read` (it downloads the next-build
artefact) and the caller job here never widened its grant — a reusable workflow may not
request more than its caller allows, and the refusal is a startup failure of the WHOLE
run, so the `release` job that attaches the installers, the source archives and the
SBOM never ran either. Nothing about it is visible through the API (no jobs, no
check-runs); only the run page shows the annotation.

- publish-npm: `actions: read` added, with the rule written down (keep the block a
  superset of every job in npm-publish.yml).
- workflow_dispatch: new boolean input `publish_npm` (default true) and the npm leg
  is gated on it, so re-attaching assets to a release whose package already shipped
  does not try to publish the same version twice.
- web-build / build / release checkouts pin `ref: needs.validate.outputs.version`:
  a dispatch builds the tag it names, not the dispatching branch (a tag push resolves
  to the same commit, so nothing changes on the normal path).

actionlint clean; electron-release-desktop-channel-8949, electron-release-efficiency,
build-next-isolated-windows-home-2402, electron-release-latest-yml.repro and
check-workflows suites pass. Next step: dispatch on main with version=v3.8.50 and
publish_npm=false to attach the missing assets.
…ob and the main run (main twin of cheaper-inference#11972) (cheaper-inference#11978)

Same change as cheaper-inference#11972 on release/v3.8.51: the Coverage job had timeout-minutes: 20,
the c8 merge across 8 shards takes ~10 min and the informational Codecov upload hung
for the rest of the budget on two consecutive main runs (33207760653, 33215115341),
ending the job cancelled and turning the run's conclusion cancelled with every
blocking job green. Codecov step: 5-minute ceiling + continue-on-error; job: 30 min.
…rom a repaired ref (cheaper-inference#11982)

* fix(release): resync the electron lockfile and let a dispatch build from a repaired ref

The v3.8.50 desktop re-dispatch (run 33238093090) lost its Linux leg at
`npm ci` in electron/: "Missing: electron-builder-squirrel-windows@26.15.3 from
lock file" plus its 12 transitive entries — the optional Windows-installer subtree of
electron-builder had been dropped when the lock was last regenerated, and no CI ran
the desktop legs between then and the tag (v3.8.49 never ran them; v3.8.50 died at
startup, cheaper-inference#11973). `npm install --package-lock-only` restores the 13 entries; a clean
`npm ci --ignore-scripts` on the result adds 284 packages with no complaint.

The tag itself carries the broken lock, and the workflow now checks out the tag on
dispatch (cheaper-inference#11973), so a dispatch input `build_ref` (default: the version tag) lets the
operator name the repaired line — the v3.8.50 assets will be rebuilt from main, which
is 3.8.50 plus its post-release fixes. Push-triggered runs are unaffected.

actionlint clean; electron-release-desktop-channel-8949, electron-release-efficiency,
electron-release-latest-yml.repro and check-workflows suites pass.

* fix(release): do not regenerate release notes on a re-attach dispatch

`generate_release_notes: true` on an existing release APPENDS GitHub's auto-generated
"What's Changed" block to the curated body — the v3.8.50 re-dispatch (run 33238093090)
added 1,416 chars to the 121 KB notes. Only the tag push should generate notes.
…shes too (cheaper-inference#12020)

The step was gated on github.event_name == 'release'. v3.8.50's package shipped
through a workflow_dispatch (the staged publish, 11 attempts) and the step was
skipped, so the GitHub Release carried no SBOM — it was attached by hand from the
run's sbom-npm artifact (5.0 MB, 1,886 components). Now it attaches on release or
workflow_dispatch whenever a release for the published tag exists, and says so
when it does not (the workflow artifact remains the durable copy either way).

actionlint and prettier clean; npm-publish-artifact-provenance and
check-workflows-provenance-runner suites pass.
… is dispatched on (cheaper-inference#12032)

Twin of cheaper-inference#12022 on main: CodeQL flagged the same input-controlled checkout + npm cache pattern (cache-poisoning/poisonable-step) on main since it's the default branch. Checkouts go back to github.ref; dispatch still works via --ref (documented in the workflow's own on: contract).

Also fixes the packaged-app smoke: it now waits on /api/monitoring/health (which touches the DB) instead of /login (which doesn't), so the smoke can actually distinguish "native driver selected" from "database never opened." electron-smoke-script.test.ts 9/9 (2 new cases).
…lose base-red issues, guard Scorecard on the default branch (main twin) (cheaper-inference#12086)

* fix(ci): accept CVE-2025-68121 in the prebuilt tls-client .so, auto-close base-red issues, guard Scorecard on the default branch

- .trivyignore: CVE-2025-68121 (Go stdlib crypto/tls inside bogdanfinn/tls-client
  v1.15.1, built with go 1.24.1) with justification, expiry and tracker cheaper-inference#12084.
  No upstream rebuild exists; the blocking Trivy gate now also names the ignore
  file explicitly.
- nightly-release-green: close the "not green" issue when the validation passes
  again (the workflow only ever opened/commented it, so stale issues outlived
  the fix and stamped new PRs as base-red inherited).
- scorecard: the action only accepts the DEFAULT branch (the active release
  branch, not main) - guard the job on it so pushes to main stop failing.

Refs cheaper-inference#12084

(cherry picked from commit 8adf34b)

* fix(release): never let the tag-push Create Release append auto notes to the curated body

Twin of the release/v3.8.51 commit (see cheaper-inference#12085).

Refs cheaper-inference#12084
… image's build memory guards (cheaper-inference#11719)

Validated in an isolated worktree against main: typecheck:core clean, 15/15 focused tests pass (docker-build-memory-budget, bun-support, resolve-next-build-bundler-flag). Root cause confirmed against the current workflow config (docker-publish.yml triggers on push to both main and release/v*, so this genuinely needed to target main). One out-of-scope change dropped before merging: config/alibaba-free-tier-allowlist.json's validUntil bump (2026-08-27 -> 2027-12-31) was unrelated to the Docker/Bun fix — reverted to the current value, keeping only the Docker/Bun/memory-guard changes this PR is actually about. Thanks for the thorough root-cause writeup and the worker-pool math.
…heaper-inference#11968) (cheaper-inference#12246)

* test(infra): retry recursive temp-dir removal on main (main twin of cheaper-inference#11968)

`main` has been red since b342c1a on the vitest and integration gates:

  ✖ tests/unit/autoCombo/provider-family-combos.test.ts > auto/<family>
  ✖ chat pipeline applies Codex OAuth fingerprint and priority tier inside combos

Both call resetStorage() from beforeEach, which does an fs.rmSync(TEST_DATA_DIR,
{recursive: true, force: true}) with no retry, and intermittently loses the race
with a not-yet-released SQLite handle (ENOTEMPTY).

release/v3.8.51 fixed this in cheaper-inference#11968 with a mechanical codemod adding
maxRetries/retryDelay to every recursive rm/rmSync/rmdirSync under tests/, but
that PR landed only on the release branch. Because main only receives work at
the release squash, it stayed broken for the whole cycle — and repo-wide gates
then turn every open PR into main red on checks unrelated to their diff.

This is the --base main twin: re-runs the same codemod that already shipped on
the release branch (scripts/ad-hoc/codemod-rm-maxretries.mjs), so the two
branches converge on identical test-teardown semantics. Test-only; no product
logic is touched.

The remaining three failures reported on cheaper-inference#12133 (unit full suite exceeding its
4800s ceiling, package-artifact exceeding 1200s, and the boot-smoke that is
skipped as a consequence) are runner-contention timeouts, not code defects —
validate-release-green.mjs runs those heavy gates concurrently on one shared
hosted runner. There is no fix to port for those.

* chore(scripts): carry the rm-maxretries codemod onto main alongside its output

The codemod that generated the previous commit lives in the repo on
release/v3.8.51 (added by cheaper-inference#11968) but was never on main. Bringing it over keeps
the tool next to the change it produced, so the transformation stays
reproducible and auditable from either branch.
…aper-inference#12363) (cheaper-inference#12618)

Porta para `main` o fix do gate de ESLint que só havia entrado na branch de release — o padrão de PR-companheiro que `_shared/merge-gates.md` §8 prescreve.

As 12 falhas de CI foram discriminadas como o **outro** base-red do main, não deste diff. Todas descendem de um único ponto: `Package Artifact` falha e os 9 shards de E2E mais os 2 Electron Package Smoke consomem esse artefato. A própria issue cheaper-inference#12363 lista os dois separadamente:

- `❌ ESLint: could not parse eslint json` — que é justamente o que este PR conserta;
- `❌ Package artifact (npm pack policy): gate exceeded its 1200s ceiling` — a raiz da cascata.

O PR toca apenas `scripts/quality/validate-release-green.mjs` e seu teste, então não tem caminho para afetar o build do pacote. Teste portado primeiro e falhando no script atual do main (TDD).
…sation-system passthrough (cheaper-inference#13072)

On the Claude Code semantic passthrough with a 1M-context model (system + tools
present), system-role messages are deliberately kept inside messages[] and only
directive-only messages (content: [] + output_config) are relocated off
messages[0]. Anthropic also rejects a text-bearing system message at messages[0]:

    messages.0: use the top-level 'system' parameter for the initial system
    prompt; the directive-only form (content: [] with output_config) is
    accepted at any position

That is exactly where the Output Styles injection lands
(open-sse/services/compression/outputStyles/apply.ts unshifts a system message),
so every Claude Code turn with an output style active on such a model fails
with 400.

Add hoistLeadingTextSystemMessages(): move only the leading run of text-bearing
system-role messages (everything before the first user/assistant turn) into the
top-level system parameter, and call it before relocateDirectiveOnlyMessages()
on that path. Genuine mid-conversation system turns keep their position and
cache prefix; directive-only messages in the run are left for the existing
relocation. Unit tests cover the injected-style case, the string top-level
system case, mixed directive/text runs, and the no-op case.

Repro: POST /v1/messages with Claude Code client headers (user-agent
claude-cli/..., x-app: cli), model claude-opus-5, a top-level system, one tool,
and messages[0] = {role: "system", content: "[OmniRoute Output Styles] ..."}.
Before: 400 from Anthropic. After: 200.

Co-authored-by: ai-stack <ops@ai-stack.local>
…r-inference#13955)

validate-release-green imported the yaml package, which is not a
direct dependency on main. The nightly job then crashed before the
verdict fence, so the tracker comment was empty. js-yaml is already
declared and load() is the same parse for this workflow YAML.

Signed-off-by: Minxi Hou <houminxi@gmail.com>
* deps/main: restore npm 11 lockfile compatibility

Remove obsolete brace-expansion overrides already dropped on the release branch so npm 11 can resolve the locked dependency tree on main. Preserve the scoped libxmljs2 and rimraf constraints and refresh only the affected transitive packages.

Signed-off-by: Minxi Hou <houminxi@gmail.com>

* deps/security: keep main installs within vulnerability limits

Update affected direct and transitive dependencies without relaxing audit thresholds. Keep Next and its ESLint configuration aligned and regenerate the lockfile with npm 11.

Signed-off-by: Minxi Hou <houminxi@gmail.com>

* tests/embeddings: isolate the optional runtime in UI tests

Vite resolves the lazy Transformers import before the runtime fallback can catch an absent optional package. Alias it to an explicitly unavailable backend for UI unit tests and cover both the failure path and injected embeddings without downloading models.

Signed-off-by: Minxi Hou <houminxi@gmail.com>

* memory/embed: do not let Next resolve optional transformers

A literal import("@huggingface/transformers") is still traced at compile
time. Integration /health then 500s when the optional package is absent,
and the combo fingerprint cases time out behind it. Assemble the specifier
at runtime and mark webpackIgnore, same pattern as sql.js.

Signed-off-by: Minxi Hou <houminxi@gmail.com>

* electron/security: clear the desktop lockfile advisories

The desktop lockfile carried three high-severity advisories that the main
tree's sweep never reached, because it has its own package.json and its own
audit step. Move the three override floors up to the fixed releases and
regenerate the lockfile.

xmldom gains the requireWellFormed bypass fixes and the quadratic parse and
dedup fixes; fast-uri gains the host-confusion and IPv6 normalization fixes;
js-yaml gains the merge-key CPU fix. Their consumers here are the packaging
path only: plist for Info.plist, ajv for schema validation, and the update
manifest reader.

Signed-off-by: Minxi Hou <houminxi@gmail.com>

* sse/timing: stop asserting wall-clock setTimeout against Date.now

CI Unit Tests (3/8) failed once npm ci started succeeding: setTimeout(15)
can fire before Date.now() has moved 15ms. Tests now drive createStreamTiming
with an injected clock; production still uses Date.now. The rewritten test
no longer trips unused-vars, so drop that file's leftover eslint suppression.

Signed-off-by: Minxi Hou <houminxi@gmail.com>

* ci: retrigger Build after the runner shutdown

Head 482228c compiled (`Next.js 16.3.3`, 4.4 min) and then the
hosted runner received a shutdown signal while collecting page data.
Every other required check on that SHA is green. Empty commit so the
suite runs again; no product change.

Signed-off-by: Minxi Hou <houminxi@gmail.com>

---------

Signed-off-by: Minxi Hou <houminxi@gmail.com>
Brings the 12 main-only commits into the frozen release branch so the release
PR (cheaper-inference#11442) is no longer CONFLICTING. Most are main twins of fixes the release
branch already carries in a newer shape; resolved by proving the release side
contains them:
- cheaper-inference#12246 temp-dir retry: release uses cleanupTempDataDir / maxRetries (kept ours).
- cheaper-inference#11719 Bun image: release is already on oven/bun 1.4.2 (kept ours).
- cheaper-inference#12086 CVE-2025-68121 .trivyignore: dropped on release by cheaper-inference#12429 (wreq-js).
- cheaper-inference#13865 npm 11 / electron overrides: release pins are newer (kept ours);
  the optional transformers webpack-ignore change merged cleanly.
- streamTiming clock injection: release moved to a monotonic clock (kept ours).
Real main-only fix ported: cheaper-inference#13072 hoistLeadingTextSystemMessages, added to the
release's claudeSystemRole.ts (the call site in chatCore.ts merged cleanly).
The Devin tool-request and summary-envelope parsers and the PromptQL
final_response extractor matched model output with patterns of the shape
<tag>\s*([\s\S]*?)\s*</tag>, whose overlapping whitespace classes are cubic
on a long run of spaces after an unclosed tag (2,000 spaces cost about 3 s,
4,000 about 22 s), blocking the event loop for every other request. Find the
blocks with findTagBlocks and anchored prefix/suffix checks instead; the
result for well-formed input is unchanged.

Signed-off-by: Minxi Hou <houminxi@gmail.com>
@HouMinXi
HouMinXi force-pushed the fix/devin-promptql-tag-scan branch from ab30ebb to 2bbb5c6 Compare September 29, 2026 14:03
@HouMinXi
HouMinXi changed the base branch from release/v3.8.51 to release/v3.8.52 September 29, 2026 16:19
@diegosouzapw
diegosouzapw merged commit 1aa0af9 into cheaper-inference:release/v3.8.52 Sep 29, 2026
11 of 16 checks passed
diegosouzapw added a commit that referenced this pull request Sep 29, 2026
Carries #15099 into release/v3.8.51 (cherry-picked, red->green proven on the tip). Thanks @HouMinXi!

Co-authored-by: Minxi Hou <houminxi@gmail.com>
@diegosouzapw

Copy link
Copy Markdown
Collaborator

Thanks @HouMinXi! Heads-up on how this landed: the PR was retargeted to release/v3.8.52 shortly before the merge, so the squash went to .52 (and, because the branch carried a merge of the v3.8.51 tip, that squash also brought .51-only commits into .52; they will reconcile at the sync-back). The fix itself reached release/v3.8.51 through #15120 (your commit cherry-picked, authorship preserved).

@diegosouzapw diegosouzapw mentioned this pull request Sep 29, 2026
@HouMinXi
HouMinXi deleted the fix/devin-promptql-tag-scan branch October 28, 2026 10:12
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants