Skip to content

fix(cli): handle escaped dollars around inline math - #7741

Merged
wenshao merged 3 commits into
QwenLM:mainfrom
CubeLander:fix/cli-escaped-dollar-rendering
Jul 26, 2026
Merged

fix(cli): handle escaped dollars around inline math#7741
wenshao merged 3 commits into
QwenLM:mainfrom
CubeLander:fix/cli-escaped-dollar-rendering

Conversation

@CubeLander

@CubeLander CubeLander commented Jul 26, 2026

Copy link
Copy Markdown
Contributor

What this PR does

This PR completes the CLI's escaped-dollar inline-math contract with a bounded, parity-aware delimiter scanner. Escaped dollars can now appear inside formulas or immediately beside a formula boundary, while Markdown escapes in visible prose are removed and inline code remains byte-preserving.

Rendering, table rendering, /copy inline-latex, table splitting, and rendered-width measurement continue to share the same recognition result. The existing 1024-character bound and the currency, shell-variable, adjacent-span, and exact-backtick guards remain in place.

Why it's needed

The recognizer merged in #7701 correctly stopped an escaped opener from becoming math, but its regex still excluded every dollar from a formula body and inspected only one preceding backslash. As a result, a literal dollar inside math, a formula next to an escaped dollar, and even-vs-odd backslash runs could not be represented correctly. The prose renderer also exposed \ from a Markdown escape instead of displaying the intended literal dollar.

This is the narrow follow-up confirmed in #7740. It does not change delimiters or broaden TeX support.

Reviewer Test Plan

How to verify

Enable inline math and render these seven lines exactly:

$x^2$ is valid math
it costs $5 and $10
$5-$10
literal \$x$
formula $x + \$5$
literal then math: \$$x^2$
math then literal: $x^2\$$

Confirm that the first formula renders as , currency and the compact range remain prose, the prose escape displays as literal $x$, the mixed formula displays as x + $5, and the two boundary cases display as $x² and x²$. Inline code containing \$ must retain the backslash.

Run /copy inline-latex against the same output. The four formula sources should be selected in document order, preserving \$ in the copied TeX source where present.

Evidence (Before & After)

Before — merge-base 7959fdb27:

Before: escaped dollars remain raw

After — patched TUI:

After: escaped dollars render correctly

Case Before After
literal \$x$ Backslash visible Displays literal $x$
$x + \$5$ Entire source left raw Renders x + $5; copy retains x + \$5
\$$x^2$ Entire source left raw Displays literal $ followed by rendered
$x^2\$$ Entire source left raw Renders x²$
Inline code containing \$ Literal Still literal and byte-preserving

The baseline was reproduced through the real React/Ink inline renderer and shared copy recognizer on current main: only the first expression was recognized. On this branch, five focused suites exercise prose rendering, table rendering, source copy, table splitting, width measurement, code-span isolation, backslash parity, currency, and the complete seven-line corpus: 300/300 tests passed.

Tested on

OS Status
🍏 macOS
🪟 Windows ⚠️
🐧 Linux ⚠️

Environment (optional)

macOS 26.5.2 with Node.js v26.4.0. A clean npm ci completed the repository build and bundle. The CLI package build, typecheck, lint, and five focused suites (300 tests) passed on the rebased head.

Risk & Scope

  • Main risk or tradeoff: the delimiter implementation changes from one regex to a small bounded scanner so it can model backslash parity; the scanner retains the previous length and context guards and is exercised through every CLI consumer.
  • Not validated / out of scope: Web Shell, new delimiter forms, broader TeX rendering, Windows, and Linux runtime validation.
  • Breaking changes / migration notes: escaped dollars in rendered Markdown prose now follow normal Markdown escape semantics and display without the escape backslash; inline code and copied TeX source remain unchanged.

Linked Issues

Fixes #7740

中文说明

本 PR 的改动

本 PR 用一个有长度边界、能够判断反斜杠奇偶性的定界符扫描器,补全 CLI 对转义美元符号与行内公式的处理契约。转义美元符号现在可以出现在公式内部或紧贴公式边界;可见普通文本中的 Markdown 转义会被去除,而行内代码仍逐字节保留。

普通文本渲染、表格渲染、/copy inline-latex、表格拆分和渲染宽度测量继续共享同一份识别结果。原有的 1024 字符上限、货币、shell 变量、相邻 span 和精确反引号守卫均保持不变。

为什么需要

#7701 合入的识别器已经能阻止转义 opener 被误认成公式,但其正则仍禁止公式主体出现任何美元符号,并且只检查紧邻的一个反斜杠。因此,公式内的字面美元、紧邻转义美元的公式以及奇偶反斜杠 run 都无法正确表达;普通文本渲染器还会把 Markdown 转义中的 \ 直接显示出来,而不是只显示目标美元符号。

这是 #7740 中已确认的窄范围后续修复,不改变定界符,也不扩大 TeX 支持。

Reviewer 测试计划

如何验证

启用行内公式,并原样渲染上方七行语料。确认第一行显示为 ,货币与紧凑价格区间仍是普通文本,普通文本转义显示为 literal $x$,混合公式显示为 x + $5,两种边界分别显示为 $x²x²$。行内代码中的 \$ 必须保留反斜杠。

对同一输出执行 /copy inline-latex。四个公式源码应按文档顺序参与选择,并在适用位置保留复制出的 TeX 源码中的 \$

证据(修复前后)

情况 修复前 修复后
literal \$x$ 反斜杠可见 显示 literal $x$
$x + \$5$ 整段源码保持原样 渲染为 x + $5;复制仍得到 x + \$5
\$$x^2$ 整段源码保持原样 显示字面 $,随后是渲染后的
$x^2\$$ 整段源码保持原样 渲染为 x²$
\$ 的行内代码 保持字面形式 仍保持字面形式且逐字节不变

基线已通过当前 main 的真实 React/Ink 行内渲染器与共享复制识别器复现:只有第一个公式被识别。本分支的五组聚焦测试覆盖普通文本渲染、表格渲染、源码复制、表格拆分、宽度测量、代码段隔离、反斜杠奇偶性、货币和完整七行语料,共 300/300 项通过

已测试平台

OS 状态
🍏 macOS
🪟 Windows ⚠️
🐧 Linux ⚠️

环境(可选)

macOS 26.5.2、Node.js v26.4.0。全新 npm ci 已完成仓库 build 与 bundle;在 rebase 后的 head 上,CLI package 的 build、typecheck、lint 以及五组聚焦测试(300 项)全部通过。

风险与范围

  • 主要风险或取舍:定界符实现从单一正则改为小型有界扫描器,以便表达反斜杠奇偶性;扫描器保留旧有长度与上下文守卫,并通过全部 CLI 消费路径测试。
  • 未验证或范围外:Web Shell、新定界符、更广泛的 TeX 渲染、Windows 和 Linux 运行时验证。
  • Breaking changes / 迁移说明:渲染后的 Markdown 普通文本中的转义美元现在遵循正常 Markdown 转义语义,不再显示转义反斜杠;行内代码与复制出的 TeX 源码保持不变。

关联 Issue

Fixes #7740

Copilot AI review requested due to automatic review settings July 26, 2026 06:29
@github-actions github-actions Bot added the review/self-reported The linked issue was opened by the PR author (self-reported) label Jul 26, 2026
@CubeLander

CubeLander commented Jul 26, 2026

Copy link
Copy Markdown
Contributor Author

E2E/component-path report

Baseline reproduction

On current main, the real React/Ink RenderInline component plus the shared /copy inline-latex recognizer produced this result for the seven-line corpus:

  • $x^2$ rendered as and was available to source copy.
  • Currency and $5-$10 remained prose.
  • literal \$x$ visibly leaked the backslash.
  • $x + \$5$, \$$x^2$, and $x^2\$$ remained entirely raw and were absent from source-copy numbering.
  • Even backslash runs before a delimiter were also rejected because the old lookbehind inspected only the immediately preceding character.

Post-fix verification

On macOS 26.5.2 with Node.js v26.4.0 at 09c6c1371:

  • 5 focused CLI test files passed: 300/300 tests.
  • The exercised paths include the inline Ink renderer, rendered-width measurement, table rendering, /copy inline-latex, and pending table splitting.
  • The seven-line corpus, odd/even backslash parity, inline-code byte preservation, currency, shell variables, CJK adjacency, exact backtick runs, and the 1024-character bound are covered.
  • A clean npm ci completed the repository build and bundle.
  • CLI build, typecheck, and lint passed after the final change and rebase.

Real-terminal evidence

A real TUI before/after comparison using the same ModelStudio model response is now attached in the PR body and in the screenshot comment below. Windows and Linux were not run locally; fork CI covers the available Ubuntu job, while platform-specific skipped jobs remain a maintainer/CI limitation.

@qwen-code-ci-bot

qwen-code-ci-bot commented Jul 26, 2026

Copy link
Copy Markdown
Collaborator

⚠️ Deferred approval withheld — 1 PR CI workflow run(s) on 09c6c13 did not finish green; see the updated table in the Stage 2 comment. Re-run @qwen-code /triage after fixes. finalize run

⚠️ 延迟审批已搁置 —— 09c6c13 有 1 个 PR CI workflow 未以绿色完成,详见 Stage 2 评论中已更新的表格。修复后可重新运行 @qwen-code /triage查看 finalize 运行

@qwen-code-ci-bot

Copy link
Copy Markdown
Collaborator

Thanks for the PR!

Template looks good ✓

Problem: observed bug with clear evidence. Issue #7740 provides a seven-line corpus with expected vs actual output, root cause analysis, and was reproduced on current main through the real RenderInline component. This is a narrow follow-up to the already-merged #7701 — not theoretical hardening.

Direction: aligned. Inline math rendering is squarely within the CLI's Markdown rendering scope, and this completes the escaped-dollar contract that #7701 started. CHANGELOG has no direct reference to inline math, but the area is clearly relevant. No auth, sandbox, model, or telemetry concerns.

Size: not applicable — all changes are in packages/cli/src/ui/utils/ and integration-tests/, no core paths. Production logic: ~405 lines (inline-math.ts 163, InlineMarkdownRenderer.tsx 160, TableRenderer.tsx 80, scenario file 2). Test code: ~118 lines across five test files.

Approach: the scope feels right. Replacing the regex with a bounded scanner is the natural fit here — backslash parity (odd vs even runs) is inherently a counting operation that regex handles poorly, and the three failure modes (literal dollar inside math, formula adjacent to escaped dollar, even-vs-odd runs) all require it. The mergeInlineMathMatches generator is a clean way to keep all consumers (renderer, table, copy, width measurement) sharing one recognition result. Every change in the diff serves the stated goal — no unrelated edits or drive-by refactors.

Moving on to code review. 🔍

中文说明

感谢贡献!

模板完整 ✓

问题:已观测到的 bug,有明确证据。Issue #7740 提供了七行语料及期望/实际输出对比、根因分析,并在当前 main 上通过真实 RenderInline 组件复现。这是已合入的 #7701 的窄范围后续修复,不是理论性加固。

方向:对齐。行内公式渲染完全在 CLI Markdown 渲染范围内,本 PR 补全了 #7701 开始的转义美元符号契约。CHANGELOG 无直接引用,但该领域明显相关。不涉及认证、沙箱、模型或遥测。

规模:不适用——所有改动在 packages/cli/src/ui/utils/integration-tests/ 中,未触及核心路径。生产逻辑约 405 行,测试代码约 118 行。

方案:范围合理。用有界扫描器替换正则是自然选择——反斜杠奇偶性本质上是计数操作,正则难以处理,而三种失败模式都需要它。mergeInlineMathMatches 生成器让所有消费方共享同一份识别结果,设计干净。diff 中每项改动都服务于既定目标,无无关编辑或顺手重构。

进入代码审查 🔍

Qwen Code · qwen3.8-max-preview

Reviewed at 09c6c1371d2bdfd99fb28a315c91afb54d5eb529 · re-run with @qwen-code /triage

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR updates the CLI’s inline-math recognizer to correctly handle escaped dollars inside inline math and adjacent to inline-math boundaries by replacing the previous regex-based approach with a bounded, backslash-parity-aware scanner. It also ensures Markdown backslash-escapes for literal dollars are removed in rendered prose while inline code remains byte-preserving, keeping recognition consistent across rendering, table behavior, width measurement, and /copy inline-latex.

Changes:

  • Replace the inline-math regex with a bounded scanner that tracks backslash-run parity and skips escaped dollars while searching for the closing delimiter.
  • Unescape Markdown dollar-escapes in rendered prose (including within markup like bold/italic/link labels), while keeping inline code spans verbatim.
  • Expand unit + integration coverage for escaped-dollar cases across rendering, table splitting, width measurement, and copy selection.

Reviewed changes

Copilot reviewed 9 out of 9 changed files in this pull request and generated 1 comment.

Show a summary per file
File Description
packages/cli/src/ui/utils/TableRenderer.tsx Switch table ANSI rendering to shared tokenization and prose-dollar unescaping.
packages/cli/src/ui/utils/pending-rendered-height.test.ts Add coverage for table splitting when math contains escaped dollars and pipes.
packages/cli/src/ui/utils/MarkdownDisplay.test.tsx Update table rendering expectations to reflect prose dollar unescaping and mixed math.
packages/cli/src/ui/utils/InlineMarkdownRenderer.tsx Use shared merged token stream (markup + math) and unescape dollars in visible prose while preserving code spans.
packages/cli/src/ui/utils/InlineMarkdownRenderer.test.tsx Add full corpus coverage for escaped dollars inside/around inline math and code-span preservation.
packages/cli/src/ui/utils/inline-math.ts Implement bounded parity-aware inline-math scanner + shared merge generator + prose-dollar unescaping helpers.
packages/cli/src/ui/utils/inline-math.test.ts Add tests for escaped-dollar parity, boundary cases, and dollar-unescaping helpers.
packages/cli/src/ui/commands/copyCommand.test.ts Verify /copy inline-latex ordering and preservation of \\$ in copied TeX source.
integration-tests/terminal-capture/scenarios/markdown-rendering.ts Extend integration prompt sample to include escaped-dollar math/prose boundary cases.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

5. One display math block using $$ fences.
6. One checked and one unchecked task list item.
7. Literal inline code \`$zz$\` and escaped math source \\\\$xy$.
7. Literal inline code \`$zz$\`, escaped prose \\\\$xy$, formula $x + \\\\$5$, and the boundaries \\\\$$x^2$ and $x^2\\\\$$.
@qwen-code-ci-bot

qwen-code-ci-bot commented Jul 26, 2026

Copy link
Copy Markdown
Collaborator

Code Review

Independent proposal: I'd replace the single-regex math recognizer with a character scanner that counts the full backslash run for parity, add a merge generator to interleave markup and math tokens in document order, add Markdown unescape helpers for prose, and update all four consumers (React renderer, ANSI table renderer, copy-command expression list, width-measurement span reader). Remove the old combined regex from both renderers.

Comparison: the PR matches this almost exactly. The implementation is clean:

  • isEscapedAt counts the complete backslash run rather than inspecting only the immediately preceding character — this is the core fix that makes even/odd parity work.
  • readInlineMathSpan preserves every existing context guard (no word char before opener, no space/digit/$ after opener, no trailing space in content, no word/$ after closer) and the 1024-char bound.
  • mergeInlineMathMatches uses a cursor to skip overlapping tokens, with markup winning on index ties — correct, since inline code spans must take priority over math recognition.
  • unescapeMarkdownDollars implements standard Markdown backslash-escape semantics ((\\+)\$floor(n/2) backslashes + $). unescapeMarkdownBeforeMath is a neat trick for trailing backslash runs adjacent to a math delimiter — temporarily appends $, unescapes, slices it off.
  • All four consumers are updated consistently. The old INLINE_MATH_PATTERN_SOURCE export and both INLINE_MARKDOWN_WITH_MATH_REGEX instances are cleanly removed — no other consumers exist (verified via grep).
  • enableInlineMath=false path: nextMath is null, only markup tokens flow. Prose still gets unescapeMarkdownDollars, which is correct — \$$ is Markdown semantics, not math semantics. This is a minor behavioral change (previously \$ rendered literally when math was off) and the PR acknowledges it under "Breaking changes."
  • Inline code content is never unescaped — byte-preserving, as required.

No correctness bugs, security issues, or regressions found. No AGENTS.md violations (ESM, no any, collocated tests, kebab-case filenames, minimal comments).

Testing

This is an unattended CI run — no local tmux testing. CI evidence from the PR's own checks:

The Test (ubuntu-latest, Node 22.x) job failed at the npm run audit:runtime:critical step — the npm registry's audit endpoint returned 400 Bad Request with "This endpoint is being retired" and "Invalid package tree." This is a pre-existing infrastructure issue: the PR does not modify package.json or package-lock.json, and the failure occurs before the test suite runs (no junit.xml artifacts were produced). macOS and Windows test jobs were skipped (matrix fail-fast). The web-shell E2E Smoke job is still in progress.

The author reports 300/300 tests passing across five focused suites on their local macOS environment (author's claim, not independently verified here).

Final CI results for 09c6c13 (auto-updated by the triage finalize job after CI completed):

Check Conclusion
Test (ubuntu-latest, Node 22.x) ❌ failure
Classify PR ✅ success
web-shell E2E Smoke (ubuntu-latest, Node 22.x) ✅ success

One row per check name (latest run); skipped checks omitted; failures sort first. / 每个检查名一行(取最新一次运行),省略 skipped,失败项排在最前。

Not verified: unit test results (CI failed before reaching the test step), Windows/Linux runtime behavior (author tested macOS only). A maintainer can trigger the isolated @qwen-code /tmux job for real-scenario TUI coverage.

中文说明

代码审查

独立方案: 我会用字符扫描器替换单一正则的公式识别器,通过计算完整反斜杠 run 来判断奇偶性;添加合并生成器按文档顺序交错标记和公式 token;添加 Markdown 反转义辅助函数处理普通文本;更新全部四个消费方(React 渲染器、ANSI 表格渲染器、复制命令表达式列表、宽度测量 span 读取器)。移除两个渲染器中的旧组合正则。

对比: PR 与上述方案几乎完全一致。实现干净:isEscapedAt 计算完整反斜杠 run 而非仅检查前一个字符;readInlineMathSpan 保留所有现有上下文守卫和 1024 字符上限;mergeInlineMathMatches 用游标跳过重叠 token,标记在索引相同时优先(正确——行内代码必须优先于公式识别);反转义辅助函数实现标准 Markdown 语义。所有消费方一致更新,旧导出干净移除(grep 确认无其他消费方)。未发现正确性 bug、安全问题或回归。

测试

本次为无人值守 CI 运行,无本地 tmux 测试。CI 证据:Ubuntu 测试在 npm audit 步骤失败——npm 注册表审计端点返回 400("该端点即将退役"),属于预存基础设施问题,与 PR 无关(PR 未修改 package.json)。测试套件未执行。macOS/Windows 测试被跳过。作者报告本地 macOS 上 300/300 测试通过(作者声明,非独立验证)。

未验证:单元测试结果(CI 在测试步骤前失败)、Windows/Linux 运行时行为。维护者可触发隔离的 @qwen-code /tmux 任务获取真实 TUI 覆盖。

Qwen Code · qwen3.8-max-preview

Reviewed at 09c6c1371d2bdfd99fb28a315c91afb54d5eb529 · re-run with @qwen-code /triage

@qwen-code-ci-bot

Copy link
Copy Markdown
Collaborator

Confidence: 4/5 — clean implementation that solves a real rendering bug with the right tool; docking one point because CI's npm audit infra failure means no independent test signal yet.

The problem is real and well-evidenced — issue #7740 shows three distinct failure modes on current main, all stemming from the regex recognizer's inability to model backslash parity. The scanner is the natural fix: parity is a counting operation, and the old regex's single-character lookbehind was the root cause. The mergeInlineMathMatches generator is a genuine improvement over the previous approach — it centralizes the markup/math interleaving that was previously handled by a combined regex, and all four consumers now share one recognition path instead of each reimplementing the priority logic.

Every change in the diff earns its place. The test additions cover the full corpus from the issue, backslash parity edge cases, code-span isolation, table splitting with escaped dollars, and the copy command. No scope creep, no drive-by refactors.

The one reservation: CI didn't produce test results. The Ubuntu job failed at npm audit (retiring endpoint, 400 Bad Request) before the test suite ran, and macOS/Windows were skipped via fail-fast. This is pre-existing infra noise — the PR doesn't touch package.json — but it means I'm relying on code review and the author's local report rather than independent CI verification. Once CI lands green, this is ready to ship.

Approval deferred until CI lands green on 09c6c1371d2bdfd99fb28a315c91afb54d5eb529.

中文说明

置信度:4/5 —— 干净的实现,用正确的工具解决了真实的渲染 bug;扣一分因为 CI 的 npm 审计基础设施故障导致尚无独立测试信号。

问题真实且有充分证据——issue #7740 展示了当前 main 上的三种不同失败模式,均源于正则识别器无法建模反斜杠奇偶性。扫描器是自然的修复方案:奇偶性是计数操作,旧正则的单字符向后查看是根因。mergeInlineMathMatches 生成器是对先前方案的真正改进——它集中了标记/公式交错逻辑,四个消费方现在共享一条识别路径。

diff 中每项改动都有存在价值。测试覆盖了 issue 中的完整语料、反斜杠奇偶边界、代码段隔离、含转义美元的表格拆分和复制命令。无范围蔓延,无顺手重构。

唯一保留:CI 未产生测试结果。Ubuntu 任务在 npm audit 步骤失败(退役端点,400 错误),测试套件未执行。这是预存基础设施问题,与 PR 无关。一旦 CI 全绿,即可合入。

审批已推迟至 CI 在 09c6c1371d2bdfd99fb28a315c91afb54d5eb529 上全绿后执行。

Qwen Code · qwen3.8-max-preview

Reviewed at 09c6c1371d2bdfd99fb28a315c91afb54d5eb529 · re-run with @qwen-code /triage

@CubeLander

Copy link
Copy Markdown
Contributor Author

Added a reproducible terminal-capture scenario and real TUI evidence using the same ModelStudio response corpus on both trees.

Before — merge-base 7959fdb27:

Before: escaped dollars remain raw

After — patched TUI:

After: escaped dollars render correctly

Both runs used qwen3.7-max-2026-06-08 and the identical seven-line prompt. The before tree leaves lines 4–7 raw; the patched tree removes the prose escape and renders all three escaped-dollar math boundary cases.

@qwen-code-ci-bot qwen-code-ci-bot left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reviewed. Not reviewed: build-and-test — Integration Tests (CLI, No Sandbox) was skipped in CI and its suite did not run locally.

— qwen3.7-max via Qwen Code /review

@doudouOUC doudouOUC left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No issues found. LGTM! ✅

— qwen3.7-max via Qwen Code /review

@gwinthis

Copy link
Copy Markdown
Collaborator

Independent local verification report (head cdb897e)

Verdict: the fix works as claimed. Verified on a real Linux TUI via tmux — all seven corpus lines render exactly as specified, and /copy inline-latex finds all four formulas.

1. Real-terminal evidence (patched TUI, tmux 120×40, Linux, Node v22.23.1)

Built this branch locally (npm install && npm run build && npm run bundle), launched dist/cli.js inside tmux, and had the model echo the seven-line corpus verbatim. Captured pane output:

x² is valid math
it costs $5 and $10
$5-$10
literal $x$
formula x + $5
literal then math: $x²
math then literal: x²$

Every line matches the PR's expected rendering: math renders, currency/range guards hold, the prose escape drops its backslash, and both boundary cases (\$$x^2$, $x^2\$$) split correctly. /copy inline-latex then reported "Inline LaTeX expression 4 copied" — all four formula sources recognized, versus one on main.

2. Baseline reproduction (main's recognizer, same corpus)

Ran origin/main's INLINE_MATH_PATTERN_SOURCE regex against the identical seven lines:

"$x^2$ is valid math"           => [ '$x^2$' ]
"literal \$x$"                  => NO MATH
"formula $x + \$5$"             => NO MATH
"literal then math: \$$x^2$"    => NO MATH
"math then literal: $x^2\$$"    => NO MATH

This independently confirms the stated defect: the old regex bans $ from the formula body and inspects only one preceding backslash, so lines 5–7 cannot be represented at all.

3. Code-path review

  • All five consumers (RenderInline, TableRenderer ANSI path, /copy inline-latex, splitMarkdownTableRow, getPlainTextLength) now funnel through the single mergeInlineMathMatches generator — one recognition result, no divergence.
  • The scanner preserves every prior guard: 1024-char bound (over-length caught by the post-loop content.length check), currency ((?=[\s\d$]) next-char reject), adjacent spans (following reject), exact-backtick code spans (markup wins ties at equal index).
  • unescapeMarkdownDollars handles backslash-run parity correctly (\$$, \\$\$, \\\$\$... floor(n/2) semantics); inline code branches bypass it, keeping code spans byte-preserving.
  • Renderer resets markupRegex.lastIndex before matchAll, avoiding shared-global-regex state leaks.

4. Test results on this branch (Linux)

  • inline-math.test.ts, InlineMarkdownRenderer.test.tsx, pending-rendered-height.test.ts, copyCommand.test.ts: 114/114 passed.
  • MarkdownDisplay.test.tsx: 182/186 — the 4 failures (getFrameController is not a function in screen-buffer.ts) reproduce identically on current origin/main (27927c4), so they are a pre-existing environment issue, not a regression from this PR.
  • Full npm run build + npm run bundle completed clean on this branch.

Conclusion

Thesis: this PR does exactly what it claims — completes the escaped-dollar contract via a parity-aware bounded scanner — and nothing more. Evidence: independent baseline reproduction (§2), real-TUI after-state (§1), consumer unification check (§3), and green focused suites (§4). The only test failures observed are pre-existing on main. LGTM from the Linux runtime side, which also closes the PR's stated "Linux not validated" gap.

中文摘要

在 Linux 真实终端(tmux 120×40,Node v22.23.1)独立构建并验证本 PR(head cdb897e):七行语料全部按预期渲染(公式渲染、货币守卫保持、转义美元去反斜杠、两个边界用例正确拆分),/copy inline-latex 识别出全部 4 个公式(main 上只有 1 个)。用 main 的旧正则独立复现了缺陷基线。聚焦测试 114/114 通过;MarkdownDisplay.test.tsx 的 4 个失败在 main 上同样复现,属存量环境问题,非本 PR 回归。结论:修复符合声明,且补上了 PR 自述未验证的 Linux 运行时。

— independent review loop, real-TUI verification on Linux

@wenshao

wenshao commented Jul 26, 2026

Copy link
Copy Markdown
Collaborator

Maintainer local verification — real build, real terminal (head cdb897e)

Verdict: merge-ready. I reproduced the defect and the fix on a real pty, then attacked the change from angles the earlier reports did not cover — a 1,586-input cross-consumer differential against the merge-base, a 13-mutant matrix over the new scanner, and a perf-scaling A/B. Nothing blocking. Three small test-coverage suggestions at the end, all optional (and all pre-existing on main).

Verified on macOS 26.5, Node v22.23.1, worktrees at head cdb897e and merge-base 0f56e35, node_modules from a checkout whose lockfile is byte-identical to the PR base.


1. Real-terminal before/after — production component, real pty

Rather than drive a model, I rendered the production <MarkdownDisplay /> with real Ink inside a real pty (@lydell/node-pty@xterm/xterm → Playwright, i.e. this repo's own integration-tests/terminal-capture harness), with the PR's seven-line corpus fixed as input. Deterministic, no API key, same component the CLI uses for model output. The /copy inline-latex list underneath is computed by the shared recognizer the copy command actually calls.

Before — merge-base 0f56e35:

before

After — PR head cdb897e:

after

All seven lines match the stated contract, and /copy inline-latex goes from 1 → 4 expressions, in document order, with \$ preserved in the copied TeX source (x + \$5, x^2\$).

2. No regression on the guard surface

Same harness, a corpus of everything the recognizer is supposed to refuse:

Before:

guards before

After:

guards after

Currency ($19.99, $750, $5-$10, costs $5), shell variables, inline code (`\$x$` stays byte-identical, in both prose and a table cell), CJK adjacency and table cells all behave identically or improve. The only differences are the intended ones: prose escapes lose their backslash, and the mixed table cell now renders x + $5.

3. Cross-consumer differential vs the merge-base

I ran 1,586 inputs (81 hand-written + 5 length-bound + 1,500 seeded-deterministic fuzz over a $ / \ / backtick / markup alphabet) through all four real production consumersRenderInline, TableRenderer's ANSI path, /copy inline-latex's findInlineMathExpressions, and splitMarkdownTableRow — plus getPlainTextLength, on both trees, and diffed the dumps.

Result Value
Recognition gained 42
Recognition lost 7 — every one a fuzz string of backtick/dollar soup
Same-count-but-different 3 — likewise fuzz only
Realistic-prose sentences that regressed 0 of 10
Exceptions thrown, either tree, any input 0
INLINE_MATH_MAX_CHARS behaviour identical: 1024 accepted, 1025 rejected on both
Currency / shell-var / adjacent-span / code-span guards no divergence on any realistic input

The 7 losses are cases like '\$\ 2x5`$`aa$值^' where a code span now wins the position it should have won all along, so I checked whether that is right rather than just rare:

Renderer ↔ copy agreement improved: 40 → 35 disagreements over the same corpus (9 fixed, 4 new, all four new ones fuzz). The change moves the two consumers closer together, which is what the PR claims.

Pre-existing gap worth a follow-up issue (not this PR): the 4 realistic residual disagreements are unchanged from main

**bold $x$**                    renderer: no math   /copy offers: x
*ital $x$*                      renderer: no math   /copy offers: x
[link $x$](http://example.com)  renderer: no math   /copy offers: x
https://example.com/$x$         renderer: no math   /copy offers: x

i.e. /copy inline-latex still offers formulas nested in bold/italic/link/URL that the renderer never rendered. Identical on the merge-base, so it is neither caused nor worsened here.

4. Mutation matrix over the new scanner — 13 mutants

Each mutant reverts exactly one guard, then runs the PR's own five suites.

Mutants Notes
Killed 7 backslash-run parity, preceding-$ guard, trailing-whitespace guard, adjacent-span guard, the 1024 bound, floor(n/2) unescape semantics, the unescapeMarkdownBeforeMath boundary sentinel
Provably equivalent 3 see below
Survived, non-equivalent 3 see below

The 3 equivalent mutants are dead code, confirmed by probing all 1,616 inputs for any observable difference (zero):

  • markupIndex <= mathIndex vs < — no markup alternative can start with $, so the tie never occurs.
  • markupRegex.lastIndex = 0String.prototype.matchAll clones the regex, so the reset can never matter. Harmless defensiveness, not a bug.
  • the !next end-of-string fast path — the closer loop already returns null there.

The 3 real survivors are guards that no test pins:

Guard Removing it makes…
preceding word char (/\w/.test(previous)) a$b$, y$x$, costs5$x$ become math
next char (/[\s\d$]/.test(next)) $ x$ becomes math; $$ yields an empty copy entry
newline (text[closingIndex] !== '\n') $x\ny$ spans lines

These were already untested on main. I mutated the equivalent regex assertions on the merge-base ((?<![\\\w$]), (?![\s\d$]), [^$\n]) and the base suites stayed 108/108 green in all three cases. So this PR neither introduces nor inherits blame for the gap — but it does move those guards from a declarative regex, where they are visually obvious, into imperative code where a future refactor could drop one silently. Cheap to close (see §7).

5. Performance — no new complexity class

The change swaps one bounded regex for a hand-written scanner, so I measured scaling rather than a single length. Five shapes (realistic prose, $-dense, escaped-$-dense, an adversarial longest-scan shape, and backtick soup), 2k → 32k chars:

Shape base ms @32k PR ms @32k PR/base base per-doubling PR per-doubling
prose 0.069 0.546 7.9× 2.01× 2.00×
dollar-dense 0.076 0.570 7.5× 1.98× 2.08×
escaped-dense 0.078 0.423 5.4× 1.99× 1.95×
longest-scan (adversarial) 0.077 0.802 10.4× 1.93× 2.02×
backtick soup 0.145 0.467 3.2× 1.95× 1.97×

Both trees are linear (~2.0× per doubling on every shape, including the adversarial one) — the scanner's inner loop is bounded by the distance to the next unescaped $, so it stays O(n) amortized. The PR pays a 3–10× constant factor, which at a realistic per-line size (~100 chars) is sub-microsecond. No ReDoS class introduced.

6. Test and gate results

Check Result
5 focused suites (inline-math, InlineMarkdownRenderer, MarkdownDisplay, pending-rendered-height, copyCommand) 300/300 passed — the PR's claim reproduces
Full packages/cli suite 15,356 passed, 3 skipped, 4 failed
Those 4 failures on the merge-base identical (extensions/list.test.ts, AuthDialog.test.tsx) — pre-existing, not regressions
prettier --check on changed files clean
eslint on changed files clean
tsc --noEmit -p packages/cli 2 TS6305 project-reference errors, identical count on the merge-base — my symlinked-worktree artifact, not the PR

Note for @gwinthis: the 4 MarkdownDisplay.test.tsx failures you saw do not reproduce here — 186/186 on both trees on macOS, so that looks environment-specific on your side rather than something latent.

7. Optional follow-ups (none blocking)

  1. Three one-line tests to pin the surviving guards: a$b$ stays prose; $ x$ stays prose and $$ yields no expression; $x\ny$ does not cross the newline. Pre-existing gap, but this PR is the natural place to close it since it rewrote those guards.
  2. Scope note: \$ is now unescaped in prose even when inline math is off (renderMode !== 'render') — 371 of my corpus inputs differ on that path. This is CommonMark-correct ($ is escapable ASCII punctuation) and I think it is the right behaviour, but it is broader than the PR title suggests and no test covers the math-off path. Worth one test and a line in the description.
  3. Follow-up issue for the pre-existing renderer/copy disagreement in §3 (math nested in bold/italic/link/URL).
中文版

维护者本地验证 —— 真实构建、真实终端(head cdb897e

结论:可以合并。 我在真实 pty 中复现了缺陷与修复,并从此前报告未覆盖的角度进行了攻击:针对 merge-base 的 1,586 条输入跨消费者差分、对新扫描器的 13 个变异体矩阵、以及性能伸缩性 A/B。没有阻塞项。文末三条测试覆盖建议均为可选,且在 main 上已经存在同样的缺口。

环境:macOS 26.5、Node v22.23.1,worktree 分别位于 head cdb897e 与 merge-base 0f56e35node_modules 取自 lockfile 与 PR base 完全一致的 checkout。

1. 真实终端前后对比 —— 生产组件 + 真实 pty

我没有驱动模型,而是用真实 Ink 在真实 pty 中渲染生产组件 <MarkdownDisplay />@lydell/node-pty@xterm/xterm → Playwright,即本仓库自带的 integration-tests/terminal-capture 工具链),输入固定为 PR 的七行语料。确定性复现,无需 API key,且用的正是 CLI 渲染模型输出的同一组件。下方 /copy inline-latex 列表由 copy 命令实际调用的同一识别器计算。

七行全部符合声明的契约,/copy inline-latex1 条增加到 4 条,按文档顺序排列,复制出的 TeX 源码保留了 \$x + \$5x^2\$)。

2. 守卫面无回归

同一工具链,语料为识别器应当拒绝的全部情况:货币($19.99$750$5-$10costs $5)、shell 变量、行内代码(`\$x$` 在普通文本与表格单元格中均逐字节保持)、CJK 邻接、表格单元格,行为全部相同或更好。唯一的差异正是预期内的:普通文本转义去掉反斜杠,mixed 单元格现在渲染为 x + $5

3. 与 merge-base 的跨消费者差分

1,586 条输入(81 条手写 + 5 条长度边界 + 1,500 条确定性 fuzz,字母表覆盖 $ / \ / 反引号 / markdown 标记),跑过全部四个真实生产消费者RenderInlineTableRenderer 的 ANSI 路径、/copy inline-latexfindInlineMathExpressionssplitMarkdownTableRow)以及 getPlainTextLength,两棵树各跑一遍后对比。

结果 数值
新增识别 42
丢失识别 7 —— 全部是反引号/美元符号乱码的 fuzz 串
数量相同但内容不同 3 —— 同样只出现在 fuzz
真实语句中的回归 10 条中 0 条
任一树、任一输入抛异常 0
INLINE_MATH_MAX_CHARS 行为 完全一致:1024 接受、1025 拒绝
货币 / shell 变量 / 相邻 span / 代码段守卫 真实输入上无任何差异

那 7 条丢失都是代码段现在赢得了本就该赢的位置,因此我进一步确认了方向是否正确:渲染器与复制路径的分歧从 40 降到 35(修好 9 条,新增 4 条且全为 fuzz)。这正是 PR 所声称的"两个消费者共享同一识别结果"。

值得单独开 issue 的存量问题(不属于本 PR): 4 条真实的残余分歧与 main 完全一致 —— **bold $x$***ital $x$*[link $x$](url)https://example.com/$x$ 中,渲染器不渲染公式,但 /copy inline-latex 仍然把它们列为可复制项。merge-base 上表现相同,既非本 PR 引入,也未被本 PR 恶化。

4. 新扫描器的变异矩阵 —— 13 个变异体

每个变异体只回退一个守卫,然后运行 PR 自带的五组测试。

  • 被杀死 7 个:反斜杠奇偶性、前置 $ 守卫、尾随空白守卫、相邻 span 守卫、1024 上限、floor(n/2) 反转义语义、unescapeMarkdownBeforeMath 的边界哨兵。
  • 可证明等价 3 个(对全部 1,616 条输入探测,零可观测差异,属于无害的防御性死代码):markupIndex <= mathIndex<(没有任何 markup 分支能以 $ 开头,平局不可能发生);markupRegex.lastIndex = 0matchAll 会克隆正则,重置永远不起作用);!next 的字符串末尾快速返回(闭合循环本就会返回 null)。
  • 真实存活 3 个(没有任何测试钉住):前置单词字符守卫(去掉后 a$b$y$x$costs5$x$ 会变成公式);后继字符守卫(去掉后 $ x$ 变成公式,$$ 产生一条空的复制项);换行守卫(去掉后 $x\ny$ 会跨行)。

这三个缺口在 main 上已经存在。 我在 merge-base 上对等价的正则断言((?<![\\\w$])(?![\s\d$])[^$\n])做了同样的变异,base 的测试三次都是 108/108 全绿。所以本 PR 既没有引入这个缺口,也不该为此背锅;但它确实把这些守卫从一眼可见的声明式正则搬进了命令式代码,未来重构时更容易被悄悄删掉。补上成本很低(见第 7 节)。

5. 性能 —— 没有引入新的复杂度类

本次把一个有界正则换成了手写扫描器,因此我测的是伸缩性而非单点耗时。五种形状(真实文本、$ 密集、转义 $ 密集、对抗性最长扫描、反引号乱码),2k → 32k 字符:

形状 base @32k PR @32k 倍率 base 每翻倍 PR 每翻倍
真实文本 0.069 ms 0.546 ms 7.9× 2.01× 2.00×
$ 密集 0.076 ms 0.570 ms 7.5× 1.98× 2.08×
转义密集 0.078 ms 0.423 ms 5.4× 1.99× 1.95×
对抗性最长扫描 0.077 ms 0.802 ms 10.4× 1.93× 2.02×
反引号乱码 0.145 ms 0.467 ms 3.2× 1.95× 1.97×

两棵树都是线性的(所有形状包括对抗形状均为每翻倍约 2.0×)—— 扫描器的内层循环被"到下一个未转义 $ 的距离"所界定,因此摊还仍是 O(n)。PR 付出 3–10× 的常数因子,按真实单行长度(约 100 字符)折算是亚微秒级。未引入 ReDoS 风险。

6. 测试与门禁

检查 结果
五组聚焦测试 300/300 通过,PR 的说法可复现
packages/cli 全量 15,356 通过、3 跳过、4 失败
这 4 个失败在 merge-base 上 完全相同extensions/list.test.tsAuthDialog.test.tsx),属存量问题,非回归
改动文件 prettier --check 通过
改动文件 eslint 通过
tsc --noEmit -p packages/cli 2 个 TS6305 项目引用错误,merge-base 上数量完全相同 —— 是我 symlink worktree 的环境产物,与 PR 无关

补充给 @gwinthis:你看到的 4 个 MarkdownDisplay.test.tsx 失败在我这里没有复现 —— macOS 上两棵树均为 186/186,所以更可能是你那侧的环境问题,而非潜在缺陷。

7. 可选后续(均不阻塞合并)

  1. 三条一行测试钉住存活的守卫:a$b$ 保持普通文本;$ x$ 保持普通文本且 $$ 不产生表达式;$x\ny$ 不跨行。缺口虽是存量,但本 PR 重写了这些守卫,是补齐的自然时机。
  2. 范围提示: 即使行内公式关闭renderMode !== 'render'),普通文本中的 \$ 现在也会被反转义 —— 我的语料中有 371 条输入在该路径上产生差异。这符合 CommonMark($ 属于可转义的 ASCII 标点),我认为行为是对的,但它比 PR 标题暗示的范围更广,且没有测试覆盖关闭公式的路径。建议补一条测试并在描述里加一句。
  3. 为第 3 节中的存量渲染器/复制分歧(嵌套在 bold/italic/link/URL 中的公式)单独开一个 issue。

— maintainer local verification: real pty + Ink MarkdownDisplay, 1,586-input cross-consumer differential, 13-mutant matrix, perf-scaling A/B

@wenshao
wenshao added this pull request to the merge queue Jul 26, 2026
Merged via the queue into QwenLM:main with commit 7422bf8 Jul 26, 2026
62 checks passed
@qwen-code-ci-bot

Copy link
Copy Markdown
Collaborator

Released in v0.21.1.

@QwenLM QwenLM deleted a comment Aug 6, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

review/self-reported The linked issue was opened by the PR author (self-reported)

Projects

None yet

Development

Successfully merging this pull request may close these issues.

fix(cli): handle escaped literal dollars around inline math

6 participants