feat(tui): improve fenced code block readability

[nicezic]

What does this PR do?

Fenced code blocks in the Ink TUI previously relied on indentation and a small language separator, so code could blend into surrounding prose in long technical responses.

This PR gives fenced code a restrained, theme-aware panel at normal widths:

╭─ python ────────────────────────────╮
│ def hello():                        │
╰─────────────────────────────────────╯

At widths below 20 columns, and whenever compact mode is active, it falls back to a left accent and compact language label. That keeps useful code width and avoids making a four-sided outline dominate heavily wrapped content.

The follow-up commit 695df6881 (on top of the previous two) addresses the final-review findings:

1. md / markdown fence height. The renderer recurses those fences through <Md cols={cols}> instead of painting the rounded CodeBlock panel, so the body is prose at the full body width and the panel's top + bottom border rows don't apply. estimateBodyHeight now stores the normalized lang string and, when it is md or markdown, counts the body at bodyWidth with no chrome rows. Closed and unclosed fences both go through this branch. The renderer is unchanged.
2. Real regex sharing. The renderer had its own local FENCE_RE / FENCE_CLOSE_RE; codeBlockLayout.ts had its own FENCE_OPEN_RE / FENCE_CLOSE_RE. Both call sites now use the shared exports from ui-tui/src/lib/codeBlockLayout.ts. The renderer's local copies are removed.
3. Trim codeBlockLayout.ts. Dropped FenceMatch, isFenceOpenLine, fenceLangOf (none had callers) and fenceWrapWidth (a one-line wrapper around innerContentWidth that the estimator now calls directly). The module is down from 145 to 103 lines with the long comments compressed.

The earlier commit 291c9422c covers the original two review blockers:

• CodeBlock was passing the raw lang string to Ink borderText. Ink's border-embedding path (packages/hermes-ink/src/ink/render-border.ts::embedTextInBorder) takes a JS-substring fallback whenever stringWidth(text) >= borderLength - 2, which corrupts a CJK / emoji / mixed label mid-glyph. CodeBlock now derives a displayLang via a grapheme-safe truncateToWidth (Intl.Segmenter with Array.from fallback) and uses that for both the border label and the narrow / compact language row. The original lang is preserved for syntax-highlight detection.
• virtualHeights.ts did not account for the panel's chrome rows or reduced content width. A source line that fits at the body width can wrap inside the panel's narrower inner width, so the original estimator undercounted and the virtual transcript spacer snapped on first mount. estimateBodyHeight walks the source linearly (no split, no pre-built fence-span list), recognizes the same backtick / tilde fence forms as the renderer, and counts each fence body at the panel's inner width plus the panel's chrome rows. The walk is bounded by the same MAX_ESTIMATE_LINES cap the prose estimator uses, so a 1M-char single line still returns in O(width) time.

Related issue

Closes #75781

Related to #12130 and #46905. Builds on the width-safe TUI rendering discussion in #15534 and #17114. Adjacent but non-duplicative work includes #71849 and #48095. PRs #5617 and #75326 address decorated fenced-code rendering in the classic Python interactive CLI rather than this Ink TUI renderer.

Type of change

• Bug fix
• New feature
• Security fix
• Documentation update
• Tests
• Refactor
• New skill

Changes made

• Adds a small CodeBlock renderer inside ui-tui/src/components/markdown.tsx.
• Uses t.color.border and t.color.muted; no hard-coded colors or new theme token.
• Places the language identifier in the top border at normal widths.
• Uses a left-only accent below 20 columns and in compact mode.
• Gives code lines wrap="wrap-char", so long unbroken source stays within the allocated Markdown body width.
• Preserves the existing syntax-highlighted token spans and diff added/removed/hunk styling.
• Keeps decoration outside the source strings. This does not alter Markdown parsing or clipboard infrastructure; feat(tui): add exact code block copying #71849's raw-fence copying approach remains compatible because display children and raw copied content are separate.
• Extends the visual harness with Python, diff, narrow-width, Korean, and emoji examples in dark/light default and slate themes.
• Adds ui-tui/src/lib/codeBlockLayout.ts (103 lines after the final-review trim) — a small shared module that holds CODE_PANEL_MIN_WIDTH, the isNarrowPanel / innerContentWidth / borderLabelWidth / chromeRows helpers, the grapheme-safe truncateToWidth, and the FENCE_OPEN_RE / FENCE_CLOSE_RE regexes that the renderer and the virtual-height estimator both import.
• Updates ui-tui/src/lib/virtualHeights.ts with estimateBodyHeight that recognizes fenced blocks in a single linear pass, counts their body rows at the panel's inner width (or at the full body width for md / markdown fences), and adds the matching chrome rows (or none for md / markdown fences).
• Keeps the original lang for syntax-highlight detection; only the displayed label is truncated.

Design notes

A full outline was retained for normal widths because the current @hermes/ink border renderer subtracts visible left and right border cells from its measured Yoga width, and the CodeBlock receives cols, the actual transcript-body allocation, rather than reading the full terminal width. Its one-cell horizontal padding is consequently part of the same width budget.

The table precedent in #17114 deliberately avoided a full outline when display-width measurement was unreliable. The current TUI now uses its own stringWidth implementation and the focused tests exercise Korean and emoji widths. Even so, a full outline costs four horizontal cells after borders and padding, so constrained and compact layouts deliberately use only a left accent.

The width budget for the trimmed borderText content is cols - 5: the two corner cells, the two surrounding spaces inside the border (${label}), and one cell for the leading ─ after the ╭. The truncation grapheme-walks the original label with Intl.Segmenter(undefined, { granularity: 'grapheme' }) (falling back to Array.from for environments without the Segmenter) and includes the ellipsis in the budget so stringWidth(result) <= borderLabelWidth(cols) always holds.

For the virtual-height estimator, the renderer-facing chrome rules are mirrored exactly:

• normal panel: bodyWidth - 4 content width, 2 chrome rows (top + bottom border);
• narrow / compact: bodyWidth - 2 content width, hasLang ? 1 : 0 chrome rows (language row only, no border);
• md / markdown fence: body is recursed with <Md>, so the estimator counts it as prose at the full body width with no chrome rows.

The opener / closer fence lines themselves are not counted as visible rows, matching the renderer. Empty fences still receive a minimum 1 code row so the renderer-side <Text> </Text> placeholder isn't undercounted.

Streaming remains unchanged: StreamingMd continues to hold open fences in its mutable tail and freezes only complete Markdown blocks. The same Md renderer draws partial and completed fenced blocks, so there is no second streaming-specific style path.

No user-facing configuration was added because this is a focused rendering treatment with a deterministic narrow-width fallback.

How to test

cd ui-tui
npm run build:ink
npm test -- --run \
  src/__tests__/markdown.test.ts \
  src/__tests__/streamingMarkdown.test.ts \
  src/__tests__/syntax.test.ts \
  src/__tests__/virtualHeights.test.ts
npm run typecheck
npx eslint \
  src/components/markdown.tsx \
  src/lib/virtualHeights.ts \
  src/lib/codeBlockLayout.ts \
  src/__tests__/markdown.test.ts \
  src/__tests__/virtualHeights.test.ts \
  scripts/visual/render.tsx
npm run visual
npm run build
git diff --check

Validation results

• npm run build:ink — clean (esbuild, dist/entry-exports.js 433.8kb).
• npm test --run src/__tests__/markdown.test.ts src/__tests__/streamingMarkdown.test.ts src/__tests__/syntax.test.ts src/__tests__/virtualHeights.test.ts — 64 / 64 passed in the two changed test files (markdown.test.ts 44, virtualHeights.test.ts 20). The +1 over the prior 63 is the new md / markdown regression.
• npm test --run (full TUI suite) — 1,464 / 1,480 passed, 8 skipped, 8 failed. The 8 failures reproduce identically on clean upstream main (commit 3572d4bca) and are in editor.test.ts, terminalParity.test.ts, and terminalSetup.test.ts — all environment-dependent (PATH editor lookups, VS Code config dir detection) and unrelated to this PR.
• npm run typecheck — clean.
• npx eslint on the touched files — 0 errors, 0 warnings.
• npm run visual — wrote Z:\Temp\hermes-tui-visual\tui-visual.html and tui-visual.png; standard inputs (Python / diff / narrow Korean+emoji / default + slate themes on dark + light) unchanged, so the existing after.png is not regenerated.
• npm run build — dist/entry.js 3.5mb.
• git diff --check — clean.

CI

gh pr checks 75783 currently reports 1 check in action_required on the current head 695df6881. This is the fork workflow approval gate — the upstream ci.yml workflow is configured to require explicit maintainer approval for runs on a contributor fork, and a first-time contributor is gated behind that approval. It is not a test failure introduced by either of the fix commits. Once a maintainer approves the workflow run the checks proceed normally.

Tests added

ui-tui/src/__tests__/markdown.test.ts (in the fenced code panels describe block):

1. Long Korean language label at the 20-col threshold — header fits, label ends with …, stringWidth(label) <= 15.
2. Emoji-containing label (mixed CJK + emoji + ASCII) — no U+FFFD replacement char, every line stringWidth(line) <= width.
3. Mixed Korean / emoji / ASCII label that exceeds the panel — no U+FFFD, every line within width.
4. Label wider than the entire panel — header still forms a closed ╭…╮, contains …, no U+FFFD.
5. No broken surrogate / no replacement char verified in every test that uses a CJK or emoji label.
6. lines.every(line => stringWidth(line) <= width) invariant asserted at widths 19 (narrow), 20 (normal threshold), 21 (normal).

ui-tui/src/__tests__/virtualHeights.test.ts:

• A 30-char source line at body width 30, normal panel — estimator returns 4 (2 wrapped code rows + 2 chrome).
• bodyWidth - 4 width proof via the 22-char line at body width 24 (normal: 2 + 2 = 4) and body width 18 (narrow: 2 + 1 = 3).
• Top + bottom panel rows included in normal mode.
• Narrow fallback for body width 18.
• Compact mode (bodyWidth 30, compact: true) routes through narrow layout regardless of width.
• No-language fence: chrome is 0 in narrow mode and 2 in normal mode.
• Mixed prose + fence source — every non-fence line counts via the existing prose formula so existing non-fence behavior is unchanged.
• 1M-char single-fence body — estimator returns <= 800 and runs in under 50 ms.
• Empty fence — at least 1 code row + chrome.
• Unclosed fence — treated as code to end-of-text, mirroring the renderer.
• The full estimatedMsgHeight(msg, 35, { compact: false, details: false }) regression on a 30-char source line — must be >= 4 (was 3 before).
• md / markdown fence regression: a 30-char source line at body width 30 returns 1 row for md / markdown (full body width, no chrome) and 4 rows for python (panel path). Same body at narrow width 18 returns 2 rows for md and the panel path still produces the expected count.

Files changed

• ui-tui/src/lib/codeBlockLayout.ts (new, 103 lines after the final-review trim)
• ui-tui/src/components/markdown.tsx (modified; now imports FENCE_OPEN_RE / FENCE_CLOSE_RE from the shared module)
• ui-tui/src/lib/virtualHeights.ts (modified; estimateBodyHeight with md / markdown aware chrome and width)
• ui-tui/src/__tests__/markdown.test.ts (modified; 6 width-safe label regressions)
• ui-tui/src/__tests__/virtualHeights.test.ts (modified; 8 fence-aware regressions + 1 md / markdown regression + the 1M-char giant-fence performance cap)

The pre-existing entries in this PR remain as well:

• ui-tui/scripts/visual/render.tsx (visual harness extension for the new code-block scene)
• .github/pr-screenshots/75783/before.png and after.png (visual fixtures)
• The renderer additions to ui-tui/src/components/markdown.tsx and the existing renderer tests in ui-tui/src/__tests__/markdown.test.ts (kept; the fix commits only add width-safe label handling, md / markdown fence handling, and additional regression tests on top of them).

Commits on this branch

1. 34c54c3b4 feat(tui): improve fenced code block readability
2. 77a67cf25 docs(tui): add code block visual comparison
3. 291c9422c fix(tui): make fenced code panels width-safe — width-safe border label + fence-aware virtual-height estimator (blockers 1 and 2 from the first review pass)
4. 695df6881 fix(tui): align estimator with md/markdown fence recursion — md / markdown fence handling, real regex sharing, codeBlockLayout.ts trim (final-review follow-ups)

The branch is rebased onto current upstream main (3572d4bca); pushing was done with --force-with-lease.

Checklist

Code

• Read the contributing guide
• Conventional commit format
• Searched open and closed issues/PRs; no exact duplicate found
• Focused changes only
• Added behavior-level renderer tests
• Tested on WSL2

Documentation and housekeeping

• User documentation: not applicable; no config or command changed
• cli-config.yaml.example: not applicable
• CONTRIBUTING.md / AGENTS.md: not applicable
• Cross-platform and Unicode width behavior considered
• Tool schemas: not applicable

Screenshots

Before

After

Both sheets use the same examples and cover normal-width Python, diff, narrow Korean/emoji wrapping, and default/slate themes on dark/light terminal backgrounds.

[nicezic]

Visual validation artifacts were generated from the same harness on WSL2:

• before: /tmp/code-block-before.png
• after: /tmp/code-block-after.png

The after sheet covers normal-width Python, diff, narrow Korean/emoji wrapping, and default/slate themes on dark/light terminal backgrounds. GitHub CLI does not support uploading local image attachments directly, so these paths are recorded for manual attachment to the PR description.

[teknium1]

Thanks for the focused TUI treatment and the width-oriented renderer coverage. The main idea addresses a real current-main gap: fenced code is still rendered as an indented column in ui-tui/src/components/markdown.tsx:812-848.

Problems

• ui-tui/src/components/markdown.tsx:584 sends arbitrary fence-info text to borderText. Ink detects width with stringWidth, but its overflow path truncates with substring (ui-tui/packages/hermes-ink/src/ink/render-border.ts:43-45); wide CJK/emoji labels can therefore write past a 20-column panel. Please display-width-truncate the label and cover wide labels, not only wide code content.
• ui-tui/src/components/markdown.tsx:897 adds a bordered, four-cells-narrower code body, while ui-tui/src/lib/virtualHeights.ts:106-116 continues estimating raw text at full body width with no fenced-panel adjustment. Add a width-aware fenced-code estimate and regression coverage so virtual transcript spacers are correct before Yoga convergence.

Suggested changes

• Add CJK/emoji fence-info tests at the normal-width threshold.
• Add a virtual-height test for a long unbroken fenced line.

Automated hermes-sweeper review.

[GottZ]

This was generated by AI during triage.

Summary

One PR addresses Issue #75781. #75783 replaces indented fenced-code rendering with a theme-aware rounded panel, adds a narrow/compact left-accent fallback, preserves syntax and diff coloring, and aligns wrapping and virtual-height estimation with the panel layout.

Related pull requests

• feat(tui): improve fenced code block readability #75783 best fix — (+691/-47) — salvageable fix: feat(tui): improve fenced code block readability #75783 directly addresses the reported lack of visual separation and now pre-truncates CJK/emoji fence labels by display width, shares fence/layout calculations, and adds fence-aware virtual-height estimation with targeted regressions. This follows the contributor keep_open review on feat(tui): improve fenced code block readability #75783: the current diff addresses its two documented width-safety and virtualization findings, but no subsequent contributor review confirms them as cleared.

Suggested consolidation

Keep #75783 open with a salvage path: retain the CodeBlock treatment, shared width helpers, fence-aware height estimator, and Unicode/narrow-layout regressions, then obtain contributor re-review of the changes addressing the visible keep_open review and validate the remaining issue requirements for streaming and exact copy semantics. There are no duplicate PRs to close.

Complex graph

flowchart LR
    classDef open fill:#dbeafe,stroke:#1d4ed8,color:#1e3a8a
    classDef merged fill:#dcfce7,stroke:#15803d,color:#14532d
    classDef closed fill:#e5e7eb,stroke:#6b7280,color:#1f2937
    classDef unverified fill:#f3f4f6,stroke:#9ca3af,color:#374151
    classDef best stroke-width:3px,stroke:#b45309
    classDef target stroke-width:3px,stroke:#4338ca
    I75781(["issue #75781 (open)"])
    P75783["PR #75783 (open)"]
    P75783 -->|best fix| I75781
    class I75781 open
    class P75783 open
    class P75783 best
    class P75783 target
    click I75781 "https://github.com/NousResearch/hermes-agent/issues/75781"
    click P75783 "https://github.com/NousResearch/hermes-agent/pull/75783"

Loading

Graph: solid arrow = fixes / best fix, dashed arrow = partial or unverified (see edge label); boxed group = PRs duplicating each other; amber border = best fix; indigo border = target; gray node = closed (state tag in the node label).

Cross-PR triage: Reviewed 1 pull request and 1 issue in this complex. Each diff was read against this issue; Assessment working set: 37 kB of PR diffs, 18 kB of issue/PR text, 9 kB of discussion (7 comments), 3 verify verdicts. verdicts reflect diff content, not PR titles. Part of an automated triage batch.

[nicezic]

Follow-up on the automated triage note:

• Re-ran the focused Markdown, streaming, syntax, and virtual-height suites from a clean worktree at HEAD 695df6881.
• All four files passed individually, and the combined 90-test suite passed in three consecutive clean runs.
• Partial and completed fences continue through the existing StreamingMd / Md rendering path.
• The panel does not modify the source Markdown or fenced-code strings, so source-based message and code-copy paths remain unchanged.
• Terminal visual selection is WYSIWYG and reads rendered Ink cells; selecting panel chrome can therefore include the border or language row. I am calling this out explicitly for reviewer confirmation rather than claiming visual-selection output is undecorated.
• The two original width and virtualization review threads are resolved.

An earlier run from a non-clean local environment was not reproducible in the clean worktree.

No code changes were made in this follow-up. A contributor re-review of the current head would be appreciated.