Skip to content

feat: block-level and inline markdown rendering for LLM responses - #3

Closed
KUSH42 wants to merge 1 commit into
feat/tool-output-highlightingfrom
feat/rich-output-renderer
Closed

feat: block-level and inline markdown rendering for LLM responses#3
KUSH42 wants to merge 1 commit into
feat/tool-output-highlightingfrom
feat/rich-output-renderer

Conversation

@KUSH42

@KUSH42 KUSH42 commented Apr 1, 2026

Copy link
Copy Markdown
Owner

PR: block-level markdown rendering

⚠️ Stacked on feat/rich-output-renderer (inline markdown PR) — merge that first.

Branch: feat/markdown-block-rendering

Summary

  • Adds apply_block_line(line) — single-line-detectable block elements rendered to ANSI
  • Extends apply_inline_markdown with images, links, and HTML inline tags
  • Chains apply_block_line before apply_inline_markdown in both format_response and the streaming path

What's rendered

Element Input Output
h1 # Foo bold bright-white, # stripped
h2 ## Foo bold white
h3 ### Foo bold
h4–h6 #### Foo bold dim
Horizontal rule --- / *** / ___ dim line across terminal
Blockquote > text dim gutter + dim text
Nested blockquote >> text collapses to single
Unordered list - item at depth 0, at 1, at 2, · at 3
Ordered list 1. item unchanged (already readable)
Reference link def [ref]: https://… suppressed (metadata, not prose)
Image ![alt](url) [img: alt] dim placeholder
Link [text](url) underlined text, URL discarded
<em> <em>text</em> italic, tags stripped
<strong> <strong>text</strong> bold, tags stripped

Architecture

apply_block_line is purely line-local — no lookahead or state. Two early-exit guards:

  • "\x1b" in line — already ANSI-rendered, skip
  • "\n" in line — multi-line block from StreamingBlockBuffer (PR4), skip

Headings and blockquotes apply inline spans internally (with reset_suffix set to the heading/gutter style) so a closing \033[0m from a span doesn't kill the outer colour.

List lines do not inject ANSI, so the outer apply_inline_markdown call handles their inline spans naturally.

format_response(text):
  pass 1: fenced code blocks → ANSI highlighted   (existing)
  pass 2: per non-ANSI line:
            apply_block_line(line) → apply_inline_markdown(result)

Streaming (_emit_stream_text / _flush_stream):
  out is line → apply_inline_markdown(apply_block_line(line), reset_suffix=_tc)

Files changed

  • agent/rich_output.pyapply_block_line, extended apply_inline_markdown, updated format_response; removed dead _MD_OL_RE; fixed _MD_REF_LINK_RE to suppress titled reference-link definitions
  • cli.py — import apply_block_line, chain in _emit_stream_text and _flush_stream
  • tests/test_rich_output.pyTestApplyBlockLine (23 tests), 5 new TestApplyInlineMarkdown cases, 3 new TestFormatResponseInlineMarkdown cases

Bug fixes included

  • format_response newline losssplitlines(keepends=True) fed newline-bearing strings into apply_block_line whose capture groups stop at \n; matched block elements (heading, hr, blockquote, list) were returned without \n and concatenated with the following line. Fixed by switching to splitlines() + manual "\n".join with trailing-newline restoration.
  • Blockquote reset_suffix — inline spans inside blockquotes were resetting to terminal default instead of restoring the dim gutter style. Fixed by passing reset_suffix=_BLOCKQUOTE_ANSI to apply_inline_markdown in the blockquote branch (mirrors the existing heading branch pattern).
  • _MD_REF_LINK_RE titled forms — the $ anchor prevented suppression of reference-link definitions that include a title ([ref]: url "Title", [ref]: url (Title)). Removed $; the URL \S+ is sufficient to distinguish ref-links from prose.

Known interim regression

--- immediately after a paragraph renders as an hr instead of a setext h2. Corrected in PR4 (StreamingBlockBuffer intercepts the setext case before apply_block_line sees it; see docs/spec-markdown-stateful-blocks.md). Setext headings are rare in LLM output; the interim behaviour is acceptable.

Non-goals (deferred to PR4)

  • Setext headings (=== / --- underline)
  • Tables
  • Multi-line blockquote continuation (no > prefix on continuation lines)

@KUSH42
KUSH42 marked this pull request as draft April 1, 2026 20:58
@KUSH42
KUSH42 force-pushed the feat/tool-output-highlighting branch from d9ecc84 to 47ce516 Compare April 1, 2026 21:41
@KUSH42
KUSH42 force-pushed the feat/rich-output-renderer branch from 83c49e9 to 8a7fad8 Compare April 1, 2026 21:48
@KUSH42
KUSH42 force-pushed the feat/tool-output-highlighting branch from d3045a2 to 47ce516 Compare April 1, 2026 21:49
…responses

Adds apply_block_line (headings, hr, blockquotes, lists, ref-link
suppression) and extends apply_inline_markdown (images, links, HTML
inline tags).  format_response gains a pass-2 that applies both
renderers to every non-highlighted line.  The streaming path chains
the same pair in _emit_stream_text and _flush_stream.

Bug fixes included:
- _code_highlight_active was gating apply_block_line/apply_inline_markdown
  in the streaming path; removed the guard (_RICH_RESPONSE is the correct
  gate; _code_highlight_active controls only tool-output highlighting)
- _MD_ITALIC_UNDER_RE rejected phrases with spaces; changed [^_\s\n]+ to
  [^_\n]+ (word-boundary lookbehind prevents snake_case false positives)
- _MD_REF_LINK_RE had a $ anchor that blocked titled reference-link
  definitions from being suppressed; removed $
- Blockquote inline spans reset to terminal default; added reset_suffix
  to the apply_inline_markdown call in the blockquote branch
- format_response splitlines(keepends=True) fed \n into capture groups,
  silently dropping block elements; switched to splitlines() + manual join
- CommonMark backslash escapes (\] → ]) were passed through literally;
  added step-7 re.sub pass in apply_inline_markdown
- Pygments plain-text lexer emits lines with no ANSI codes; pass-2 guard
  "\x1b" in l then fell through and applied markdown to code-fence content
  (### was stripped, ** rendered, etc.); _highlight now prepends \x1b[0m
  to bare lines so the guard is always satisfied
@KUSH42
KUSH42 force-pushed the feat/rich-output-renderer branch from 8a7fad8 to eff70ff Compare April 1, 2026 21:49
@KUSH42

KUSH42 commented Apr 1, 2026

Copy link
Copy Markdown
Owner Author

Superseded by NousResearch#4501

@KUSH42 KUSH42 closed this Apr 1, 2026
KUSH42 pushed a commit that referenced this pull request May 10, 2026
## Summary
- Forwards chat-completions `timeout` into the Codex Responses stream call.
- Adds total elapsed-time enforcement while the Responses stream is still yielding events.
- Closes the underlying client on timeout to unblock stalled streams, then raises `TimeoutError`.
- Adds focused tests for timeout forwarding and total timeout enforcement.

## Why
The Codex auxiliary adapter can be used by non-interactive auxiliary work such as context compression. If the stream keeps yielding progress-like events but never completes, SDK socket/read timeouts do not necessarily protect the full operation. This makes the CLI look stuck until the user force-interrupts the whole session.

This is a refreshed upstream-ready version of the earlier fork fix around `d3f08e9a0` / PR #3.

## Verification
- `python -m py_compile agent/auxiliary_client.py tests/agent/test_auxiliary_client.py`
- `python -m pytest -o addopts='' tests/agent/test_auxiliary_client.py::TestCodexAuxiliaryAdapterTimeout -q`
- `git diff --check`
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant