Repository navigation
fix(ui): render dollar math next to punctuation without matching prices - #53860
Merged
Merged
Conversation
3 of 6 tasks
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Problem
marked-katex-extensiononly opens$...$at the start of the text or after a space, and only closes it before whitespace or? ! . , :. Agents routinely put math against punctuation, so text such asack ($440\text{ ms}$ IPC p50, …)renders as raw LaTeX. The same rule also rejectsthe $x$-axis,is $x$; y, and因此$x^2$是正数.Its inline
startalso retries the regex from each$on every text token, and that regex can scan to the end of the line. Paragraphs with many$therefore take seconds or minutes to lex. This happens in both the markdown worker and the live streaming lexer.Change
Replace the extension with three small marked extensions in
marked-parser.tsx, and drop themarked-katex-extensiondependency.$...$follows Pandoc'stex_math_dollarsboundaries:$must be followed by a non-space.$must follow a non-space, and must not be followed by a letter, digit, or underscore.$does not matter.$HOME/$USERand$PATH:$HOMEas text. The cost is that$n$thdoes not render, which is already true today.$$...$$renders as display math anywhere in a line.\(...\)keeps its existing tokenizer.$$/$blocks keep the extension's block regex unchanged.Now renders:
($440\text{ ms}$),(i.e. $x$),the $x$-axis,is $x$; y,$x$'s,因此$x^2$是正数,($$x$$)Stays text:
$5 and $10.,$5-$10,$5, originally $10,$5 (or $6),$5/$6.,between $5 and $6 dollars$HOME and $PATH,$HOME/$USER,$PATH:$HOME`$x$`\$x$$ x $(whitespace inside the delimiters is now rejected, as in Pandoc)Regex performance
Single match attempt
\\.starts with a backslash and the other class excludes it. There are no nested quantifiers, and the trailing unit is tried once per position, so backtracking is linear.$...$content cannot contain an unescaped$. A failed attempt stops at the next$.$$...$$attempt stops at the end of the line. Any later reachable$$on that line would close it successfully.recheckreports the new regex as safe.Across a paragraph
start(src)again for every text token, passing the rest of the paragraph. The extension'sstartretried the regex from every$in that remainder, which is O(tokens ×$count × line length).startnow returns the next$, like the existing\(extension. When the tokenizer fails there, marked merges the text tokens back together.start, across 300k fuzzed strings and every.mdfile in this repo andanomalyco/anomaly.Benchmark
Median lexer time with Bun 1.4.2 and the repo's marked:
none: marked with only the\(extensionold:marked-katex-extension5.1.13 plus\(new: this PR`a` $5repeatedIt costs $5,repeated$arepeated$$a\$repeated$a\\\repeated\$$arepeatedAcross every shape tested,
newstays within about 2× ofnone. The remaining superlinear growth on single-line 100 KB paragraphs comes from marked itself:nonegrows from 0.45 ms at 10 KB to about 97 ms at 100 KB.A guard test lexes six of these shapes at 10 KB within a 500 ms budget. It takes about 36 ms with this change and about 50 s if
startretries the regex from every$.Tests
bun test srcinpackages/uiandpackages/session-uitsgotypecheck for both packages\(and$$blocks.Requested by: @Brendonovich (Brendan via Slack)