Improve source code for `highlight.rs` #146992

GuillaumeGomez · 2025-09-24T16:34:12Z

I was very bothered by the complexity of this file, in particular the handling of pending_elems which was very tricky to follow.

So instead, here comes a more sane approach: the content is store in a stack-like type which handles "levels" of HTML (ie, a macro expansion can contain other HTML tags which can themselves contain other, etc). Making it much simpler to keep track of what's going on.

r? @lolbinarycat

GuillaumeGomez · 2025-09-24T16:34:52Z

Also need to check the impact on performance (likely slower).

@bors try @rust-timer queue

Improve source code for `highlight.rs`

rust-bors · 2025-09-24T18:48:51Z

☀️ Try build successful (CI)
Build commit: 6020c97 (6020c97e3046a35e53fd9885eda164c570010a6c, parent: 15283f6fe95e5b604273d13a428bab5fc0788f5a)

src/librustdoc/html/highlight.rs

yotamofek · 2025-09-24T19:44:11Z

Code reads much better IMHO!

rust-timer · 2025-09-24T19:59:22Z

Finished benchmarking commit (6020c97): comparison URL.

Overall result: ❌ regressions - please read the text below

Benchmarking this pull request means it may be perf-sensitive – we'll automatically label it not fit for rolling up. You can override this, but we strongly advise not to, due to possible changes in compiler perf.

Next Steps: If you can justify the regressions found in this try perf run, please do so in sufficient writing along with @rustbot label: +perf-regression-triaged. If not, please fix the regressions and do another perf run. If its results are neutral or positive, the label will be automatically removed.

@bors rollup=never
@rustbot label: -S-waiting-on-perf +perf-regression

Instruction count

Our most reliable metric. Used to determine the overall result above. However, even this metric can be noisy.

	mean	range	count
Regressions ❌ (primary)	2.2%	[0.7%, 6.6%]	19
Regressions ❌ (secondary)	3.2%	[0.1%, 13.3%]	17
Improvements ✅ (primary)	-	-	0
Improvements ✅ (secondary)	-	-	0
All ❌✅ (primary)	2.2%	[0.7%, 6.6%]	19

Max RSS (memory usage)

Results (primary 0.4%, secondary 12.0%)

A less reliable metric. May be of interest, but not used to determine the overall result above.

	mean	range	count
Regressions ❌ (primary)	6.2%	[6.2%, 6.2%]	1
Regressions ❌ (secondary)	12.0%	[2.6%, 31.9%]	6
Improvements ✅ (primary)	-2.5%	[-2.8%, -2.2%]	2
Improvements ✅ (secondary)	-	-	0
All ❌✅ (primary)	0.4%	[-2.8%, 6.2%]	3

Cycles

Results (primary 2.8%, secondary 5.9%)

A less reliable metric. May be of interest, but not used to determine the overall result above.

	mean	range	count
Regressions ❌ (primary)	4.3%	[3.2%, 6.5%]	3
Regressions ❌ (secondary)	8.3%	[3.6%, 12.8%]	6
Improvements ✅ (primary)	-1.6%	[-1.6%, -1.6%]	1
Improvements ✅ (secondary)	-1.5%	[-1.5%, -1.4%]	2
All ❌✅ (primary)	2.8%	[-1.6%, 6.5%]	4

Binary size

This benchmark run did not return any relevant results for this metric.

Bootstrap: 471.213s -> 470.794s (-0.09%)
Artifact size: 387.83 MiB -> 387.93 MiB (0.03%)

GuillaumeGomez · 2025-09-24T22:39:50Z

Code reads much better IMHO!

Agreed (unsurprisingly 😆), but sadly I think this solution is unlikely to get much better performance-wise so unlikely it'll be merged.

However I now have a much cleaner code, so I think I'll go back to the original "streaming content" but with a much cleaner approach.

lolbinarycat · 2025-09-25T16:31:24Z

If I had to guess the reason for the perf regression, I would say it probably has to do with all the extra intermediate string allocations. I feel like if we had a way to delay formatting (maybe using an enum, or with dyn Display, or just make it so the final writer is given up front), this would have a lot less overhead.

GuillaumeGomez · 2025-09-25T16:46:33Z

Possibly. Want to give a try pushing it even further before I try to turn this back into a streaming algorithm? Same question for you @yotamofek. 😉

Start from my branch and open PRs with your commits so we can run perf check on them.

yotamofek · 2025-09-25T17:07:32Z

I'll give it a go, but my gut says the extra string allocations aren't causing the lion's share of the regressions. Worth a shot though

bors · 2025-09-26T01:17:09Z

☔ The latest upstream changes (presumably #147037) made this pull request unmergeable. Please resolve the merge conflicts.

bors · 2025-10-06T16:52:19Z

☔ The latest upstream changes (presumably #147397) made this pull request unmergeable. Please resolve the merge conflicts.

GuillaumeGomez · 2025-10-07T13:05:38Z

So I realized that there are some bugs in the existing code which are fixed by this PR:

In the example above, only self should be underlined, not the whitespace. Anyway, looking a bit more into this performance issue.

rustbot · 2025-10-23T16:10:19Z

This PR was rebased onto a different master commit. Here's a range-diff highlighting what actually changed.

Rebasing is a normal part of keeping PRs up to date, so no action is needed—this note is just to help reviewers.

GuillaumeGomez · 2025-10-23T16:10:25Z

Applied suggestions and rebased.

… `closing_tags` Improve documentation Improve code

GuillaumeGomez · 2025-10-23T19:48:01Z

Arf, english typo, my nemesis. Convenient check. Anyway, the typo checker should be happy now. :)

yotamofek

Approving, but see this comment: #146992 (comment)
Nothing I can think of other than that.

Really cool change!

View changes since this review

GuillaumeGomez · 2025-10-26T14:36:22Z

Thanks for the review. Considering it's fixing bugs, gonna improve the code in a follow-up.

@bors r=yotamofek,lolbinarycat rollup

bors · 2025-10-26T14:36:25Z

📌 Commit ab1dcee has been approved by yotamofek,lolbinarycat

It is now in the queue for this repository.

bors · 2025-10-26T17:31:25Z

⌛ Testing commit ab1dcee with merge f977dfc...

bors · 2025-10-26T20:35:55Z

☀️ Test successful - checks-actions
Approved by: yotamofek,lolbinarycat
Pushing f977dfc to master...

github-actions · 2025-10-26T20:39:20Z

What is this?

This is an experimental post-merge analysis report that shows differences in test outcomes between the merged PR and its parent PR.

Comparing 2888098 (parent) -> f977dfc (this PR)

Test differences

Show 5 test diffs

5 doctest diffs were found. These are ignored, as they are noisy.

Test dashboard

Run

cargo run --manifest-path src/ci/citool/Cargo.toml -- \
    test-dashboard f977dfc388ea39c9886b7f8c49abce26e6918df6 --output-dir test-dashboard

And then open test-dashboard/index.html in your browser to see an overview of all executed tests.

Job duration changes

dist-aarch64-msvc: 5386.0s -> 6290.8s (16.8%)
dist-various-1: 3687.9s -> 4219.5s (14.4%)
pr-check-1: 1294.5s -> 1463.9s (13.1%)
aarch64-apple: 7673.6s -> 6895.9s (-10.1%)
dist-apple-various: 3604.6s -> 3241.1s (-10.1%)
x86_64-gnu-gcc: 3070.7s -> 2776.9s (-9.6%)
arm-android: 5419.7s -> 5917.9s (9.2%)
x86_64-gnu-stable: 7065.9s -> 6521.7s (-7.7%)
x86_64-mingw-2: 7979.9s -> 7381.3s (-7.5%)
dist-x86_64-musl: 8340.5s -> 7762.2s (-6.9%)

How to interpret the job duration changes?

Job durations can vary a lot, based on the actual runner instance
that executed the job, system noise, invalidated caches, etc. The table above is provided
mostly for t-infra members, for simpler debugging of potential CI slow-downs.

rust-timer · 2025-10-26T21:46:51Z

Finished benchmarking commit (f977dfc): comparison URL.

Overall result: ❌✅ regressions and improvements - please read the text below

Our benchmarks found a performance regression caused by this PR.
This might be an actual regression, but it can also be just noise.

Next Steps:

If the regression was expected or you think it can be justified,
please write a comment with sufficient written justification, and add
@rustbot label: +perf-regression-triaged to it, to mark the regression as triaged.
If you think that you know of a way to resolve the regression, try to create
a new PR with a fix for the regression.
If you do not understand the regression or you think that it is just noise,
you can ask the @rust-lang/wg-compiler-performance working group for help (members of this group
were already notified of this PR).

@rustbot label: +perf-regression
cc @rust-lang/wg-compiler-performance

Instruction count

Our most reliable metric. Used to determine the overall result above. However, even this metric can be noisy.

	mean	range	count
Regressions ❌ (primary)	0.2%	[0.1%, 0.2%]	6
Regressions ❌ (secondary)	0.2%	[0.1%, 0.3%]	3
Improvements ✅ (primary)	-	-	0
Improvements ✅ (secondary)	-0.1%	[-0.1%, -0.1%]	1
All ❌✅ (primary)	0.2%	[0.1%, 0.2%]	6

Max RSS (memory usage)

Results (primary -1.2%, secondary -0.2%)

A less reliable metric. May be of interest, but not used to determine the overall result above.

	mean	range	count
Regressions ❌ (primary)	-	-	0
Regressions ❌ (secondary)	2.4%	[2.4%, 2.4%]	1
Improvements ✅ (primary)	-1.2%	[-1.2%, -1.2%]	1
Improvements ✅ (secondary)	-2.8%	[-2.8%, -2.8%]	1
All ❌✅ (primary)	-1.2%	[-1.2%, -1.2%]	1

Cycles

Results (secondary -2.7%)

A less reliable metric. May be of interest, but not used to determine the overall result above.

	mean	range	count
Regressions ❌ (primary)	-	-	0
Regressions ❌ (secondary)	-	-	0
Improvements ✅ (primary)	-	-	0
Improvements ✅ (secondary)	-2.7%	[-2.7%, -2.7%]	1
All ❌✅ (primary)	-	-	0

Binary size

This benchmark run did not return any relevant results for this metric.

Bootstrap: 476.488s -> 473.751s (-0.57%)
Artifact size: 390.47 MiB -> 390.43 MiB (-0.01%)

panstromek · 2025-10-27T16:51:51Z

perf triage:

Small regressions were expected base from the pre-merge run and following comment: #146992 (comment). Overall the impact is not big and looks partially like noise, since the results got better in next PR (which is unrelated).

@rustbot label: +perf-regression-triaged

…ht, r=yotamofek,lolbinarycat Simplify code to generate line numbers in highlight Follow-up of rust-lang#146992. cc `@yotamofek` r? `@lolbinarycat`

…ht, r=yotamofek,lolbinarycat Simplify code to generate line numbers in highlight Follow-up of rust-lang#146992. cc ``@yotamofek`` r? ``@lolbinarycat``

…ht, r=yotamofek,lolbinarycat Simplify code to generate line numbers in highlight Follow-up of rust-lang#146992. cc ```@yotamofek``` r? ```@lolbinarycat```

…ht, r=yotamofek,lolbinarycat Simplify code to generate line numbers in highlight Follow-up of rust-lang#146992. cc ````@yotamofek```` r? ````@lolbinarycat````

…ht, r=yotamofek,lolbinarycat Simplify code to generate line numbers in highlight Follow-up of rust-lang#146992. cc `````@yotamofek````` r? `````@lolbinarycat`````

Rollup merge of #148171 - GuillaumeGomez:line-number-highlight, r=yotamofek,lolbinarycat Simplify code to generate line numbers in highlight Follow-up of #146992. cc `````@yotamofek````` r? `````@lolbinarycat`````

rustbot assigned lolbinarycat Sep 24, 2025

This comment has been minimized.

Sign in to view

rust-bors bot added a commit that referenced this pull request Sep 24, 2025

Auto merge of #146992 - GuillaumeGomez:improve-highlight, r=<try>

6020c97

Improve source code for `highlight.rs`

This comment has been minimized.

Sign in to view

rustbot added the S-waiting-on-perf Status: Waiting on a perf run to be completed. label Sep 24, 2025

This comment has been minimized.

Sign in to view

yotamofek reviewed Sep 24, 2025

View reviewed changes

src/librustdoc/html/highlight.rs Outdated Show resolved Hide resolved

yotamofek reviewed Sep 24, 2025

View reviewed changes

src/librustdoc/html/highlight.rs Outdated Show resolved Hide resolved

yotamofek reviewed Sep 24, 2025

View reviewed changes

src/librustdoc/html/highlight.rs Outdated Show resolved Hide resolved

rustbot added perf-regression Performance regression. and removed S-waiting-on-perf Status: Waiting on a perf run to be completed. labels Sep 24, 2025

GuillaumeGomez force-pushed the improve-highlight branch from dfcb725 to 957ca47 Compare September 27, 2025 21:25

This comment has been minimized.

Sign in to view

yotamofek mentioned this pull request Sep 27, 2025

Small highlight.rs optimizations #147106

Open

GuillaumeGomez force-pushed the improve-highlight branch from 957ca47 to 2ae1bf4 Compare October 7, 2025 12:42

This comment has been minimized.

Sign in to view

Merge ClassInfo::open and ClassInfo::link_closing_tag fields into…

ab1dcee

… `closing_tags` Improve documentation Improve code

GuillaumeGomez force-pushed the improve-highlight branch from e8f2462 to ab1dcee Compare October 23, 2025 19:47

yotamofek approved these changes Oct 26, 2025

View reviewed changes

bors added S-waiting-on-bors Status: Waiting on bors to run and complete tests. Bors will change the label on completion. and removed S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. labels Oct 26, 2025

bors added the merged-by-bors This PR was explicitly merged by bors. label Oct 26, 2025

bors merged commit f977dfc into rust-lang:master Oct 26, 2025
12 checks passed

rustbot added this to the 1.93.0 milestone Oct 26, 2025

GuillaumeGomez deleted the improve-highlight branch October 27, 2025 10:25

GuillaumeGomez mentioned this pull request Oct 27, 2025

Simplify code to generate line numbers in highlight #148171

Merged

rustbot added the perf-regression-triaged The performance regression has been triaged. label Oct 27, 2025

dtolnay mentioned this pull request Oct 27, 2025

ICE: exit_elem called on empty class stack #148184

Open

fmease mentioned this pull request Oct 31, 2025

Fix invalid tag closing when leaving expansion "original code" #148213

Open

Improve source code for highlight.rs #146992

Improve source code for highlight.rs #146992

Uh oh!

Conversation

GuillaumeGomez commented Sep 24, 2025

Uh oh!

GuillaumeGomez commented Sep 24, 2025

Uh oh!

This comment has been minimized.

This comment has been minimized.

rust-bors bot commented Sep 24, 2025

Uh oh!

This comment has been minimized.

Uh oh!

Uh oh!

Uh oh!

yotamofek commented Sep 24, 2025

Uh oh!

rust-timer commented Sep 24, 2025

Overall result: ❌ regressions - please read the text below

Uh oh!

GuillaumeGomez commented Sep 24, 2025

Uh oh!

lolbinarycat commented Sep 25, 2025

Uh oh!

GuillaumeGomez commented Sep 25, 2025

Uh oh!

yotamofek commented Sep 25, 2025

Uh oh!

bors commented Sep 26, 2025

Uh oh!

This comment has been minimized.

bors commented Oct 6, 2025

Uh oh!

This comment has been minimized.

GuillaumeGomez commented Oct 7, 2025

Uh oh!

rustbot commented Oct 23, 2025

Uh oh!

GuillaumeGomez commented Oct 23, 2025

Uh oh!

This comment has been minimized.

GuillaumeGomez commented Oct 23, 2025

Uh oh!

yotamofek left a comment • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

GuillaumeGomez commented Oct 26, 2025

Uh oh!

bors commented Oct 26, 2025

Uh oh!

bors commented Oct 26, 2025

Uh oh!

bors commented Oct 26, 2025

Uh oh!

Uh oh!

github-actions bot commented Oct 26, 2025

Test differences

Job duration changes

Uh oh!

rust-timer commented Oct 26, 2025

Overall result: ❌✅ regressions and improvements - please read the text below

Uh oh!

panstromek commented Oct 27, 2025

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

9 participants

Improve source code for `highlight.rs` #146992

Improve source code for `highlight.rs` #146992

yotamofek left a comment •

edited

Loading