Skip to content

Optimize empty token streams - #162047

Merged
rust-bors[bot] merged 1 commit into
rust-lang:mainfrom
nnethercote:empty-token-stream
Sep 2, 2026
Merged

Optimize empty token streams#162047
rust-bors[bot] merged 1 commit into
rust-lang:mainfrom
nnethercote:empty-token-stream

Conversation

@nnethercote

@nnethercote nnethercote commented Aug 31, 2026

Copy link
Copy Markdown
Contributor

Empty token streams are common. Every one causes an allocation for the Arc::new(). We can instead have a single shared empty TokenStream and AttrTokenStream.

r? @Kobzol

Empty token streams are common. Every one causes an allocation for the
`Arc::new()`. We can instead have a single shared empty `TokenStream`
and `AttrTokenStream`.
@rustbot rustbot added S-waiting-on-author Status: This is awaiting some action (such as code changes or more information) from the author. T-compiler Relevant to the compiler team, which will review and decide on the PR/issue. labels Aug 31, 2026
@nnethercote

Copy link
Copy Markdown
Contributor Author

@bors try @rust-timer queue

@rust-timer

This comment has been minimized.

@rustbot rustbot added the S-waiting-on-perf Status: Waiting on a perf run to be completed. label Aug 31, 2026
@rust-bors

This comment has been minimized.

rust-bors Bot pushed a commit that referenced this pull request Aug 31, 2026
@rust-bors

rust-bors Bot commented Aug 31, 2026

Copy link
Copy Markdown
Contributor

☀️ Try build successful (CI)
Build commit: 11dd43e (11dd43efe8e73314a31dde8b820722be36da3904)
Base parent: 5321a4f (5321a4f40c957cf3587c055e77461febc2ebc865)

@rust-timer

This comment has been minimized.

@rust-timer

Copy link
Copy Markdown
Collaborator

Finished benchmarking commit (11dd43e): comparison URL.

Overall result: no relevant changes - no action needed

Benchmarking means the PR may be perf-sensitive. Consider adding rollup=never if this change is not fit for rolling up.

@rustbot label: -S-waiting-on-perf -perf-regression

Instruction count

This perf run didn't have relevant results for this metric.

Max RSS (memory usage)

This perf run didn't have relevant results for this metric.

Cycles

This perf run didn't have relevant results for this metric.

Binary size

This perf run didn't have relevant results for this metric.

Bootstrap: missing data
Artifact size: 402.71 MiB -> 403.53 MiB (0.20%)

@rustbot rustbot removed the S-waiting-on-perf Status: Waiting on a perf run to be completed. label Aug 31, 2026
@Kobzol

Kobzol commented Aug 31, 2026

Copy link
Copy Markdown
Member

(Sorry, a rustc-perf failure, looking into it)

@Kobzol

Kobzol commented Aug 31, 2026

Copy link
Copy Markdown
Member

@rust-timer build 11dd43e

@rust-timer

This comment has been minimized.

@rust-timer

Copy link
Copy Markdown
Collaborator

Finished benchmarking commit (11dd43e): comparison URL.

Overall result: ✅ improvements - no action needed

Benchmarking means the PR may be perf-sensitive. It's automatically marked not fit for rolling up. Overriding is possible but disadvised: it risks changing compiler perf.

@bors rollup=never rustc-perf
@rustbot label: -S-waiting-on-perf -perf-regression

Instruction count

Our most reliable metric. Used to determine the overall result above. However, even this metric can be noisy.

mean range count
Regressions ❌
(primary)
0.2% [0.2%, 0.2%] 1
Regressions ❌
(secondary)
0.1% [0.1%, 0.1%] 2
Improvements ✅
(primary)
-0.2% [-0.3%, -0.1%] 18
Improvements ✅
(secondary)
-0.3% [-0.6%, -0.1%] 27
All ❌✅ (primary) -0.2% [-0.3%, 0.2%] 19

Max RSS (memory usage)

Results (primary -0.8%, secondary -0.6%)

A less reliable metric. May be of interest, but not used to determine the overall result above.

mean range count
Regressions ❌
(primary)
1.5% [1.0%, 1.9%] 2
Regressions ❌
(secondary)
4.2% [2.7%, 8.3%] 7
Improvements ✅
(primary)
-1.4% [-2.3%, -0.6%] 9
Improvements ✅
(secondary)
-2.4% [-5.0%, -0.6%] 18
All ❌✅ (primary) -0.8% [-2.3%, 1.9%] 11

Cycles

Results (primary -2.6%, secondary 9.4%)

A less reliable metric. May be of interest, but not used to determine the overall result above.

mean range count
Regressions ❌
(primary)
- - 0
Regressions ❌
(secondary)
9.4% [6.6%, 12.6%] 4
Improvements ✅
(primary)
-2.6% [-2.6%, -2.6%] 1
Improvements ✅
(secondary)
- - 0
All ❌✅ (primary) -2.6% [-2.6%, -2.6%] 1

Binary size

This perf run didn't have relevant results for this metric.

Bootstrap: 477.635s -> 475.679s (-0.41%)
Artifact size: 402.71 MiB -> 403.53 MiB (0.20%)

@Kobzol

Kobzol commented Aug 31, 2026

Copy link
Copy Markdown
Member

Makes sense. You can r=me.

@rustbot

rustbot commented Aug 31, 2026

Copy link
Copy Markdown
Collaborator

Kobzol is not on the review rotation at the moment.
They may take a while to respond.

@nnethercote
nnethercote marked this pull request as ready for review August 31, 2026 22:07
@rustbot rustbot added the S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. label Aug 31, 2026
@nnethercote

Copy link
Copy Markdown
Contributor Author

@bors r=Kobzol

@rustbot rustbot removed the S-waiting-on-author Status: This is awaiting some action (such as code changes or more information) from the author. label Aug 31, 2026
@rust-bors

rust-bors Bot commented Aug 31, 2026

Copy link
Copy Markdown
Contributor

📌 Commit 5f96a1c has been approved by Kobzol

It is now in the queue for this repository.

@rust-bors rust-bors Bot added S-waiting-on-bors Status: Waiting on bors to run and complete tests. Bors will change the label on completion. and removed S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. labels Aug 31, 2026
@JonathanBrouwer

Copy link
Copy Markdown
Member

@bors rollup=iffy
Creating a rollup of perf-sensitive PRs

rust-bors Bot pushed a commit that referenced this pull request Sep 2, 2026
…uwer

Rollup of 3 perf-sensitive pull requests

Successful merges:

 - #161850 (Store LiveLoans more densely packed)
 - #162031 (Reduce next-solver memory usage by interning CanonicalQueryInput)
 - #162047 (Optimize empty token streams)
@rust-bors
rust-bors Bot merged commit d5b8ff4 into rust-lang:main Sep 2, 2026
14 checks passed
@rustbot rustbot added this to the 1.100.0 milestone Sep 2, 2026
rust-bors Bot pushed a commit that referenced this pull request Sep 2, 2026
Rollup merge of #162047 - nnethercote:empty-token-stream, r=Kobzol

Optimize empty token streams

Empty token streams are common. Every one causes an allocation for the `Arc::new()`. We can instead have a single shared empty `TokenStream` and `AttrTokenStream`.

r? @Kobzol
@rust-timer

Copy link
Copy Markdown
Collaborator

Note

This PR was benchmarked as part of triage of its containing rollup: triage URL.

Finished benchmarking commit (2d373c0): comparison URL.

Overall result: ❌✅ regressions and improvements - please read:

Our benchmarks found a performance regression caused by this PR.
This might be an actual regression, but it can also be just noise.

Next Steps:

  • If the regression was expected or you think it can be justified,
    please write a comment with sufficient written justification, and add
    @rustbot label: +perf-regression-triaged to it, to mark the regression as triaged.
  • If you think that you know of a way to resolve the regression, try to create
    a new PR with a fix for the regression.
  • If you do not understand the regression or you think that it is just noise,
    you can ask the @rust-lang/wg-compiler-performance working group for help (members of this group
    were already notified of this PR).

@rustbot label: +perf-regression
cc @rust-lang/wg-compiler-performance

Instruction count

Our most reliable metric. Used to determine the overall result above. However, even this metric can be noisy.

mean range count
Regressions ❌
(primary)
- - 0
Regressions ❌
(secondary)
0.2% [0.1%, 0.4%] 14
Improvements ✅
(primary)
-0.2% [-0.4%, -0.1%] 18
Improvements ✅
(secondary)
-0.3% [-0.7%, -0.1%] 29
All ❌✅ (primary) -0.2% [-0.4%, -0.1%] 18

Max RSS (memory usage)

Results (primary 1.1%, secondary -3.9%)

A less reliable metric. May be of interest, but not used to determine the overall result above.

mean range count
Regressions ❌
(primary)
2.7% [2.7%, 2.7%] 1
Regressions ❌
(secondary)
- - 0
Improvements ✅
(primary)
-0.4% [-0.4%, -0.4%] 1
Improvements ✅
(secondary)
-3.9% [-6.2%, -1.2%] 5
All ❌✅ (primary) 1.1% [-0.4%, 2.7%] 2

Cycles

Results (primary -3.6%)

A less reliable metric. May be of interest, but not used to determine the overall result above.

mean range count
Regressions ❌
(primary)
- - 0
Regressions ❌
(secondary)
- - 0
Improvements ✅
(primary)
-3.6% [-3.6%, -3.6%] 1
Improvements ✅
(secondary)
- - 0
All ❌✅ (primary) -3.6% [-3.6%, -3.6%] 1

Binary size

This perf run didn't have relevant results for this metric.

Bootstrap: missing data
Artifact size: 400.83 MiB -> 400.64 MiB (-0.05%)

@rustbot rustbot added the perf-regression Performance regression. label Sep 2, 2026
@JonathanBrouwer

JonathanBrouwer commented Sep 7, 2026

Copy link
Copy Markdown
Member

The regressions are noise
@rustbot label: +perf-regression-triaged

@rustbot rustbot added the perf-regression-triaged The performance regression has been triaged. label Sep 7, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

perf-regression Performance regression. perf-regression-triaged The performance regression has been triaged. S-waiting-on-bors Status: Waiting on bors to run and complete tests. Bors will change the label on completion. T-compiler Relevant to the compiler team, which will review and decide on the PR/issue.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants