Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

new lint: format_collect #11116

Merged
merged 2 commits into from
Jul 17, 2023
Merged

new lint: format_collect #11116

merged 2 commits into from
Jul 17, 2023

Conversation

y21
Copy link
Member

@y21 y21 commented Jul 6, 2023

A perf lint that looks for format!ing inside of map, then collecting it into a String. Did a quick benchmark locally and it's a bit more than 2x faster with fold.
write! is still not optimal (presumably because the fmt stuff goes through dynamic dispatch), but it's still a lot better than creating a new string on every element.
I thought about making a machine applicable suggestion, but there's a lot of suggestions that need to be made here, so I decided to just add help messages.

changelog: new lint: format_collect

@rustbot
Copy link
Collaborator

rustbot commented Jul 6, 2023

r? @Manishearth

(rustbot has picked a reviewer for you, use r? to override)

@rustbot rustbot added the S-waiting-on-review Status: Awaiting review from the assignee but also interested parties label Jul 6, 2023
@y21 y21 force-pushed the format_collect branch from a365e48 to c329421 Compare July 6, 2023 23:08
@y21
Copy link
Member Author

y21 commented Jul 6, 2023

The regex tests failing CI looks unrelated. Should hopefully be fixed when #11111 gets merged

@Manishearth
Copy link
Member

r? @Centri3

@rustbot
Copy link
Collaborator

rustbot commented Jul 7, 2023

Failed to set assignee to Centri3: invalid assignee

Note: Only org members, users with write permissions, or people who have commented on the PR may be assigned.

@y21 y21 force-pushed the format_collect branch 2 times, most recently from cf86887 to 570dff6 Compare July 7, 2023 15:39
Copy link
Member

@Centri3 Centri3 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Implementation looks pretty good :) Not sure about the lint category

use rustc_lint::LateContext;
use rustc_span::Span;

fn tail_expr<'tcx>(expr: &'tcx Expr<'tcx>) -> Option<&'tcx Expr<'tcx>> {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Maybe I'm missing something, but can this not use clippy_utils::peel_blocks? Unless from_expansion matters here, which it may in the case of format! (and also that it returns None if block.expr is None)

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

yep, the important bit is the check that it doesn't come from an expansion. If we always remove blocks, we would also remove parts of the format! expansion, which makes code further below not work (root_macro_call_first_node will return None, just tested it)

Copy link
Member

@Centri3 Centri3 Jul 8, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think we could check the syntax context (.ctxt()) of the span, then .outer_expn_data(), so we only ignore blocks if it's from format! but that's a major nit (There also may be a better way to do that, no idea)

Also, this should probably be recursive so something like {{ format!("{x:?}) }} works (which I believe doesn't?)

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The name is also a bit odd as well, no suggestions though. Perhaps documenting why it isn't using peel_blocks could be nice

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think we could check the syntax context (.ctxt()) of the span, then .outer_expn_data(), so we only ignore blocks if it's from format! but that's a major nit (There also may be a better way to do that, no idea)

I'm not sure. I think doing it like that will have FPs (unless I misunderstood) because outer_expn_data() returns the "outermost"(?) expansion, not the very first. In other words, it would trigger here:

macro_rules! x {
    ($b:expr) => {
        format!("{:02X}", $b)
    };
}
fn hex_encode(bytes: &[u8]) -> String {
    bytes.iter().map(|b| x!(b)).collect()
}

when it shouldn't, because format! comes from another macro expansion

Also, this should probably be recursive so something like {{ format!("{x:?}) }} works (which I believe doesn't?)

Good point.

The name is also a bit odd as well, no suggestions though. Perhaps documenting why it isn't using peel_blocks could be nice

Are you referring to the name of the function or the lint? I agree with both though 😅

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm not sure. I think doing it like that will have FPs (unless I misunderstood) because outer_expn_data() returns the "outermost"(?) expansion, not the very first. In other words, it would trigger here:

Is that so? That's unfortunate then

Are you referring to the name of the function or the lint? I agree with both though sweat_smile

Function, the lint's name is fine imo

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Oh I think this could use marks or remove_mark on Span::ctxt instead, but I highly doubt this is worth the effort ^^ This won't change the actual outcome afaik

/// ```
#[clippy::version = "1.72.0"]
pub FORMAT_COLLECT,
perf,
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm not sure whether this should be perf. It's faster, but the example given doesn't seem to be worth the code smell (it should also probably use try_fold so errors from write! are propagated)

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

write! never errors on Strings, so I'm not too sure on that. But yes, I agree that this might be too annoying for perf.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Interesting, that should probably be documented in the lint description with a comment then as that's rather unexpected behavior (at least from the reader's perspective)

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I guess it's only really a useful suggestion if the iterator is gonna yield lots of small items, like bytes or chars (which seems hard to put into a lint, other than maybe restricting it to small types), otherwise it feels pedantic, like here: https://github.com/rust-lang/rust-clippy/pull/11116/files#diff-42cb6807ad74b3e201c5a7ca98b911c5fa08380e942be6e4ac5807f8377f87fcL135

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

write! never errors on Strings, so I'm not too sure on that. But yes, I agree that this might be too annoying for perf.

Display impls can return errors even if String does not (which is not advisable though)

@Centri3
Copy link
Member

Centri3 commented Jul 16, 2023

Pretty sure this is done ^^

cc @Manishearth

@Manishearth
Copy link
Member

@bors r=centri3

@bors
Copy link
Contributor

bors commented Jul 16, 2023

📌 Commit a68923d has been approved by centri3

It is now in the queue for this repository.

@bors
Copy link
Contributor

bors commented Jul 16, 2023

⌛ Testing commit a68923d with merge 9dfb0b3...

bors added a commit that referenced this pull request Jul 16, 2023
new lint: `format_collect`

A perf lint that looks for `format!`ing inside of `map`, then collecting it into a `String`. Did a quick benchmark locally and it's a bit more than 2x faster with fold.
`write!` is still not optimal (presumably because the fmt stuff goes through dynamic dispatch), but it's still a lot better than creating a new string on every element.
I thought about making a machine applicable suggestion, but there's a lot of suggestions that need to be made here, so I decided to just add help messages.

changelog: new lint: `format_collect`
@bors
Copy link
Contributor

bors commented Jul 16, 2023

💔 Test failed - checks-action_dev_test

@Centri3
Copy link
Member

Centri3 commented Jul 17, 2023

Formatting, likely a let..else somewhere (or imports). There's no conflicts but rebase then run cargo dev fmt

@y21 y21 force-pushed the format_collect branch from a68923d to c83d58f Compare July 17, 2023 10:23
@Manishearth
Copy link
Member

@bors r+

@bors
Copy link
Contributor

bors commented Jul 17, 2023

📌 Commit c83d58f has been approved by Manishearth

It is now in the queue for this repository.

@bors
Copy link
Contributor

bors commented Jul 17, 2023

⌛ Testing commit c83d58f with merge 410456d...

@Manishearth
Copy link
Member

oops, should have been r=Centri3, by habit

@bors
Copy link
Contributor

bors commented Jul 17, 2023

☀️ Test successful - checks-action_dev_test, checks-action_remark_test, checks-action_test
Approved by: Manishearth
Pushing 410456d to master...

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
S-waiting-on-review Status: Awaiting review from the assignee but also interested parties
Projects
None yet
Development

Successfully merging this pull request may close these issues.

6 participants