Update F541 to use new f-string tokens

[dhruvmanila]

Summary

This PR updates the F541 rule to use the new f-string tokens.

Test Plan

Add new test case and uncomment a broken test case.

fixes: #7292

[dhruvmanila]

Current dependencies on/for this PR:

• main
  • PR Add support for PEP 701 #7376
    • PR Update logical line rules for the new f-string tokens #7690
    • PR Remove StringKind::FString and StringKind::RawFString #7667
    • PR Ignore quote escapes in expression part of f-string #7597
    • PR Separate Q003 to accomodate f-string context #7588
      • PR Update Q000, Q001 with the new f-string tokens #7589
    • PR Use the new f-string tokens in string formatting #7586
    • PR Update ISC001, ISC002 to check in f-strings #7515
    • PR Update RUF001, RUF003 to check in f-strings #7477
    • PR Update PLE2510, PLE2512-2515 to check in f-strings #7387
    • PR Allow NUL character in f-strings #7378
    • PR Update W605 to check in f-strings #7329
    • PR Update Stylist quote detection with new f-string token #7328
    • PR Update F541 to use new f-string tokens #7327 👈
    • PR Detect noqa directives for multi-line f-strings #7326
    • PR Update Indexer to use new f-string tokens #7325

This comment was auto-generated by Graphite.

[MichaReiser]

How does this work with multiple nested fstrings?

f"{f"{a + f"{b}"}"}"

I would expect that f"{b}" overrides the start position of f"{a + f"{b}"}"

[dhruvmanila]

To give some context, the autofix for the rule F541 is to remove the f prefix from the f-string without any placeholders. The placeholder checks are done at an expression level by checking if there's any FormattedValue. The function you're referring to is to get the f prefix range and the f-string range. At this point there can't be any nested f-strings.

Also, the reason the function returns an iterator is because of implicitly concatenated strings.

[dhruvmanila]

I think the function name is a bit confusing which I'll rename. I've also added docs to the function.

[MichaReiser]

Hmm, but the documentation mentions that this function returns all nested f-string ranges, meaning there need to be placeholders? It's also unclear to me where the filtering out of f-strings without placeholders happens.

So I think we need to update the documentation to match its actual behavior (or it's too early in the morning and I shouldn't be reviewing PR's 😅 )

[dhruvmanila]

Hmm, but the documentation mentions that this function returns all nested f-string ranges, meaning there need to be placeholders?

I've updated the documentation but I'm curious as to where was "nested f-string ranges" mentioned? 😅

[dhruvmanila]

It's also unclear to me where the filtering out of f-strings without placeholders happens.

@MichaReiser

[MichaReiser]

Thanks. The issue I see is that find_useless_f_strings returns all fstrings but with potentially incorrect ranges. So naming that function find_useless_f_strings is misleading in my view because not all fstrings returned by that function are useless.

[dhruvmanila]

Yes, I totally agree. I've renamed it to fstring_prefix_and_tok_range but the main reason it returns an iterator is because the f-string can contain multiple implicitly concatenated (f-)strings.

because not all fstrings returned by that function are useless.

Can you expand on this? For example, in the below code snippet it would return (prefix range, token range) for the "first" and the "last" string and not for the "normal" string. Both f-strings are useless in the sense that there are no placeholders used and so the f prefix is useless. I don't think it ever returns incorrect ranges.

  f"first" "normal" rf"second"
# ^                  ^             (prefix range)
# ^^^^^^^^          ^^^^^^^^^^     (token range)

# (0..1, 0..8)      - `f"first"`
# (19..20, 18..28)  - `rf"second"`