Update RUF001, RUF003 to check in f-strings

[dhruvmanila]

Summary

This PR updates the rule RUF001 and RUF003 to check in f-strings using the
FStringMiddle token which contains the non-expression part of a f-string.

For reference,

Code	Name	Message
RUF001	ambiguous-unicode-character-string	String contains ambiguous {}. Did you mean {}?
RUF003	ambiguous-unicode-character-comment	Comment contains ambiguous {}. Did you mean {}?

Test Plan

cargo test

[dhruvmanila]

Current dependencies on/for this PR:

• main
  • PR Add support for PEP 701 #7376
    • PR Update logical line rules for the new f-string tokens #7690
    • PR Remove StringKind::FString and StringKind::RawFString #7667
    • PR Ignore quote escapes in expression part of f-string #7597
    • PR Separate Q003 to accomodate f-string context #7588
      • PR Update Q000, Q001 with the new f-string tokens #7589
    • PR Use the new f-string tokens in string formatting #7586
    • PR Update ISC001, ISC002 to check in f-strings #7515
    • PR Update RUF001, RUF003 to check in f-strings #7477 👈
    • PR Update PLE2510, PLE2512-2515 to check in f-strings #7387
    • PR Allow NUL character in f-strings #7378
    • PR Update W605 to check in f-strings #7329
    • PR Update Stylist quote detection with new f-string token #7328
    • PR Update F541 to use new f-string tokens #7327
    • PR Detect noqa directives for multi-line f-strings #7326
    • PR Update Indexer to use new f-string tokens #7325

This comment was auto-generated by Graphite.

[dhruvmanila]

Should we add a new context to make this message say "F-string contains ..." instead of "String contains ..."?

[MichaReiser]

Can fstrings be docstrings?

[dhruvmanila]

No. Reference: https://docs.python.org/3/reference/lexical_analysis.html#formatted-string-literals

Formatted string literals cannot be used as docstrings, even if they do not include expressions.

def foo():
    f"Not a docstring"

foo.__doc__ is None
# True

[MichaReiser]

I would then rewrite it for better clarity

if tok.is_string() && is_docstring {
	Context::Docstring
} else if tok.is_comment() {
	Context::Comment
} else {
	Context::String
}

or even using a match

let context = match tok {
	Tok::String => if is_docstring { Context::Docstring } else { Context::String },
	Tok::FStringMiddle => Context::String,
	Tok::Comment => Context::Comment,
	_ => unreachable!()
};

[dhruvmanila]

Thanks!

[MichaReiser]

Does this "state machine" need any updating to correctly handle fstrings?

[dhruvmanila]

No, as f-strings cannot be used as docstrings. The state machine is mainly used for docstring detection.

[MichaReiser]

That's correct. But does it need any updating to not get confused by the new tokens emitted by the lexer?

[dhruvmanila]

Good point. No, it doesn't. The reason being that the detection mechanism resets the state if it finds any token other than the string token after a function / class / module.