📋 Formatter: Syntax Formatting

[MichaReiser]

This task coordinates the work on implementing formatting for the Python syntax.

Please create an issue and comment that you intend to work on before starting a new syntax. This allows us to better coordinate the work.

Module

• Module (includes TypeIgnore)  #4800
• Formatter: Interactive
• Formatter: Expression Format ModExpression #5689
• Formatter: FunctionType

Note: Our infrastructure only supports testing Module. We should, therefore, wait to implement the other variants.

Statements

• FunctionDef / AsyncFunctionDef (Includes Arguments, and Arg) #4799
• Formatter: ClassDef #4955
• Formatter: Return (Implement StmtReturn #4960)
• Formatter: Delete #5287
• Formatter: Assign (basic implementation done, missing is wrapping the left hand side like black which is hard)
• Formatter: AugAssign Format StmtAugAssign #5655
• Formatter: AnnAssign #5662
• Formatter: For / AsyncFor #5288
• Formatter: While #4806
• Formatter: If Statement #4956
• Formatter: With / AsyncWith (includes WithItem) #5368
• Formatter: Match (includes MatchCase and Patterns) #5834
• Format raise statement #5595
• Formatter: Try / TryStar (includes ExceptHandler) #5245
• Formatter: Assert (Format assert statement #5168)
• Formatter: Import Format import statements #5493
• Formatter: ImportFrom (includes Alias) Format import statements #5493
• Formatter: Global Add global and nonlocal formatting #6170 (deviation from black: break over multiple lines if it exceeds the line-width Line too long: Multi-line global psf/black#1357)
• Formatter: Nonlocal Add global and nonlocal formatting #6170 (deviation from black: break over multiple lines if it exceeds the line-width Line too long: Multi-line global psf/black#1357)
• Formatter: Expr Format StmtExpr #4788
• Formatter: Pass (Implement StmtPass #4959)
• Formatter: Break
• Formatter: Continue
• Formatter: TypeAlias #5929 (Add formatting of type parameters in class and function definitions #6161, Add formatting of type alias statements #6162)

Expressions

• Formatter: BoolOp #4962
• Formatter: NamedExpr Format assignment expressions (FormatExprNamedExpr) #5613
• Formatter: BinOp (hard) #4896
• Formatter: UnaryOp #5216
• Formatter: Lambda #5826
• Formatter: IfExp Format ExprIfExp (ternary operator) #5597
• Formatter: Dict #5206
• Formatter: Set
• format ExprListComp #5600
• Formatter: SetComp (depends on ListComp) #5674
• Formatter: DictComp (depends on ListComp) #5778
• Formatter: GeneratorExp (depends on ListComp) #5676
• Formatter: Await
• Formatter: Yield / YieldFrom (good first issue) #5916
• Formatter: Compare #5242
• Formatter: Call (Format call expressions (without call chaining) #5341, Formatter: Attribute and call chains "fluent interface" #5343)
• Formatter: F-Strings (FormattedValue and JoinedStr) #5913
• Formatter: Constant (Add basic Constant formatting #4954)
  • String constant formatting #5294
  • Bytes string formatting #6064 (Format bytes string #6166)
  • Tuple constant formatting
  • Float constant formatting Format numeric constants #5972
  • Int constant formatting Format numeric constants #5972
  • Complex constant formatting Format numeric constants #5972
  • Bool constant (good first issue)
  • Ellipsis constant (good first issue)
  • None constant (good first issue)
• Formatter: Attribute #5257 (Formatter: Attribute and call chains "fluent interface" #5343)
• Formatter: Subscript #5286
• Formatter: Starred #5650
• Formatter: Name #4802
• Formatter: List (Format binary expressions #4862)
• Formatter: Tuple (Format ExprTuple #4963)
• Formatter: Slice (Format Slice Expressions #5047)

Improvements

• Remove parentheses from tuples in comprehension target #5779
• Comment placement of expressions with keywords #5780
• Remove unnecessary walrus operator parentheses #5781

Cross-cutting

• Formatter: Parenthesized expressions #5671
• Formatter: JoinNodesBuilder #4801
• Formatter: Docstrings #6196
• Formatter: Suppression comments #5336
• Formatter: Pragma comments #6197
• Introduce SuiteStatement #5326
• Decide on trailing comments formatting stop having end-of-line comments expand parent #5353

Bugs

See open issues

Notes

#4730 removed the prototype from @charliermarsh. It's worth taking a look at it when implementing new formatting because it already formatted a lot of syntaxes correctly.

[JonathanPlasse]

Alilas?

[MichaReiser]

Alilas?

It seems to be used in ImportFrom. I moved it to ImportFrom.

[MichaReiser]

If someone's interested in familiarizing themselves with the formatter API, then the following are probably good starters:

• Pass, Break, and Continue: I don't expect this to change the formatter output (also because we only support formatting while statements) but you can see how the IR and write macro work
• Constant::None and Bool: Same as for Pass...
• Constant::Ellipsis: this should be easy, but I believe black removes ellipsis sometimes. It would be good to understand when this is the case. We don't necessarily have to implement it in the first PR
• nolocal and global
• These may be good starts, but could also be difficult.
  • UnaryOp
  • Await
  • YieldFrom
  • Return
  • Raise
  • Assert
  • Delete

Let me know if you're interested.

For testing. We have tests that automatically compare our output with black's output. You can run them with cargo test and review the changes with cargo insta review. You can also add your own tests by adding a file to crates/ruff_python_formatter/resources/test/fixtures/ruff (it may sometimes be necessary to run a cargo clean in case your test doesn't get picked up)

What's worth testing:

• Comments in all possible positions :D

[addisoncrump]

Is the goal here to reach parity with black?

[MichaReiser]

Is the goal here to reach parity with black?

The goal is to be black-compatible except for pathological cases.

[addisoncrump]

Okay. Depending on how strictly you want to be having the exact same output as black, this could potentially be tested with differential fuzzing. If that's desirable, please let me know and I can write a harness for this 🙂

Otherwise, I will likely swing back around once this reaches a steady state and add an idempotency fuzzer.

[warsaw]

The goal is to be black-compatible except for pathological cases

I would dearly love at least the option to choose a different default quote character. At least on US keyboards, double quotes are ergonomically inferior. It's the number one reason why blue exists.

[charliermarsh]

We still plan to make the quote style and indentation style configurable, though the exact settings scheme etc. haven’t been designed yet.

[MichaReiser]

See #1904 for an in-depth discussion around the goal. The purpose of this task is to track the implementation work

[MichaReiser]

Some other good first issues could be

• Star expression
• Yield and Yield from

[LouisDISPA]

Hello, I'm interested in the formatter and while having a look at the code I did the raise formatter code. I'm not sure it's good but I can open a PR if you want. I took inspiration from the return formatter.

[MichaReiser]

Hello, I'm interested in the formatter and while having a look at the code I did the formatter code. I'm not sure it's good but I can open a PR if you want. I took inspiration from the return formatter.

Feel free to open a PR. On which node where you working?

[LouisDISPA]

Feel free to open a PR. On which node where you working?

Sorry I edited my comment, it was the StmtRaise node.
Ok I'll open the PR!

[cnpryer]

Any chance we can open an issue for lambda? I have a lambda branch but I'm not sure I'm ready to claim it. If I have any questions it'd be nice to ask there.

[MichaReiser]

Any chance we can open an issue for lambda? I have a lambda branch but I'm not sure I'm ready to claim it. If I have any questions it'd be nice to ask there.

Sure: #5826

[MichaReiser]

I'm closing this one. There's still some work left to be done but we'll track this as part of #6203