Improve auto-indentation behaviour

[balta2ar]

Environment data

VS Code version: Version 1.19.0
Commit 816be6780ca8bd0ab80314e11478c48c70d09383
Date 2017-12-14T09:56:48.842Z
Shell 1.7.9
Renderer 58.0.3029.110
Node 7.9.0
Architecture x64

Python Extension version: 0.9.1
Python Version: Python 3.6.3
OS and version: ArchLinux

Case 1

The cursor is at | position (about to add second argument to print), and I press Enter:

Actual

def case1():
    print('first_argument', |)

It becomes:

def case1():
    print('first_argument', 
    |)

Expected 1

def case1():
    print('first_argument'
        |)

OR Expected 2 (this one is preferred)

def case1():
    print('first_argument'
          |)

Case 2

Actual

The cursor is right after the closing parenthesis and I press Enter.

def case2():
    print(
        'some_very_very_very_very_long_function_argument_',
        'second_argument')|

It becomes:

def case2():
    print(
        'some_very_very_very_very_long_function_argument_',
        'second_argument')
        |

Expected

def case2():
    print(
        'some_very_very_very_very_long_function_argument_',
        'second_argument')
    |

Case 3

Actual

The cursor after the comment. Press Enter:

def case3(): # some comment|

It becomes:

def case3(): # some comment
|

Expected

def case3(): # some comment
    |

Same with comments after for loops:

def case3(): # some comment
    for i in range(1): # another comment|

after pressing Enter becomes

def case3(): # some comment
    for i in range(1): # another comment
    |

Expected:

def case3(): # some comment
    for i in range(1): # another comment
        |

Also look at #3284

[balta2ar]

Added case 3.

[balta2ar]

Case 3 actually has been reported in #385.

[MikhailArkhipov]

Case 3 is fixed by #385
Case 1 is a matter of taste. TypeScript or C# don't auto-indent to arguments. Also, some people prefer their own continuation rules - by opening brace, by first letter, by comma as in this style

print(a
       , b

Case 2 requires AST analysis and extension currently has no facilities for this.

[DonJayamanne]

Case 1 is a matter of taste

Agreed. Guess this is why VS has options to configure how these things get formatted.

[balta2ar]

Case 1 is a matter of taste. TypeScript or C# don't auto-indent to arguments

Really? That's surprising to me. I'm so used to such indentation in vim and I find it very natural to have. Think about it: you're inside parenthesis, typing function arguments. Why would you not want to have an indentation, at least any of the two? It's a continuation of a function call and we always indent them, aren't we?

Guess this is why VS has options to configure how these things get formatted

Could you please direct me to such configuration? I looked through the VSCode settings filtering them by "indent" and "format" and only found this relevant option: "editor.autoIndent": true,.

Case 2 requires AST analysis and extension currently has no facilities for this.

Are there any plans to add such facilities? Maybe this case could be moved out into a separate issue so that you could keep it on your backlog if it's not possible to implement it with the current facilities?

[DonJayamanne]

Could you please direct me to such configuration?

May be i was wrong, I just assumed this was one of the settings. Could be something in Reshparer. Either way I could be mistaken.

[MikhailArkhipov]

@balta2ar - it depends on the language and I meant big VS IDE. C# simple preserves user indentation of arguments. By default it indents one level deeper and not by ( or first argument

In R there is setting

Yes, there are plans to have AST analysis, but this is pretty big work item. Working on it :-)

[balta2ar]

Yes, there are plans to have AST analysis, but this is pretty big work item. Working on it :-)

That's great! Thanks for giving a heads up! Oh, and speaking of AST. There is no doubt you know how to do it right, but I noticed this comment today: atom/atom#13877 (comment)
There is this thing https://github.com/tree-sitter/tree-sitter:

Tree-sitter is a C library for incremental parsing, intended to be used via bindings to higher-level languages. It can be used to build a concrete syntax tree for a program and efficiently update the syntax tree as the program is edited. This makes it suitable for use in text-editing programs

I haven't tried it in practice myself yet, but it sounds promising from the description. Just spreading the word :-)

[MikhailArkhipov]

I can ensure you we have lots of parsers here at MS :-).

[gasteigerjo]

I agree with case 1. It makes sense to have the default indentation behavior comply with PEP8 (https://www.python.org/dev/peps/pep-0008/#indentation).

[tino]

Case 1 is a matter of taste. TypeScript or C# don't auto-indent to arguments. Also, some people prefer their own continuation rules - by opening brace, by first letter, by comma as in this style

Well, isn't this lib about Python? It has PEP8 which has rules about indentation. There is some room left for matter of taste on where to add linebreaks in the case of long lines, but in this case the user makes a choice about that linebreak.

[brettcannon]

@tino PEP 8 is technically the style guide for Python's standard library, it just so happens people often adopt it as their own style guide. That isn't saying that we shouldn't aim to follow PEP 8, just that we also have to be flexible for those that choose to not follow PEP 8.

[tino]

Fair enough. But this also isn't a formatter or a linter, it's just a default indentation. You can always remove the spaces, as you now have to insert them. And then defaulting to the default Python style guide doesn't seem like a bad idea to me :D.

[cbrnr]

Atom has a nice plugin that works perfectly: https://atom.io/packages/python-indent

IIRC the author tried to implement the same functionality for VS code, but concluded that it wasn't possible 😞.

[krisjobs]

I think it's still not there :/

[kbrose]

Hello, I'm the person who wrote the parser for the python-indent package linked above, and I'm trying to replicate the behavior in VSCode. I believe it can be done with OnTypeFormattingEditProvider but I'm running into trouble using it (I have next to 0 javascript experience).

Using the extension code below, I never see the console.log("hi!") show up in the developer tools even after typing the activation character a while running the extension. Any tips on what I may be doing wrong?

'use strict';
import * as vscode from 'vscode';

// const PY_MODE: vscode.DocumentFilter = { language: 'python', scheme: 'file' };
const PY_MODE: vscode.DocumentFilter = {};

class PythonIndent implements vscode.OnTypeFormattingEditProvider{
    public provideOnTypeFormattingEdits(
        document: vscode.TextDocument, position: vscode.Position, ch: string,
        options: vscode.FormattingOptions, token: vscode.CancellationToken):
        vscode.TextEdit[] {
            console.log("hi!");
            let pos = new vscode.Position(0, 0);
            let x = new vscode.TextEdit(new vscode.Range(pos, pos), "");
            return [x];
        }
}

export function activate(ctx: vscode.ExtensionContext): void {
    ctx.subscriptions.push(
        vscode.languages.registerOnTypeFormattingEditProvider(
            PY_MODE, new PythonIndent(), "a"));
}

[cbrnr]

I don't have experience in Javascript either, but maybe @DSpeckhals can comment?

[kbrose]

I assume Dustin is still quite happy with Atom. The only reason I'm looking to do anything with VSCode is b/c I routinely need to open large text files which Atom still struggles real hard with.

[DSpeckhals]

I started going down the path of implementing similar indentation as the Atom package into a VS Code extension, but as of several months ago, came up short. If I remember correctly, there wasn't (and perhaps still isn't) a good hook to capture the end of line event, then insert the correct indentation characters on the newline event.

If there's something that will work in VS Code that I'm unaware of, I'd love to do it here too. I just haven't invested the time to dig recently. If there were good API's, I'd probably separate the core indentation logic that was written by @kbrose into a separate repository and npm package, then write the implementation details for both Atom and VS Code.

[kbrose]

Thanks for looking at this @DSpeckhals!

I was looking at the API, and there is certainly a lot less exposed than w/ Atom. No sort of newline event to hook onto. I feel like OnTypeFormattingEditProvider is the API we'd want to use, theoretically you may be able to trigger based off '\n' and then in the provider's logic you should be able to do anything you want. In practice, I couldn't get a toy example working with OnTypeFormattingEditProvider (my above comment).

I may bang my head against it more this weekend and see if I can make something work.

[karrtikr]

Can folks please try Pylance and let us know if they still reproduce these issues? Add:

"python.languageServer": "Pylance"

to settings.json.

[brettcannon]

Can folks please try Pylance and let us know if they still reproduce these issues? Add:

"python.languageServer": "Pylance"

to settings.json.

Specifically, see the instructions and announcement at https://devblogs.microsoft.com/python/python-in-visual-studio-code-december-2022-release/#auto-indentation-with-pylance for Pylance to do more intelligent indenting.

[amueller]

I added the languageServer line and the "editor.formatOnType": true, but still see the linebreak behavior issues.
Is there a way to confirm I set it up correctly?

Repro:

def f(a,<newline>

jumps to the beginning of the next line, instead of to the opening parenthesis indentation.
Same in similar settings (calling a function, in an if clause, ...), and also if the parenthesis is closed.

Thank you for working on this :)

[karrtikr]

@amueller Thanks for checking! Could you please file this as an issue on https://github.com/microsoft/pylance-release?

[brettcannon]

@amueller how about syntactic indent/dedent? E.g. typing else while in an if block? Are the issues just stylistic and not syntactic?

[amueller]

What is the expected outcome of typing else in an if block?

this is when not doing any manual indentation.

[brettcannon]

I would expect it to dedent (which it did for me):

If that isn't what's happening for you, can you file a bug on Pylance?

[amueller]

I commented on the existing issue. Having syntactic correct indent would be great. Ideally the stylistic ones would also work, though; I think that's basic pep8 that people that don't even know about pep8 instinctively follow.

[brettcannon]

I commented on the existing issue. Having syntactic correct indent would be great. Ideally the stylistic ones would also work, though

One thing at a time. 😉 Ultimately it's up to the Pylance team to decide how far they want to take this, but I'm personally supportive of them do simple, reasonable stylistic formatting as long as people realize they will very likely not get knobs to tweak anything (I would argue you should be using a formatter in that case).

[brettcannon]

FYI if you are not getting automatic indentation with Pylance by turning on formatOnType, we think it might be a bug. Please try the pre-release version of Pylance to see if that fixes it for you.

[drorata]

I switched to the pre-release (v2023.1.11), I have:

{
    "[python]": {
        "editor.formatOnType": true
    },
    "python.languageServer": "Pylance"
}

in my .vscode/settings.json, and it still doesn't work.

I'm still seeing the test environment that I mentioned in microsoft/vscode-jupyter#12501 - It might be related.

[amueller]

So I go to the command palette and open user settings and change the editor.formatOnType to true, then I close the editor, open it again, check the settings and I see:

That seems odd. I'm also on v2023.1.11 now.

Depending on how I open and close editor windows that seems to be non-deterministic though (I could reproduce several times, now I can't reproduce).
I still don't get the de-indent for else.

[amueller]

ok open and closed settings again and it was gone again (this time without the comma for some reason)

[brettcannon]

It seems we had a bug in the Python extension that wasn't automatically opting people into the feature when formatOnType was explicitly set. We are planning to do a point release today that should hopefully fix this for our stable users, but if you are using the latest pre-release build of the Python extension as of yesterday it should fix it for you.

[brettcannon]

ok open and closed settings again and it was gone again (this time without the comma for some reason)

The trailing comma makes it invalid JSON, so there should have been a red squiggle and a warning that your settings were invalid.

[amueller]

There was a red squiggle but I didn't see a warning. Not sure if my description was clear: I copy & pasted the setting, saved it, closed VSCode, opened VSCode, open the settings, and now my settings were clearly invalid JSON.

[luabud]

@amueller sorry -- would you mind confirming if you still see this using v2023.1.10111008 of the Python extension? I was able to reproduce this with an older version, but it does no longer happen for me with the newest ones:

We merged #20480 n the Python extension a couple of days ago, which we believe has the fix for the problem you're seeing.

[brettcannon]

There was a red squiggle but I didn't see a warning. Not sure if my description was clear: I copy & pasted the setting, saved it, closed VSCode, opened VSCode, open the settings, and now my settings were clearly invalid JSON.

This was part of the bug that was fixed. We are pushing a point release today that should fix this in stable while it was fixed in our pre-release a few days ago.

[amueller]

After switching to the insider version I was able to install v2023.1.10111008 and I don't see the removal of the config flag any more, and the as-you-type indentation of else works as expected. Thank you all for your help!

[luabud]

We have enabled format on type for all Python files by default when using Pylance. For those who want to disable this behaviour, you can add the following setting to your settings file (View > Command Palette... > Preferences: Open User Settings (JSON)):

 "[python]": {
        "editor.formatOnType": false
    } 

If you have any problems or feedback, please open a separate issue at the Pylance repo detailing the behaviour you'd like to see 😊