Analysis engine should not analyze entire project

[DonJayamanne]

I don't think the analysis engine should analyze the entire workspace folder and report syntax errors.
Currently that's the default behaviour:

Problems:

• One might have tests files with errors in them for specific test cases (Analysis engine should not provide linter messages if linting is disabled #1809 (comment))
• There might be some scripts in a folder that are specific to python 2 or python 3 only, and the interpreter selected is the opposite (3 or 2, vice versa).

As a result we have a lot of noise and false positives being reported.
If the code base is quite large, then valid errors/messages returned by linters get buried in the noise.

I'm flagging this as a bug as it produces too many false positives, & would be too confusing for developers moving from Jedi to Analysis engine.

Solution:

• Provide setting to turn on/off the ability to scan entire workspace for syntax errors.

@qubitron @brettcannon cc

[MikhailArkhipov]

There is already some filtering for virtual environments residing in the workspace. Basically we just need some logic to determine which folders must be excluded. It is simpler in VS that has 'white list' model in most projects.

[patrys]

@MikhailArkhipov When I tested yesterday most errors reported were in fact from packages installed in my virtualenv (${workspaceRoot}/.direnv/python-3.6.5) so I don't think that filter works.

[brettcannon]

Having a path component of site-packages is the best solution I can think of off the top of my head.

[patrys]

I think the extensions is already aware of where the venv is? The other candidate would be node_modules as there are several Node packages using gyp as a build system.

[MikhailArkhipov]

PR is up microsoft/PTVS#4372

[DonJayamanne]

• Open PTVSD repo in VSC
• Select Python 3 interpreter in VSC
• 200+ of errors are reported in Problems Window (these are false positives as there is plenty of code written in PyDev that's Python 2 specific)

I'm still getting too many false positives.

Provide setting to turn on/off the ability to scan entire workspace for syntax errors.

[qubitron]

Should we disable full-workspace analysis by default to be consistent with current behavior? I haven't seen much ask for analyzing the entire project, mostly folks just want to see syntax errors are they're editing code.

[patrys]

Full-workspace analysis is a great feature when it's not overzealous. Consider this comment as me asking for it to be available 😄

[qubitron]

Agreed it's useful! We would have the option available, we just need to decide what the right default is -- it depends on how many people will get overwhelmed with results like Don's.

[patrys]

@qubitron It should be possible to add folders to the ignore list. I'm fine with spending some time up-front and I'm fine with "analyze the entire project" being off by default as long as it's a supported option.

[DonJayamanne]

It should be possible to add folders to the ignore list

Solutions:

• Sounds like a great candidate for a code action /c @MikhailArkhipov as well.
• New setting with a list of folders/files to be excluded

[MikhailArkhipov]

Ignore list is already supported. We pass

        const list: string[] = ['**/Lib/**', '**/site-packages/**'];
        this.getVsCodeExcludeSection('search.exclude', list);
        this.getVsCodeExcludeSection('files.exclude', list);
        this.getVsCodeExcludeSection('files.watcherExclude', list);
        this.getPythonExcludeSection('linting.ignorePatterns', list);
        this.getPythonExcludeSection('workspaceSymbols.exclusionPattern', list);

So you can add more patterns in either of those as needed.

As for new setting, or code actions and 'only report errors on active files' feature - please open separate issues so we can prioritize and track them appropriately.

I am going to close this one rather than recycling issues... ;-)

[DonJayamanne]

this.getVsCodeExcludeSection('files.watcherExclude', list);

That's the solution. Thanks

[MikhailArkhipov]

@DonJayamanne - PTVSD repo shows no errors... Apart fro Pylint.

[DonJayamanne]

I'm still getting the messages on the Mac with Python 3.6.5 - 64Bit.
What version of Python do you have selected? Most probably 2.7

[MikhailArkhipov]

Yes, but these are legal. You also get bunch for 'long' ints in 3.x. They are syntax errors for the selected interpreter.

[MikhailArkhipov]

Opened #2030