Intellisense stuck indexing when file is in an uncommon path

[Foxandxss]

• VSCode Version: 1.1.1
• OS Version: OSX El Capitan, Windows

Steps to Reproduce:

Hello, I work for angular.io and we have an issue regarding intellisense.

We have several TS projects inside public/docs/_examples.

That folder contains at least 20 TS projects inside. To not have each of them with their own tsconfig.json, package.json, node_modules, etc, we have those at public/docs/_examples and with a gulp task, we copy all the files to each of those projects (we symlink the node_modules).

So at the end, we have like 20 TS projects, 20 node_modules (in reality 20 symlinks), 20 tsconfig.json, etc.

What's the problem here? If we open a file like e2e-spec.js (is inside each TS project but in a parent directory from the tsconfig.json.), and you over something, you get a "Loading..." popup and never get any hover information nor intellisense.

I believe VSCODE is trying to index all those 20 node_module folders before show anything. A few processes at 100% could confirm that.

If I fresh open VSCODE, go to a file inside any of those projects and I hover something, it works perfectly (because it has a tsconfig.json that would exclude node_modules), but as soon as I hover something in a file that doesn't have a tsconfig for it, it will try to index all the project and blow up.

1. git clone https://github.com/angular/angular.io
2. cd angular.io
3. npm install
4. cd public/docs/_examples
5. npm install
6. cd ../../..
7. gulp add-example-boilerplate
8. open editor, go to public/docs/_examples/architecture/e2e-spec.js and hover something. Loading....
9. close the editor, wait for all Electron processes to die.
10. open VSCODE, go to public/docs/_examples/architecture/ts/app/main.ts and see how it works.
11. Go to any e2e-spec.js to trigger this massive index and if you go back to the main.ts it doesn't work anymore.

[wardbell]

Let me elaborate. This isn't just a problem with 20 folders each with 20 node_modules. It's a problem with ONE project with a single node_modules.

Imagine a structure like this:

MyProject
|_ ts
    |_ app
       |_ app.component.ts
       |_ main.ts
    |_ node_modules ...
    |_ typings ...
    |_ index.html
    |_ tsconfig.json
    |_ ...
|_ e2e.spec.ts

Observe that the e2e.spec.ts is a sibling of ts, above the tsconfig.json.

MyProject
|_ ts ...
|_ e2e.spec.ts

When I open main.ts, intellisense is almost immediate.

When I open e2e.spec.ts, intellisense is frozen while VS Code reports Loading.... The processor pegs, the fan blows, and there is no intellisense anywhere in this project ... including back in main.ts where it was working mere moments ago. Minutes pass.

I give up, return to the app folder, reload VS Code, and intellisense returns.

Why the funny spec location?

You may ask

Why is e2e.spec.ts a sibling of ts instead of inside ts?

The word ts is your clue: it stands for TypeScript and, in fact there other versions of the app written in other languages (e.g, es6, js). The end-to-end spec is one suite of tests that applies to all.

FWIW, I tried adding a tsconfig.json that exclude node_modules_ at the same level as thee2e.spect.ts`. It didn't help.

This arrangement is obviously exotic and rare. Why worry yourself?

Perhaps others will have some reason to put a file at a level above node_modules and suffer a similar fate albeit from a different cause.

Surely we can find a solution that isn't the VS Code equivalent of the infamous Steve Jobs answer: You're holding it wrong.

[dbaeumer]

I cloned the repository and here is what I see following the steps from above which IMO is not so bad:

I also noticed that VS Code shows you a tip at the buttom to exclude large folders like node_modules. I checked the setup and they are actually not excluded. Excluding them removes them from 'indexing' You can still import them and code assist works for imported modules however symbols from node_modules will not be part of the global symbol index (Ctrl+T)

[Foxandxss]

That sounds promising.

Did you install the internal node_modules? I can't see that from your gif. Without internal node_modules it is fast like we see in your gif.

If we add those 20+ node_modules folders, the Loading... take ages.

I will research on the excluding things :)

[Foxandxss]

I had that green text to exclude yesterday but I can't see it anymore. Can I trigger that somehow?

[dbaeumer]

The screen cast is without any additional exclude properties. I added them locally but they didn't improve things to drastically. The exclude should go into the closest tsconfig.json / jsconfig.json. In our case the one in the _example folder.

{
  "compilerOptions": {
    "target": "es5",
    "module": "commonjs",
    "moduleResolution": "node",
    "sourceMap": true,
    "emitDecoratorMetadata": true,
    "experimentalDecorators": true,
    "removeComments": false,
    "noImplicitAny": true,
    "suppressImplicitAnyIndexErrors": true
  },
  "exclude": [
    "node_modules"
  ]
}

However VS Code (or better the tsserver) still complains about to many files. Is there any reason why you don't have a tsconfig.json per example. The examples are self contained. And as said a tsconfig.json with excludes doesn't mean TS will not resolve imports from node_modules anymore.

[dbaeumer]

I tried the tsconfig.json per example (by creating a tsconfig.json inside the architecture folder) and that makes it actually a snappy experiences.

[dbaeumer]

@wardbell do you exclude the node_modules folder in the tsconfig.json file. If so, can I clone your project somewhere so I can have why are you seeing such a slow behavior.

[Foxandxss]

Yes, but each example should contain their own node_modules:

Like that. Since each example has a node_modules inside, it adds to the "Loading..." time. Can take hours / 100% cpu and it never finishes.

Also tried with the exclude, but I guess that there are certain files (like those e2e-spec files) that doesn't have a tsconfig in the same folder so they don't have any exclude.

[dbaeumer]

JS files are handled by the tsserver as well. So if there is a tsconfig.json file the exclude matters even for JS files. I noticed that you have a tsconfig and a jsconfig. @jrieken do you know which one takes precedence if they are both there when opening a JS file. It is my understanding that a tsconfig.json always wins. At least that file gets open in that setup when clicking on configure excludes in the status line.

[dbaeumer]

In the example you provided did you put node_modules into the exclude of the tsconfig.json file. This should help a lot since the toh-6 example is pretty small. What a tsconfig.json file does is the following:

• it limits the project scope to the ts/js files that are in the same directory or a sub directory of the directory that contains the tsconfig.json file minus the directories listed in the excludes section.

Could you in this setup open VS Code directly on the toh-6 folder to avoid any additional unnecessary work done. TS features like hover, code complete should be more or less instant then.

[Foxandxss]

Opening VS Code directly on a example is not productive because we normally need to see the whole project when doing stuff. Still, I opened VS Code directly on toh-6 folder, put the exclude on tsconfig (which I thought it was ignoring node_modules by default) and still, Loading... for a few minutes.

[Foxandxss]

I understand we are all busy, there are lot of issues and perhaps this is our own fault, but if we could schedule a quick hangouts sometime to see it live, perhaps we can see the problem in a better way.

[dbaeumer]

The request to open VS Code on toh-6 was only to isolate the problem. And I do believe you that it is slow. Having a hangout makes sense but I would first like to have an idea what to look for which I don't right now :-(.

My toh-6 folder looks a little different than yours. To ensure that I do what you do, what do I need to get to that layout.

[Foxandxss]

1. cd /public/docs/_examples
2. npm install
3. cd ../../.. (basically back to the angular.io folder again)
4. gulp add-example-boilerplate

That will add those extra files to all those folders (also node_modules).

[dbaeumer]

Here we go. I missed the gulp add-example-boilerplate step when setting up. It was already in the description. Sorry for that.

[dbaeumer]

I was able to reproduce this when running gulp add-example-boilerplate. The problem is that all node?modules folders are scanned which makes the tsserver consume 500 MB of memory. Will look what the proper exclude rule should be

[dbaeumer]

TS behaves correctly when opening a ts file (for example angular.io\public\docs_examples\architecture\ts\app\app.component.ts) because the file is covered by the tsconfig.json file here: angular.io\public\docs_examples\architecture\ts\tsconfig.json which excludes the node_modules folder. However when opening e2e-spec.js it isn't covered by that tsconfig.json which creates a new project resulting in scanning the file system. Will investigate how to best configure this to make this a more happy experience.

[dbaeumer]

Here we go: using the following content for this angular.io\public\docs_examples\architecture\tsconfig.json file:

{
  "compilerOptions": {
    "target": "es5",
    "module": "commonjs",
    "moduleResolution": "node",
    "sourceMap": true,
    "emitDecoratorMetadata": true,
    "experimentalDecorators": true,
    "removeComments": false,
    "noImplicitAny": true,
    "suppressImplicitAnyIndexErrors": true
  },
  "exclude": [
    "dart",
    "ts/node_modules"
  ]
}

Makes everything work nicely in the architecture example. The problem was the node_modules folder in ts and the node_modules folder in dart (which took me a while to discover). Since you have quite some linked node_modules folders and it is already problematic with a single example all I can suggest is to setup the exclude rules as indicated. Otherwise the tsserver is simply looking at too many files.

@Foxandxss Let me know how it goes.

[Foxandxss]

Yes, adding a tsconfig at that level fixes it for that concrete example (so we would need to add one tsconfig.json at each "root level of each project".

Even if the ts/ folders doesn't have the tsconfig.json (they have it in the parent folder), the examples work, but we had the idea of be able to copy the ts/ folder of any folder somewhere else and have it working like an independent project. If we move the tsconfig to the parent config, that is not possible anymore.

Would be awesome if we could have one tsconfig at the very root that would ignore all node_modules in any folder.

/cc @wardbell

[wardbell]

No problem creating/maintaining as many tsconfig.json files as we need at any level specified.

I tried that first. But I wrote

  "exclude": [
    "node_modules"
  ]

rather than

  "exclude": [
    ...
    "ts/node_modules"
  ]

Does tsconfig support glob patterns?

FWIW, if no globbing support (and I think not as 1927 is still open), then we should write something like the following and put it at project root's for every sample:

{
  "compilerOptions": {
    "target": "es5",
    "module": "commonjs",
    "moduleResolution": "node",
    "sourceMap": true,
    "emitDecoratorMetadata": true,
    "experimentalDecorators": true,
    "removeComments": false,
    "noImplicitAny": true,
    "suppressImplicitAnyIndexErrors": true
  },
  "exclude": [
    "dart",
    "js/node_modules",
    "ts/node_modules"
  ]
}

We don't want to exclude all of the js samples from analysis, do we? Let me see if that works and I will report back.

Update

That worked! Too bad about the inability to glob but, fortunately, my tooling makes it easy to copy a master tsconfig.json around ... and exclude it from source control of course.

[dbaeumer]

Globs are currently not support in tsconfig. But as @wardbell already mentioned it is being worked on microsoft/TypeScript#1927
microsoft/TypeScript#8841

Having one top level tsconfig.json which excludes node_modules and is being merged with an active tsconfig is currently not supported either. The concept of a tsconfig.json is to define a 'project scope'.

If you feel stong about such a feature please open an issue in the TS repository. Something I could think of to support this is having an extend rule like ESLint (http://eslint.org/docs/user-guide/configuring#extending-configuration-files)

@wardbell @Foxandxss is it ok for you to close this issue. IMO any additional work as to be tracked in the TS repository.

[Foxandxss]

Yes, thank you for your quick responses. We are cooking our solution with your guidelines.