Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Slow response when system is offline #1000

Closed
okian opened this issue Dec 4, 2015 · 23 comments
Closed

Slow response when system is offline #1000

okian opened this issue Dec 4, 2015 · 23 comments
Assignees
Labels
bug Issue identified by VS Code Team member as probable bug important Issue identified as high-priority
Milestone

Comments

@okian
Copy link

okian commented Dec 4, 2015

When system is connected to network but don't have access to internet (just local network) navigation, move , copy , rename in directories and files become very slow. it's happening periodically and seems like it's waiting for kind of timeout.

image

@bpasero bpasero added the bug Issue identified by VS Code Team member as probable bug label Dec 4, 2015
@bpasero bpasero added this to the Backlog milestone Dec 4, 2015
@bpasero
Copy link
Member

bpasero commented Dec 4, 2015

@moostad when this happens can you open chrome tools (Help > Toggle Developer Tools), open the Profile tab and run a CPU profile to see where the time is spend?

@bpasero bpasero modified the milestones: Dec 2015, Backlog Dec 4, 2015
@okian
Copy link
Author

okian commented Dec 4, 2015

@bpasero I don't see any problem in CPU profile (or most likely I'm not qualified to read it) but I can send it to you if it can help. the only thing I noticed is every time that's happening two error will print in console.

1: Error: getaddrinfo ENOTFOUND vortex.data.microsoft.com vortex.data.microsoft.com:443 at errnoException (dns.js:25:10) at GetAddrInfoReqWrap.onlookup as oncomplete

2: Error: getaddrinfo ENOTFOUND dc.services.visualstudio.com dc.services.visualstudio.com:80 at errnoException (dns.js:25:10) at GetAddrInfoReqWrap.onlookup as oncomplete

@jrieken
Copy link
Member

jrieken commented Dec 4, 2015

btw - this is issue #1000 🎉

@bpasero
Copy link
Member

bpasero commented Dec 4, 2015

@moostad if you are able to capture a CPU profile when the explorer shows the spinning progress you can send it to me.

I think the ENOTFOUND is from our telemetry reporting. @sofianhn any chance this could slowdown operations? We don't send telemetry sync do we?

@sofianhn
Copy link

sofianhn commented Dec 4, 2015

@bpasero we don't send telemetry sync, those errors are expected to be on the console. I just tried to reproduce it with no luck.

@moostad please try disable telemetry as described here and let us know if you see any difference.

@okian
Copy link
Author

okian commented Dec 4, 2015

@bpasero here's the CPU profile.
CPU-20151204T235311.cpuprofile.zip

@okian
Copy link
Author

okian commented Dec 5, 2015

@sofianhn I disabled telemetry for about an hour and problem solved and as soon as I enabled that again, I get spinning progress 4 times in less than 10 minutes.

@sofianhn
Copy link

sofianhn commented Dec 5, 2015

Interesting...thanks @moostad for the quick response. I am guessing you are on Mac, after being offline for a while please run open $TMPDIR/appInsights-node from the terminal and verify what exists in this directory. You should see a bunch of json files, I am interested how many and what's their sizes.

@bpasero I verified that the caching mechanism by AppInsights is async. My guess now that it might be writing to the disk too often ~15 seconds which might be causing this issue. I will take it from here.

@sofianhn sofianhn assigned sofianhn and unassigned bpasero Dec 5, 2015
@okian
Copy link
Author

okian commented Dec 5, 2015

@sofianhn I will send those information later however I'm not sure that have anything to do with writing because I don't get spinning when system is disconnected from network (wifi is off) it's happening only when my mac is connected to the network without internet. I guess when it's see network connection then try to connect to website, at this time everything is blocked and I get spinning until that get http timeout.

p.s. I have no problem to understanding (reading and listening) English but writing and speaking is another story, so sorry about my poor English.

@okian
Copy link
Author

okian commented Dec 5, 2015

@sofianhn this time I tried on Ubuntu to be sure it's not OS/Device related, it's happening but not as often and not as long as Mac. maybe because it has much more power then the Mac (2x raid0 SSD for OS, 12 core CPU, 32 GB ram)

Here's the list of files

Total size: 1.5 MB
Count : 43

  45K Dec  5 11:16 1449301590361.ai.json
  21K Dec  5 11:16 1449301607507.ai.json
  44K Dec  5 11:17 1449301669096.ai.json
 8.0K Dec  5 11:18 1449301685054.ai.json
  28K Dec  5 11:18 1449301701515.ai.json
  28K Dec  5 11:18 1449301722262.ai.json
  14K Dec  5 11:19 1449301741035.ai.json
  18K Dec  5 11:19 1449301757832.ai.json
 5.4K Dec  5 11:19 1449301776224.ai.json
  65K Dec  5 11:20 1449301804942.ai.json
  54K Dec  5 11:20 1449301820250.ai.json
  19K Dec  5 11:20 1449301854519.ai.json
  17K Dec  5 11:21 1449301877099.ai.json
  26K Dec  5 11:21 1449301897507.ai.json
  16K Dec  5 11:21 1449301913055.ai.json
  20K Dec  5 11:22 1449301928942.ai.json
  18K Dec  5 11:22 1449301946785.ai.json
  46K Dec  5 11:22 1449301975527.ai.json
  23K Dec  5 11:23 1449301990779.ai.json
  43K Dec  5 11:23 1449302007367.ai.json
  52K Dec  5 11:23 1449302031358.ai.json
  15K Dec  5 11:24 1449302047260.ai.json
  46K Dec  5 11:24 1449302067154.ai.json
  41K Dec  5 11:24 1449302082161.ai.json
  54K Dec  5 11:25 1449302107280.ai.json
  37K Dec  5 11:25 1449302122366.ai.json
  17K Dec  5 11:25 1449302142179.ai.json
  35K Dec  5 11:25 1449302159539.ai.json
  34K Dec  5 11:26 1449302182334.ai.json
 5.4K Dec  5 11:26 1449302198281.ai.json
  25K Dec  5 11:26 1449302215101.ai.json
  38K Dec  5 11:27 1449302231171.ai.json
  25K Dec  5 11:27 1449302248680.ai.json
  35K Dec  5 11:27 1449302264804.ai.json
  19K Dec  5 11:28 1449302280982.ai.json
  66K Dec  5 11:28 1449302299938.ai.json
  69K Dec  5 11:28 1449302315167.ai.json
  37K Dec  5 11:28 1449302330401.ai.json
  31K Dec  5 11:29 1449302349368.ai.json
 9.5K Dec  5 11:29 1449302369790.ai.json
 6.8K Dec  5 11:29 1449302399153.ai.json
  80K Dec  5 11:30 1449302418946.ai.json
  22K Dec  5 11:30 1449302433991.ai.json

@sofianhn
Copy link

sofianhn commented Dec 5, 2015

thanks @moostad for the clarification and the list of files. The update rate and the size of files seems reasonable to me, writing 3 small files a minute should not be a problem. Now that IO is less likely to have caused this, I will work on producing the case with network and without internet to see what might be going wrong here.

You english is great btw, we really appreciate working with us to get this issue fixed.

@sofianhn
Copy link

The electron folks confirmed the issue on their side. Until this issue get fixed in electron and we get the new build, the workaround is to disable telemetry.

@egamma egamma added the upstream Issue identified as 'upstream' component related (exists outside of VS Code) label Dec 16, 2015
@egamma egamma modified the milestones: Jan 2016, Dec 2015 Dec 16, 2015
@egamma
Copy link
Member

egamma commented Dec 16, 2015

We will not update to a new electron build for the December update. Therefore I'm moving the milestone to January.

The work around for how to disable telemetry is document here: https://code.visualstudio.com/Docs/supporting/FAQ#_how-to-disable-telemetry-reporting

@bpasero bpasero modified the milestones: Jan 2016, Feb 2016 Jan 29, 2016
@joaomoreno joaomoreno modified the milestones: March 2016, Feb 2016 Feb 24, 2016
@joaomoreno
Copy link
Member

Nothing happened in February for this. Ping @sofianhn

@sofianhn
Copy link

I pinged in the upstream, still waiting

@egamma egamma modified the milestones: April 2016, March 2016 Mar 17, 2016
@egamma
Copy link
Member

egamma commented Mar 17, 2016

We will not update to a new electron for March, shifting to April

@bpasero
Copy link
Member

bpasero commented Apr 22, 2016

This is super super bad, I just hit this while in transit. We have to do something about it for April, it feels you are sitting in front of a web browser that tries to connect to the file that is on disk.

@bpasero bpasero added the important Issue identified as high-priority label Apr 22, 2016
@bpasero bpasero removed the upstream Issue identified as 'upstream' component related (exists outside of VS Code) label Apr 22, 2016
@bpasero
Copy link
Member

bpasero commented Apr 22, 2016

@Microsoft/vscode to get some attention and maybe collect ideas. We could maybe:

  • take out telemetry from the critical path of opening a file
  • detect that the system is offline and just not try to connect (Electron has APIs for this)

@jrieken
Copy link
Member

jrieken commented Apr 22, 2016

+2 the telemetry service should not run in the main thread but like search and fs watch in a separate process

@jrieken
Copy link
Member

jrieken commented Apr 22, 2016

as an easy workaround we can use a setTimeout trick here to get off the critical path.

@bpasero
Copy link
Member

bpasero commented Apr 22, 2016

I find it surprising that every call to telemetry gets send to the appenders right away. I would have at least expected some kind of buffering and batching. Adding some way of buffering with timeouts at the point where we collect these events seems like a good thing to do anyway.

Still, for the actual network access I think we should just turn off network access when we detect we are offline.

@jrieken
Copy link
Member

jrieken commented Apr 22, 2016

I find it surprising that every call to telemetry gets send to the appenders right away.

I think that's fair. The appenders are the ones that know what debouncing strategy to apply. However, the telemetry service should guard itself against evil/slow appenders.

@bpasero
Copy link
Member

bpasero commented Apr 23, 2016

Thinking more about the issue I have to agree that we need to move telemetry reporting into a separate process because no matter how smart we get, the blocking issue will always be an issue on the renderer side.

@aeschli aeschli modified the milestones: May 2016, April 2016 Apr 27, 2016
jrieken added a commit that referenced this issue May 10, 2016
jrieken added a commit that referenced this issue May 10, 2016
…ice, introduce commonProperties.ts for *all* common properties, tests!, getting closer to #1000
@jrieken jrieken mentioned this issue May 10, 2016
1 task
@vscodebot vscodebot bot locked and limited conversation to collaborators Nov 18, 2017
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
bug Issue identified by VS Code Team member as probable bug important Issue identified as high-priority
Projects
None yet
Development

No branches or pull requests

7 participants