Orphaned extension host instances after running tests

[StephenWeatherford]

Issue Type: Bug

I don't have repro steps right now. If this is an unknown issue and you can't repro, let me know, I'll find something.

Very frequently when running suites, if the suites don't end successfully, the extension host instance doesn't not get closed. This is a regression from stable version.

VS Code version: Code - Insiders 1.39.0-insider (8b6e0ef, 2019-09-19T03:15:21.720Z)
OS version: Windows_NT x64 10.0.18362

System Info

Item	Value
CPUs	Intel(R) Xeon(R) W-2133 CPU @ 3.60GHz (12 x 3600)
GPU Status	2d_canvas: enabled flash_3d: enabled flash_stage3d: enabled flash_stage3d_baseline: enabled gpu_compositing: enabled metal: disabled_off multiple_raster_threads: enabled_on oop_rasterization: disabled_off protected_video_decode: unavailable_off rasterization: enabled skia_renderer: disabled_off surface_control: disabled_off surface_synchronization: enabled_on video_decode: enabled viz_display_compositor: enabled_on viz_hit_test_surface_layer: disabled_off webgl: enabled webgl2: enabled
Load (avg)	undefined
Memory (System)	31.73GB (12.18GB free)
Process Argv
Screen Reader	no
VM	0%

Extensions (9)

Extension	Author (truncated)	Version
gitlens	eam	10.0.1
vscode-cosmosdb	ms-	0.11.0
vscode-docker	ms-	0.8.1
remote-ssh	ms-	0.46.1
remote-ssh-edit	ms-	0.46.1
remote-ssh-explorer	ms-	0.46.1
azure-account	ms-	0.8.4
csharp	ms-	1.21.3
vscode-typescript-tslint-plugin	ms-	1.2.2

[StephenWeatherford]

BTW, this is when debugging a vscode extension with vscode.

[StephenWeatherford]

Also, if I'm on a breakpoint, pressing F5 frequently shows the error "There is already a debug configuration "Launch Extension" running."

[chrmarti]

Is this only with tests?

[isidorn]

@StephenWeatherford for the F5 showing "there is already a..." this is duplicate of #81373 and will be fixed in tomorrows vscode insiders

As for the vscode window not closing. I can not reproduce this. Here are my steps:

1. I have a vscode starter extension sample
2. I run the extension tests. Window opens and nicely closes

Let me know if you are doing anything differently (gif attached)

We heard other reports of this, so there is some issue here however it would be great if we could find some steps to track this down.

Thanks for using vscode insiders!

[StephenWeatherford]

Happening quite a lot with my extension, although inconsistently. Try this:

1. git clone https://github.com/microsoft/vscode-azurearmtools.git
2. cd vscode-azurearmtools
3. npm i
4. Load folder in vscode and F5 with configuration "Launch Extension"
5. Open a file, change language to "Azure Resource Manager Template"
6. Go back to the original vscode instance and press the stop button
   !) Frequently for me, the extension host instance doesn't close

Try it multiple times, maybe with different lengths of time before stopping, hopefully it will repro for you.

[StephenWeatherford]

Any more info? This is extremely annoying, I keep ending up with 5 or 6 instances that I have to close.

[StephenWeatherford]

[alexdima]

I have also tried on Windows with the steps in #81397 (comment) but the window goes down every time I press the stop button

I could reproduce once by pressing stop very quickly after the window was launched...

[alexdima]

@isidorn I'm just pressing F5 and clicking the Stop button:

I peeked into the Dev Tools of the window that was open and it said the extension host is waiting for debugger to attach... So the extension host process is still running and therefore the window does not close itself...

@StephenWeatherford Could you please open developer tools in one of your left-over windows? Does anything show in there?

[StephenWeatherford]

Nope, debugger is attached.

[isidorn]

I can reproduce with @alexandrudima steps.
So start debugging, as soon as the window is opened click stop. The window is left orphaned, however in that case the Debugger is not attached. I can also reproduce this on Stable, so I am pretty sure this is not the issue that @StephenWeatherford is hitting.

Following exact @StephenWeatherford steps on my mac produces an error when I change language to "Azure Resource Manager Template". I get an error. After that if I click stop the second window is nicely closed.

Cannot find the ARM Template Language Server at /Users/isidor/Development/w/vscode-azurearmtools/languageServer/Microsoft.ArmLanguageServer.dll. Only template string expression functionality will be available.

On windows I get the same error when following @StephenWeatherford steps

[alexdima]

@StephenWeatherford Is there anything else we are missing to reproduce on our side? I haven't seen the log statement "Initializing dotnet acquire..." when I tried things out, maybe that has something to do with being able to repro ?

[StephenWeatherford]

I don't know what changed, but I haven't been able to repro since the update a couple days ago. Thanks for looking into it.

[StephenWeatherford]

It's back with Version: 1.40.0-insider (user setup)

[StephenWeatherford]

Doesn't seem to be reproing in the latest stable, just insiders.

[chrmarti]

Could this be caused by the Electron 6 update? /cc @bpasero

[isidorn]

@chrmarti good point. Yes, the timing works perfectly, since @StephenWeatherford mentinoed it stopped ocurring when we reverted back to previous electron, and now reprots it happens again when we updated again to 6.

[chrmarti]

/cc @deepak1556 for the Electron 6 lead.

[alexdima]

@bpasero We looked at this here together with @isidorn on my machine. We have noticed that the extension host is still running, and is still paused at the breakpoint. We have connected to the extension host process from Chrome via its inspector and once we pressed continue, the extension host process ran to completion and the window closed.

So the issue is that pressing "stop" (the red box) is leaving the extension host in a paused state at the breakpoint. @isidorn has suggested that this should be investigated by @roblourens as it is a node-debug2 issue.

[isidorn]

@bpasero yes you are correct.
After investigating with @alexandrudima we have found out that the Extenino Host is not being shut down. It remains in the stopped state, since we connected to it via Chrome and verified it is still on the stopped breakpoint.
So the issue is that on Windows with newest electron the node debugger can not shut down the extenion host if it is stopped.
As a workaround the debugger could send the continue before shutting it down

[roblourens]

I can repro this, but in my case it is using a random port, not 9229. It should be using a random port unless 9229 has been specified, do you know why that would happen @isidorn?

Also, I can't repro this in the exploration build, so it could be an electron issue too.

[roblourens]

Also, the node debug adapter does not try to kill the extensionhost, it is vscode's job to manage the window.

[isidorn]

@roblourens please note that we could not reproduce this with Electron 3 also. This only reproduces with Electron 5 and 6.
I do not know why port 9229 would be specified. We never set ports explicitly on the vscode side.

[alexdima]

@roblourens

Also, the node debug adapter does not try to kill the extensionhost, it is vscode's job to manage the window.

VS Code does manage the window. In this case, VS Code is launched with some flags to indicate extension tests. VS Code will wait for the extension host process to exit and then it will close the window.

But what appears to happen when pressing "Stop" while on a breakpoint is that the extension host process remains paused at the breakpoint. VS Code is not aware that the extension host has been debugged, and the debugger detached, and that the extension host is now permanently paused at a breakpoint.

This might be some weird node.js behavior (to leave the process paused at a breakpoint when the debugger detaches), but would it not make sense for the debugger (before detaching) to clean up a bit? I mean the debugger could remove its breakpoints and resume the process being debugged...

[roblourens]

The debug adapter doesn't kill the EH process, and if you're saying vscode waits for it to exit, I guess I'm not sure who is telling it to shut down. I can send Resume before detaching, that sounds harmless, although Node should take care of that for me.

[roblourens]

edit, I deleted some comments because I realized that I misunderstood the issue. Anyway, I was able to repro this just using chrome devtools.

I can file an electron bug if I can reduce it, but I also tried to repro this just using node, and using normal EH debugging (not tests) and couldn't. Do you have any idea why it would be different when running tests?

[isidorn]

@roblourens note that the Test launch configuration has a lot of env variables being set.
It might be that one of those somehow breaks the flow

"env": {
                "MOCHA_grep": "UDF Malformed", // RegExp of tests to run (empty for all)
                "MOCHA_invert": "0", // Invert the RegExp
                "AZCODE_ARM_IGNORE_BUNDLE": "1",
                "MOCHA_enableTimeouts": "0", // Disable time-outs
                "DEBUGTELEMETRY": "1", // 1=quiet; verbose=see telemetry in console; 0=send telemetry
                "NODE_DEBUG": "",
                "DISABLE_LANGUAGE_SERVER_TESTS": "0",
                "DISABLE_SLOW_TESTS": "0"
            }

[roblourens]

I've managed to repro this with normal EH debugging in our vscode-pwa extension several times. @deepak1556 said that he didn't see the extension development host window getting whatever message it should get to shut down. I was expecting to see ExtensionHostDebugChannelClient#close called when I click the stop button but I don't see that, so I guess I don't know how shutting down the window is supposed to work at all. Can @isidorn or @weinand explain or help me figure out where to set a breakpoint?

[weinand]

Clicking the debugger's Stop button doesn't do anything special: it just terminates the extension host node.js process via the debug adapter.

VS Code should pick this up as a dying EH process here.

And then VS Code should not restart the EH process (because it is in debug mode) and should "exit" VS Code (which eventually closes the EH window).

[roblourens]

Sorry for making this take longer than it should have, I insisted that the debug adapter doesn't kill the EH, and it totally is supposed to in EH debugging, different from normal "attach" configs.

I think I see the problem, now node is sometimes returning null for process.pid right when it starts up. I added a workaround for that and it seems to be working.

[isidorn]

@roblourens Since you added a workaround, shouldn't this be closed?

[StephenWeatherford]

It does indeed seem to be fixed with 1.41.0-insider (user setup)
Thanks!

[weinand]

Since @StephenWeatherford confirmed that this is now working for him and since I could no longer reproduce the issue, I'm adding the Verified tag.