Skip to content

Conversation

@ladipro
Copy link
Member

@ladipro ladipro commented Sep 22, 2023

Fixes #9245

Context

The test was disabled to unblock PR CI.

Changes Made

Increased the relevant timeout.

Testing

The test is reliably passing now.

Notes

This turned out to be an issue with the sleep command we use on Windows. In some cases PowerShell can take a super long time to start. I have been able to reproduce locally by enabling Fusion logging. Thread times of the powershell process:

image

We spend almost 10 seconds just loading assemblies, so the timeout of 10 seconds for the entire build was not enough.

I don't have a full understanding of the mechanism that slows down PowerShell this much. At this point I'm happy we were able to confirm it's not a product issue, although I'm wondering if there is a better and more light-weight sleep command we could use on Windows instead (e.g. ping 127.0.0.1 -n <seconds>). Reviewers please opine.

EDIT: In my trace, file system operations block extensively with wdfilter.sys on the stack, so the likely explanation for the issue appearing all of a sudden is a Defender update.

@ladipro ladipro marked this pull request as ready for review September 26, 2023 09:20
@Forgind
Copy link
Contributor

Forgind commented Oct 6, 2023

Possibly idea from MiYanni:
Are they running Windows PowerShell? They'd likely see better startup times using PowerShell Core instead.

Defender also seems plausible, though 🙂

@JaynieBai JaynieBai merged commit f2bf4dd into dotnet:main Oct 11, 2023
bulatgrzegorz pushed a commit to bulatgrzegorz/selective-condition-evaluator that referenced this pull request Oct 16, 2023
…ngTargetExceptions (dotnet#9253)

Fixes dotnet#9245

Context
The test was disabled to unblock PR CI.

Changes Made
Increased the relevant timeout.

Testing
The test is reliably passing now.

Notes
This turned out to be an issue with the sleep command we use on Windows. In some cases PowerShell can take a super long time to start. I have been able to reproduce locally by enabling Fusion logging. Thread times of the powershell process:

image

We spend almost 10 seconds just loading assemblies, so the timeout of 10 seconds for the entire build was not enough.

I don't have a full understanding of the mechanism that slows down PowerShell this much. At this point I'm happy we were able to confirm it's not a product issue, although I'm wondering if there is a better and more light-weight sleep command we could use on Windows instead (e.g. ping 127.0.0.1 -n <seconds>). Reviewers please opine.

EDIT: In my trace, file system operations block extensively with wdfilter.sys on the stack, so the likely explanation for the issue appearing all of a sudden is a Defender update.
MichalPavlik pushed a commit that referenced this pull request Oct 17, 2023
…ngTargetExceptions (#9253)

Fixes #9245

Context
The test was disabled to unblock PR CI.

Changes Made
Increased the relevant timeout.

Testing
The test is reliably passing now.

Notes
This turned out to be an issue with the sleep command we use on Windows. In some cases PowerShell can take a super long time to start. I have been able to reproduce locally by enabling Fusion logging. Thread times of the powershell process:

image

We spend almost 10 seconds just loading assemblies, so the timeout of 10 seconds for the entire build was not enough.

I don't have a full understanding of the mechanism that slows down PowerShell this much. At this point I'm happy we were able to confirm it's not a product issue, although I'm wondering if there is a better and more light-weight sleep command we could use on Windows instead (e.g. ping 127.0.0.1 -n <seconds>). Reviewers please opine.

EDIT: In my trace, file system operations block extensively with wdfilter.sys on the stack, so the likely explanation for the issue appearing all of a sudden is a Defender update.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[Bug]: Investigate why IdenticalSubmissionsShouldCompleteAndNotHangTheBuildOnMissingTargetExceptions started failing

4 participants