System.IO.Tests.DirectoryLongerThanMaxLongPathWithExtendedSyntax_ThrowsException test failing

[radical]

Build, and Log

    System.IO.Tests.DirectoryInfo_Create.DirectoryLongerThanMaxLongPathWithExtendedSyntax_ThrowsException [FAIL]
      Assert.All() Failure: 2 out of 2 items in the collection did not pass.
      [1]: Item: \\?\C:\h\w\BFA20AC3\t\DirectoryInfo_Create_ipbli234.zri\DirectoryLongerThanMaxLongPathWithExtendedSyntax_ThrowsException_271_a66fa5a7\802869a5d16b495fa5d074e146cb2cd7...

Xunit.Sdk.XunitException: Expected one of: (System.IO.PathTooLongException, System.IO.DirectoryNotFoundException) -> Actual: (System.IO.IOException)
              at System.AssertExtensions.ThrowsAnyInternal(Action action, Type[] exceptionTypes) in /_/src/libraries/Common/tests/TestUtilities/System/AssertExtensions.cs:line 212
              at System.AssertExtensions.ThrowsAny[TFirstExceptionType,TSecondExceptionType](Action action) in /_/src/libraries/Common/tests/TestUtilities/System/AssertExtensions.cs:line 222
              at System.IO.Tests.Directory_CreateDirectory.<DirectoryLongerThanMaxLongPathWithExtendedSyntax_ThrowsException>b__27_0(String path) in /_/src/libraries/System.IO.FileSystem/tests/Directory/CreateDirectory.cs:line 274
              at Xunit.Assert.All[T](IEnumerable`1 collection, Action`1 action) in /_/src/xunit.assert/Asserts/CollectionAsserts.cs:line 36

[ghost]

Tagging subscribers to this area: @dotnet/area-system-io
See info in area-owners.md if you want to be subscribed.

Issue Details

---

Build, and Log

    System.IO.Tests.DirectoryInfo_Create.DirectoryLongerThanMaxLongPathWithExtendedSyntax_ThrowsException [FAIL]
      Assert.All() Failure: 2 out of 2 items in the collection did not pass.
      [1]: Item: \\?\C:\h\w\BFA20AC3\t\DirectoryInfo_Create_ipbli234.zri\DirectoryLongerThanMaxLongPathWithExtendedSyntax_ThrowsException_271_a66fa5a7\802869a5d16b495fa5d074e146cb2cd7...

Xunit.Sdk.XunitException: Expected one of: (System.IO.PathTooLongException, System.IO.DirectoryNotFoundException) -> Actual: (System.IO.IOException)
              at System.AssertExtensions.ThrowsAnyInternal(Action action, Type[] exceptionTypes) in /_/src/libraries/Common/tests/TestUtilities/System/AssertExtensions.cs:line 212
              at System.AssertExtensions.ThrowsAny[TFirstExceptionType,TSecondExceptionType](Action action) in /_/src/libraries/Common/tests/TestUtilities/System/AssertExtensions.cs:line 222
              at System.IO.Tests.Directory_CreateDirectory.<DirectoryLongerThanMaxLongPathWithExtendedSyntax_ThrowsException>b__27_0(String path) in /_/src/libraries/System.IO.FileSystem/tests/Directory/CreateDirectory.cs:line 274
              at Xunit.Assert.All[T](IEnumerable`1 collection, Action`1 action) in /_/src/xunit.assert/Asserts/CollectionAsserts.cs:line 36

Author:	radical
Assignees:	-
Labels:	area-System.IO, untriaged
Milestone:	-

[danmoseley]

I just hit this too https://helixre8s23ayyeko0k025g8.blob.core.windows.net/dotnet-runtime-refs-pull-64008-merge-e31c2ef07eb04bed87/System.IO.FileSystem.Tests/1/console.8fa36a2b.log?sv=2019-07-07&se=2022-02-08T18%3A27%3A22Z&sr=c&sp=rl&sig=9NCp1RxSJuCn0PIOUSr31uYMCVsCgRTA2nngbCPnNxI%3D
#63824
The machine names are different though (first line). Perhaps some other IO test is leaving a mess? It's too bad we can' see the exception message.

[danmoseley]

@dotnet/dnceng did something change with the "windows.11.amd64.clientpre.open" queue at about 2022-01-19 17:36:32.7150000 ? This test never failed before then, then consistently failed after that.

[MattGal]

@{Matt's team} did something change with the "windows.11.amd64.clientpre.open" queue at about 2022-01-19 17:36:32.7150000 ? This test never failed before then, then consistently failed after that.

I'll take this one, since I'm working in this space right now (also feel free to hit me up on Teams to discuss further)

Yes and no :) The image used in that queue hasn't changed since December (I can tell this because image version is still 2021.1220.204422). However, this corresponds to when we allowed Azure Security Pack to start doing its thing (this is an S360 requirement) to prevent the angry "missing AzSecPack" issues from popping up.

If there's something you can identify (ideally using a DTL VM) what you consider wrong here, we can talk about how to work around it, but unfortunately opting out of AzSecPack is not an option unless we make Helix machines automatically get deleted every 4 hours.

[danmoseley]

Hi @MattGal it would be interesting to know what error message this code produces on the machine: https://gist.github.com/danmoseley/9ba23a2455a3f28e67bf57d7ca9b2838
and also whether long paths are enabled

[HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Control\FileSystem]
"LongPathsEnabled"=dword:00000001

https://docs.microsoft.com/en-us/windows/win32/fileio/maximum-file-path-limitation?tabs=cmd#enable-long-paths-in-windows-10-version-1607-and-later

[danmoseley]

BTW is "Azure Security Pack" deployed on Win 10 as well? As the failure is unique to Win 11..

[MattGal]

BTW is "Azure Security Pack" deployed on Win 10 as well? As the failure is unique to Win 11..

It's an auto-applied extension, it'll be anywhere it's capable of installing itself

[MattGal]

@danmoseley checking the reg value the value is set correctly.

Console log: 'hello-world-long-path-0' from job 965f5af2-43ad-46c2-a893-f6f63b7a24e0 workitem 79650ac5-754c-4bb2-b497-8ccf2ee06277 (windows.11.amd64.clientpre.open) executed on machine a005CGP

HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Control\FileSystem
    DisableDeleteNotification    REG_DWORD    0x0
    FilterSupportedFeaturesMode    REG_DWORD    0x0
    LongPathsEnabled    REG_DWORD    0x1
    NtfsAllowExtendedCharacter8dot3Rename    REG_DWORD    0x0

I'll try out the gist you sent me shortly.

[MattGal]

@danmoseley running the gist on 6.0 GA on the repro machine I set up, I see System.IO.PathTooLongException despite checking the LongPathsEnabled field again. Really weirdly, I can do md on the exact string you provided (I can't do dir on it because THAT gets "the input line is too long" though). I can delete the last 2 chars off the path, do "dir (that path)*" then tab complete it to the proper length and allt his works.

I wonder if some kind of Windows update thing brought this along, since as noted the image hasn't changed at all and I assume you've already verified this test has existed for some time.

[danmoseley]

PathTooLongException is a test pass, the test was getting IOException in the failing case. Maybe that reg key had the other value before?

[MattGal]

PathTooLongException is a test pass, the test was getting IOException in the failing case. Maybe that reg key had the other value before?

I don’t think that’s likely. The Windows Helix client would not start without the ps1 that sets that registry key running and the scripts haven’t changed in years. I do have a theory though. While we try hard to exclude the Helix work directory from AV scans (instead just not trusting or keeping the machines), the only thing that did change (Azure Security Pack) very well could be ignoring the system’s settings on exclusion paths. If so, there’s good and bad news.

The bad news is unless it’s world-breaking we’re probably not getting rid of the Geneva Monitoring agent on the machines; otherwise when VMs live > 4 hours we can get alerts and issues filed on us to install it.

The GOOD news is that if my theory is correct (windows-level layer of a file API got grumpy with you because an involuntary file system scan is being performed by the Geneva Monitoring agent delayed the call), a simple sleep of a second or so and retry of the operation should get you the right exception.

If not we can keep exploring, but I am very confident that the registry key hasn’t changed in over a year and that we run the same exact powershell on every Windows we use.

[danmoseley]

@MattGal thanks. Looking a bit further, that key should be irrelevant in the case where there's a \\?\ prefix like here.

Don't change the machine configuration on this account. I can just adjust the test.