b3sum does not support long path names on Windows 10

[tERyceNzAchE]

Windows 10 1607 (and later) support path names longer than 260 characters. b3sum should support long path names as well. currently it will throw and error (os error 3).

[oconnor663]

I don't think we do anything particularly interesting with paths, other than get them from clap and pass them to standard APIs like std::fs::File::open(). The most interesting thing we do is probably Mmap, so it would be interesting to test this with the --no-mmap flag. I wasn't able to create any long filenames myself when I tried just now (both in the file explorer, and in Python). Is there some special registry key I need to set?

[tERyceNzAchE]

Same issue when using --no-mmap

[tERyceNzAchE]

Is there some special registry key I need to set?

Enable Long Paths in Windows 10, Version 1607, and Later

The registry key Computer\HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Control\FileSystem\LongPathsEnabled (Type: REG_DWORD) must exist and be set to 1.

[cesarb]

As explained in the above link, the registry key is necessary but not sufficient, you also need an application manifest embedded within the executable. AFAIK, neither rustc nor cargo currently embed an application manifest; see rust-lang/rfcs#721 and rust-lang/cargo#7986 (and as a comment on the later points out, all the operating-system-specific code in the rust standard library and in any included C or Rust library must be audited first to make sure they won't break with paths longer than 260 characters).

There seems to be a workaround to embed an application manifest: see https://crates.io/crates/embed_resource and as an example of it being used watchexec/watchexec#163 (watchexec/watchexec@570ed3a and watchexec/watchexec@838103f).

And there's also another workaround, which is to use an external (non-embedded) application manifest: if there's a file called b3sum.exe.manifest in the same directory as b3sum.exe, it will be used as the application manifest (I've done so in a distant past to sort of add an application manifest to an executable generated by a legacy compiler; I don't know whether it still works in recent Windows versions, as I haven't developed anything for Windows in a long time and don't have any machine with Windows to test). The file from watchexec/watchexec@570ed3a should work fine (as long as windows accepts longPathAware from a non-embedded application manifest).

[tERyceNzAchE]

Internally, could you access files using UNC-style long paths to get around this limit?
That seems to be the approach corz checksums uses. On Windows it claims to support paths up to 32,767 characters in length.

[yjlin0224]

Tested on Windows 10 20H2 with b3sum_windows_x64_bin.exe 0.3.7
I already enable long path feature by editing registry key.
Without external manifest file, I succeeded to run b3sum with long path with adding \\?\ with full path. It does works when the path is relative.

.\b3sum_windows_x64_bin.exe \\?\E:\Downloads\00000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000\AAA0000000000.txt
16b3a442c222b958453e73cb818a51a060bed10b9bf6649f2bbb43a9e57bff78  //?/E:/Downloads/00000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000/AAA0000000000.txt

[comanche]

These are the steps I followed to get b3sum.exe to support long file paths in Windows 10 Pro 64-bit 21H1:

Enable OS support for Windows 10 with the registry update:

Windows Registry Editor Version 5.00

[HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Control\FileSystem]
"LongPathsEnabled"=dword:00000001

Download load Windows SDK ISO: https://developer.microsoft.com/en-us/windows/downloads/windows-10-sdk/

Double click the ISO to mount it as a drive letter.

Extract the Windows SDK for Windows Store Apps Tools-x86_en-us.msi file. If your mounted drive letter is F:, and you want to extract it to drive D:, you would run:

msiexec /a "F:\Installers\Windows SDK for Windows Store Apps Tools-x86_en-us.msi" TARGETDIR="Full_Extract_Folder_Path" /qb

You'll find mt.exe inside Full_Extract_Folder_Path\Windows Kits\10\bin\10.0.19041.0\x64 where 10.0.19041.0 is the SDK version.

Verify that b3sum.exe doesn't have an existing manifest.

path-to\mt.exe -inputresource:b3sum.exe -out:exported.manifest
\Windows Kits\10\bin\10.0.19041.0\x64\mt.exe

You should see

mt.exe : general error c101008c: Failed to read the manifest from the resource of file "b3sum.exe". The specified image file did not contain a resource section.

In the same path where you have b3sum.exe, create an XML file named b3sum.exe.manifest with the following content:

<?xml version="1.0" encoding="UTF-8" standalone="yes"?>
<assembly xmlns="urn:schemas-microsoft-com:asm.v1" manifestVersion="1.0">
  <application xmlns="urn:schemas-microsoft-com:asm.v3">
    <windowsSettings>
      <longPathAware xmlns="http://schemas.microsoft.com/SMI/2016/WindowsSettings">true</longPathAware>
    </windowsSettings>
  </application>
</assembly>

Embed the manifest inside b3sum.exe.

path-to\mt.exe -manifest b3sum.exe.manifest -outputresource:b3sum.exe;1

That's it. Test b3sum.exe with a long file path.