git restore --source=HEAD :/ advice after failed git clone has no effect

[Marcono1234]

• I was not able to find an open or closed issue matching what I'm seeing

Originally asked this as StackOverflow question, but was told it might be a Git for Windows bug.

Related to the advice observed in #2438 (comment).

Setup

• Which version of Git for Windows are you using? Is it 32-bit or 64-bit?

  $ git --version --build-options
  
  git version 2.33.0.windows.2
  cpu: x86_64
  built from commit: 8735530946cced809cc6cc4c2ca3b078cdb3dfc8
  sizeof-long: 4
  sizeof-size_t: 8
  shell-path: /bin/sh
  feature: fsmonitor--daemon
  

• Which version of Windows are you running? Vista, 7, 8, 10? Is it 32-bit or 64-bit?

  $ cmd.exe /c ver
  
  Microsoft Windows [Version 10.0.19043.1165]
  

• What options did you set as part of the installation? Or did you choose the
  defaults?

  $ cat /etc/install-options.txt
  
  Editor Option: Notepad++
  Custom Editor Path:
  Default Branch Option: main
  Path Option: Cmd
  SSH Option: OpenSSH
  Tortoise Option: false
  CURL Option: OpenSSL
  CRLF Option: CRLFAlways
  Bash Terminal Option: MinTTY
  Git Pull Behavior Option: Merge
  Use Credential Manager: Core
  Performance Tweaks FSCache: Enabled
  Enable Symlinks: Disabled
  Enable Pseudo Console Support: Disabled
  Enable FSMonitor: Disabled
  

• Any other interesting things about your environment that might be related
  to the issue you're seeing?

  • Long Windows file paths have not been enabled explicitly.
  • git config core.longpaths is not set (neither for system nor globally).

Details

• Which terminal/shell are you running Git from? e.g Bash/CMD/PowerShell/other

  PowerShell, Git Bash

• What commands did you run to trigger this issue? If you can provide a
  Minimal, Complete, and Verifiable example
  this will help us understand the issue.

  1. git clone https://github.com/dependabot/dependabot-core --depth=1 file-name-test
     ℹ️ Cloning fails due to "Filename too long"; Git suggests to run git restore --source=HEAD :/
  2. cd file-name-test
  3. git config core.longpaths true
  4. Run command suggested by Git:
     git restore --source=HEAD :/
  5. git status
     ❌ Changes are reported; running git restore had no effect

• What did you expect to occur after running these commands?

  Either:

  • After running git restore --source=HEAD :/ no changes should be reported anymore
  • OR Git should not suggest running git restore --source=HEAD :/ if it would have no effect

• What actually happened instead?

  Git suggested to run git restore --source=HEAD :/; it had no effect.

  Note that checkout succeeds when running git restore --source=HEAD -SW :/ or git reset --hard.

• If the problem was occurring with a specific repository, can you provide the
  URL to that repository to help us with testing?

  any repository with long file paths

[dscho]

This might be due to this issue from our Release Notes' "Known Issues" section:

• If you specify command-line options starting with a slash, POSIX-to-Windows path conversion will kick in converting e.g. "/usr/bin/bash.exe" to "C:\Program Files\Git\usr\bin\bash.exe". When that is not desired -- e.g. "--upload-pack=/opt/git/bin/git-upload-pack" or "-L/regex/" -- you need to set the environment variable MSYS_NO_PATHCONV temporarily, like so:

MSYS_NO_PATHCONV=1 git blame -L/pathconv/ msys2_path_conv.cc

Alternatively, you can double the first slash to avoid POSIX-to-Windows path conversion, e.g. "//usr/bin/bash.exe".

[Marcono1234]

Do you mean it might be causing issues for :/?
It does work for git restore --source=HEAD -SW :/ though, so I am not sure if path conversion is the issue.

Maybe this is actually an upstream Git issue and the advice to use git restore --source=HEAD :/ is incorrect, but I have not found a way to verify this. For Linux and macOS you can (with an artificially crafted repository) also hit the file name length limit. However, there you cannot do anything to work around this (unlike Git for Windows' core.longpaths). So if you try to run git restore it would fail again due to the file name length limit. But at least the suggested git restore command seems to do something on those platforms (have tested it for macOS, see this GitHub workflow run) so this suggests it might be a Git for Windows specific bug.

[dscho]

I just verified that this reproduces here:

$ git clone https://github.com/dependabot/dependabot-core --depth=1 file-name-test
Cloning into 'file-name-test'...
remote: Enumerating objects: 4233, done.
remote: Counting objects: 100% (4233/4233), done.
remote: Compressing objects: 100% (3123/3123), done.
remote: Total 4233 (delta 1278), reused 2895 (delta 769), pack-reused 0
Receiving objects: 100% (4233/4233), 11.24 MiB | 3.41 MiB/s, done.
Resolving deltas: 100% (1278/1278), done.
error: unable to create file common/spec/fixtures/vcr_cassettes/Dependabot_MetadataFinders_Base_ReleaseFinder/_releases_text/with_a_github_repo/with_releases/when_the_release_is_present/updating_from_no_previous_release_to_new_release/1_2_1_1_1_4_1.yml: Filename too long
warning: unable to access 'common/spec/fixtures/vcr_cassettes/Dependabot_MetadataFinders_Base_ReleaseFinder/_releases_text/with_a_github_repo/with_releases/when_the_release_is_present/updating_from_uknown_previous_release_to_new_release/.gitattributes': Filename too long
error: unable to create file common/spec/fixtures/vcr_cassettes/Dependabot_MetadataFinders_Base_ReleaseFinder/_releases_text/with_a_github_repo/with_releases/when_the_release_is_present/updating_from_uknown_previous_release_to_new_release/1_2_1_1_1_4_1.yml: Filename too long
Updating files: 100% (3971/3971), done.
fatal: unable to checkout working tree
warning: Clone succeeded, but checkout failed.
You can inspect what was checked out with 'git status'
and retry with 'git restore --source=HEAD :/'

And indeed, both with and without MSYS_NO_PATHCONV defined, the suggested git restore --source=HEAD :/ fails to check out any files, which is quite disappointing.

The really interesting thing here is that this also happens if I run that command via WSL.

So let's dig deeper. The biggest problem is of course that all of the files are marked as deleted in the output of git status. And therein lies the problem: When git checkout produces that error message, it assumes that Git's index (or "staging area") was updated properly, but writing the files to the worktree failed. However, this is not the case here, for some reason even the index was not updated correctly. And according to the documentation of git restore:

Restore specified paths in the working tree with some contents from a restore source. If a path is tracked but does not exist in the restore source, it will be removed to match the source.

The command can also be used to restore the content in the index with --staged, or restore both the working tree and the index with --staged --worktree.

Note how the first sentence says: in the working tree. And the second paragraph says that the index can optionally also be restored. In other words, this command works as intended:

$ git restore --source=HEAD -S -W :/

The problem, then, is that the index was not written correctly. And I suspect that this problem lies within the original Git project,
i.e. that it might make sense to take this issue to the Git mailing list (send plain-text messages, HTML messages are dropped silently), noting that the git clone failed to check out the files properly and crucially failed to update the index correctly.

[landstander668]

For what it's worth, the clone command does appear to work as expected when run on Linux (Ubuntu 21.04 in this case).

[2021-09-07 08:12:17] adric@vm1 ~
$ git version
git version 2.33.0

[2021-09-07 08:12:32] adric@vm1 ~
$ git clone https://github.com/dependabot/dependabot-core --depth=1 file-name-test
Cloning into 'file-name-test'...
remote: Enumerating objects: 4233, done.
remote: Counting objects: 100% (4233/4233), done.
remote: Compressing objects: 100% (3123/3123), done.
remote: Total 4233 (delta 1278), reused 2893 (delta 769), pack-reused 0
Receiving objects: 100% (4233/4233), 11.24 MiB | 20.16 MiB/s, done.
Resolving deltas: 100% (1278/1278), done.

[dscho]

@landstander668 right. But if the clone on Linux would fail to write some file for whatever reason, then the index might still be left empty by mistake. That's the bug I am talking about.

[Marcono1234]

Sorry for the delay, I have reported it on the Git mailing list now.
(I hope it includes all necessary information.)

[dscho]

Let's continue the investigation there.