git url: `?checksum=<SHORT>` fails (while `?tag=<TAG>&checksum=<SHORT>` and `?checksum=<LONG>` work)

[AkihiroSuda]

Contributing guidelines and issue reporting guide

• I've read the contributing guidelines and wholeheartedly agree. I've also read the issue reporting guide.

Well-formed report checklist

• I have found a bug that the documentation does not mention anything about my problem
• I have found a bug that there are no open or closed issues that are related to my problem
• I have provided version/information about my environment and done my best to provide a reproducer

Description of bug

Bug description

In the git url queries (#6172), ?checksum=<SHORT> fails, while ?tag=<TAG>&checksum=<SHORT> and ?checksum=<LONG> work

Reproduction

FROM scratch
ADD https://github.com/moby/buildkit.git?checksum=66735c67 /buildkit

# The following variants do not fail:
# ADD https://github.com/moby/buildkit.git?tag=v0.21.1&checksum=66735c67 /buildkit
# ADD https://github.com/moby/buildkit.git?checksum=66735c67040bc80e6ed104f451683e094030a4e1 /buildkit

$ sudo buildctl build --frontend dockerfile.v0 --local dockerfile=. --local context=.
[+] Building 0.9s (3/4)                                                                                                                                                      
 => [internal] load build definition from Dockerfile                                                                                                                    0.0s
 => => transferring dockerfile: 1.00kB                                                                                                                                  0.0s
 => [internal] load .dockerignore                                                                                                                                       0.0s
 => => transferring context: 2B                                                                                                                                         0.0s
 => ERROR [1/1] ADD https://github.com/moby/buildkit.git?checksum=66735c67 /buildkit                                                                                    0.8s
------                                                                                                                                                                       
 > [1/1] ADD https://github.com/moby/buildkit.git?checksum=66735c67 /buildkit:
0.004 Initialized empty Git repository in /var/lib/buildkit/runc-overlayfs/snapshots/snapshots/65/fs/
------
error: failed to solve: failed to load cache key: repository does not contain ref 66735c67, output: ""

Version information

99a9103129006338eb86a8b2ca553d1bb5c11037

[tonistiigi]

What is the expected behavior in here? We can't fetch by short commit.

Currently, when there is a checksum/commit but no ref, then we set ref to it.

Option 1. We could make checksum=short without ref invalid.

Option 2. We could make it so that ref is not initialized from checksum. Instead, the default branch is pulled, and the build will fail if it doesn't verify against it. Note that atm even if the commit matches the default branch, this would not be exactly equivalent, as keep-git-dir is different depending on whether fetch is by commit or by ref.

[crazy-max]

Yeah short SHAs can be unsafe since they may collide as the repo grows. For reproducible builds, requiring a full SHA is safer and avoids nondeterminism. That said, using a short sha is fine when a ref (e.g. tag or branch) is provided, because the ref already determines the object set where the short sha is resolved. I'm in for option 1.