cmd/go: git fetch errors dropped when producing pseudo-versions for commits #53935

bcmills · 2022-07-18T14:55:00Z

--- FAIL: TestScript (0.01s)
    --- FAIL: TestScript/mod_get_direct (158.26s)
        script_test.go:270: 
            # (2022-07-14T21:40:47Z)
            # Regression test for golang.org/issue/34092: with an empty module cache,
            # 'GOPROXY=direct go get golang.org/x/tools/gopls@master' did not correctly
            # resolve the pseudo-version for its dependency on golang.org/x/tools. (158.242s)
            > [short] skip
            > [!net] skip
            > [!exec:git] skip
            > env GO111MODULE=on
            > env GOPROXY=direct
            > env GOSUMDB=off
            > go list -m cloud.google.com/go@main
            [stdout]
            cloud.google.com/go v0.0.0-20220714214024-fc3417be70cd
            > ! stdout 'v0.0.0-'
            FAIL: testdata/script/mod_get_direct.txt:14: unexpected match for `(?m)v0.0.0-` found in stdout: v0.0.0-
            
FAIL
FAIL	cmd/go	262.233s

greplogs -l -e 'FAIL: TestScript/mod_get_direct .*(?:\n .*)*found in stdout: v0.0.0-'
2022-07-14T21:16:23-dc00aed/linux-amd64-longtest
2022-05-19T15:38:08-e23cc08/linux-386-longtest
2022-05-18T15:25:04-1f9f7db/linux-386-longtest

The text was updated successfully, but these errors were encountered:

bcmills · 2022-07-18T14:56:57Z

This is probably a symptom of a missing error-check somewhere, combined with an intermittent flake connecting to cloud.google.com or its VCS host (compare #51051).

Independent of the flaky cloud.google.com dependency, we should find and fix the missing error check.

bcmills · 2022-08-22T15:40:48Z

A mod_invalid_version failure with a similar pathology:
2022-08-18T19:12:13-0eb56ca/linux-386-longtest

            > go mod edit -require github.com/pierrec/lz4@473cd7ce01a1
            > go list -m github.com/pierrec/lz4
            [stdout]
            github.com/pierrec/lz4 v0.0.0-20190131084431-473cd7ce01a1
            > stdout 'github.com/pierrec/lz4 v1.0.2-0.20190131084431-473cd7ce01a1'
            FAIL: testdata/script/mod_invalid_version.txt:186: no match for `(?m)github.com/pierrec/lz4 v1.0.2-0.20190131084431-473cd7ce01a1` found in stdout

gopherbot · 2022-08-27T02:43:41Z

Change https://go.dev/cl/426079 mentions this issue: cmd/go/internal/modfetch: report error on failing to derive pseudo version from recent tag

ZekeLu · 2022-08-27T03:10:32Z

I have managed to reproduce the issue with this script:

#!/bin/bash
for i in {1..10000}; do
    echo "$(date +%T): try $i times"
    echo "$(date +%T): try $i times ======" >> log.txt
    go clean -modcache && GOPROXY=direct GOSUMDB=off gotip list -x -m github.com/pierrec/lz4@473cd7ce01a1 >>log.txt 2>&1

    if tail -n 2 log.txt | grep -F v0.0.0; then
        echo "$(date +%T): reproduced"
        exit 0
    fi
done

The script is executed in the directory with the following files:

-- go.mod --
module example.com

go 1.14
-- go.sum --

Notes:

I can not access cloud.google.com so I tested against github.com/pierrec/lz4@473cd7ce01a1 instead.
the network between my computer and github.com is bad (which makes it easy to reproduce the issue 😄 )

diff between a succeeded log and a failed log

Note: The duration before # cd is modified manually to reduce the noice.

--- succeeded.log	2022-08-27 10:48:19.000000000 +0800
+++ failed.log	2022-08-27 02:15:11.406606020 +0800
@@ -1,46 +1,40 @@
 mkdir -p /home/zeke/go/pkg/mod/cache/vcs # git3 https://github.com/pierrec/lz4
 # lock /home/zeke/go/pkg/mod/cache/vcs/036595127a116abca3f2bfd7bedafedb33d890c2b102d96361815966679432e7.lock
 mkdir -p /home/zeke/go/pkg/mod/cache/vcs/036595127a116abca3f2bfd7bedafedb33d890c2b102d96361815966679432e7 # git3 https://github.com/pierrec/lz4
 cd /home/zeke/go/pkg/mod/cache/vcs/036595127a116abca3f2bfd7bedafedb33d890c2b102d96361815966679432e7; git init --bare
 0.001s # cd /home/zeke/go/pkg/mod/cache/vcs/036595127a116abca3f2bfd7bedafedb33d890c2b102d96361815966679432e7; git init --bare
 cd /home/zeke/go/pkg/mod/cache/vcs/036595127a116abca3f2bfd7bedafedb33d890c2b102d96361815966679432e7; git remote add origin -- https://github.com/pierrec/lz4
 0.001s # cd /home/zeke/go/pkg/mod/cache/vcs/036595127a116abca3f2bfd7bedafedb33d890c2b102d96361815966679432e7; git remote add origin -- https://github.com/pierrec/lz4
 cd /home/zeke/go/pkg/mod/cache/vcs/036595127a116abca3f2bfd7bedafedb33d890c2b102d96361815966679432e7; git -c log.showsignature=false log --no-decorate -n1 '--format=format:%H %ct %D' 473cd7ce01a1 --
 0.001s # cd /home/zeke/go/pkg/mod/cache/vcs/036595127a116abca3f2bfd7bedafedb33d890c2b102d96361815966679432e7; git -c log.showsignature=false log --no-decorate -n1 '--format=format:%H %ct %D' 473cd7ce01a1 --
 cd /home/zeke/go/pkg/mod/cache/vcs/036595127a116abca3f2bfd7bedafedb33d890c2b102d96361815966679432e7; git ls-remote -q origin
 34.014s # cd /home/zeke/go/pkg/mod/cache/vcs/036595127a116abca3f2bfd7bedafedb33d890c2b102d96361815966679432e7; git ls-remote -q origin
 cd /home/zeke/go/pkg/mod/cache/vcs/036595127a116abca3f2bfd7bedafedb33d890c2b102d96361815966679432e7; git tag -l
 0.003s # cd /home/zeke/go/pkg/mod/cache/vcs/036595127a116abca3f2bfd7bedafedb33d890c2b102d96361815966679432e7; git tag -l
 cd /home/zeke/go/pkg/mod/cache/vcs/036595127a116abca3f2bfd7bedafedb33d890c2b102d96361815966679432e7; git fetch -f --depth=1 origin refs/tags/v2.0.8:refs/tags/v2.0.8
 129.970s # cd /home/zeke/go/pkg/mod/cache/vcs/036595127a116abca3f2bfd7bedafedb33d890c2b102d96361815966679432e7; git fetch -f --depth=1 origin refs/tags/v2.0.8:refs/tags/v2.0.8
 cd /home/zeke/go/pkg/mod/cache/vcs/036595127a116abca3f2bfd7bedafedb33d890c2b102d96361815966679432e7; git -c log.showsignature=false log --no-decorate -n1 '--format=format:%H %ct %D' refs/tags/v2.0.8 --
 0.003s # cd /home/zeke/go/pkg/mod/cache/vcs/036595127a116abca3f2bfd7bedafedb33d890c2b102d96361815966679432e7; git -c log.showsignature=false log --no-decorate -n1 '--format=format:%H %ct %D' refs/tags/v2.0.8 --
 cd /home/zeke/go/pkg/mod/cache/vcs/036595127a116abca3f2bfd7bedafedb33d890c2b102d96361815966679432e7; git -c log.showsignature=false log --no-decorate -n1 '--format=format:%H %ct %D' 08c27939df1bd95e881e2c2367a749964ad1fceb --
 0.002s # cd /home/zeke/go/pkg/mod/cache/vcs/036595127a116abca3f2bfd7bedafedb33d890c2b102d96361815966679432e7; git -c log.showsignature=false log --no-decorate -n1 '--format=format:%H %ct %D' 08c27939df1bd95e881e2c2367a749964ad1fceb --
 cd /home/zeke/go/pkg/mod/cache/vcs/036595127a116abca3f2bfd7bedafedb33d890c2b102d96361815966679432e7; git fetch -f --depth=1 origin refs/tags/v1.0.1:refs/tags/v1.0.1
 120.220s # cd /home/zeke/go/pkg/mod/cache/vcs/036595127a116abca3f2bfd7bedafedb33d890c2b102d96361815966679432e7; git fetch -f --depth=1 origin refs/tags/v1.0.1:refs/tags/v1.0.1
 cd /home/zeke/go/pkg/mod/cache/vcs/036595127a116abca3f2bfd7bedafedb33d890c2b102d96361815966679432e7; git fetch -f origin 'refs/heads/*:refs/heads/*' 'refs/tags/*:refs/tags/*'
 239.611s # cd /home/zeke/go/pkg/mod/cache/vcs/036595127a116abca3f2bfd7bedafedb33d890c2b102d96361815966679432e7; git fetch -f origin 'refs/heads/*:refs/heads/*' 'refs/tags/*:refs/tags/*'
 cd /home/zeke/go/pkg/mod/cache/vcs/036595127a116abca3f2bfd7bedafedb33d890c2b102d96361815966679432e7; git fetch --unshallow -f origin
 121.158s # cd /home/zeke/go/pkg/mod/cache/vcs/036595127a116abca3f2bfd7bedafedb33d890c2b102d96361815966679432e7; git fetch --unshallow -f origin
 cd /home/zeke/go/pkg/mod/cache/vcs/036595127a116abca3f2bfd7bedafedb33d890c2b102d96361815966679432e7; git -c log.showsignature=false log --no-decorate -n1 '--format=format:%H %ct %D' 473cd7ce01a1113208073166464b98819526150e --
 0.005s # cd /home/zeke/go/pkg/mod/cache/vcs/036595127a116abca3f2bfd7bedafedb33d890c2b102d96361815966679432e7; git -c log.showsignature=false log --no-decorate -n1 '--format=format:%H %ct %D' 473cd7ce01a1113208073166464b98819526150e --
 cd /home/zeke/go/pkg/mod/cache/vcs/036595127a116abca3f2bfd7bedafedb33d890c2b102d96361815966679432e7; git cat-file blob 473cd7ce01a1113208073166464b98819526150e:go.mod
 0.003s # cd /home/zeke/go/pkg/mod/cache/vcs/036595127a116abca3f2bfd7bedafedb33d890c2b102d96361815966679432e7; git cat-file blob 473cd7ce01a1113208073166464b98819526150e:go.mod
 cd /home/zeke/go/pkg/mod/cache/vcs/036595127a116abca3f2bfd7bedafedb33d890c2b102d96361815966679432e7; git for-each-ref --format %(refname) refs/tags --merged 473cd7ce01a1113208073166464b98819526150e
 0.007s # cd /home/zeke/go/pkg/mod/cache/vcs/036595127a116abca3f2bfd7bedafedb33d890c2b102d96361815966679432e7; git for-each-ref --format %(refname) refs/tags --merged 473cd7ce01a1113208073166464b98819526150e
 cd /home/zeke/go/pkg/mod/cache/vcs/036595127a116abca3f2bfd7bedafedb33d890c2b102d96361815966679432e7; git fetch -f origin 'refs/heads/*:refs/heads/*' 'refs/tags/*:refs/tags/*'
-0.922s # cd /home/zeke/go/pkg/mod/cache/vcs/036595127a116abca3f2bfd7bedafedb33d890c2b102d96361815966679432e7; git fetch -f origin 'refs/heads/*:refs/heads/*' 'refs/tags/*:refs/tags/*'
+151.841s # cd /home/zeke/go/pkg/mod/cache/vcs/036595127a116abca3f2bfd7bedafedb33d890c2b102d96361815966679432e7; git fetch -f origin 'refs/heads/*:refs/heads/*' 'refs/tags/*:refs/tags/*'
-cd /home/zeke/go/pkg/mod/cache/vcs/036595127a116abca3f2bfd7bedafedb33d890c2b102d96361815966679432e7; git fetch --unshallow -f origin
-4.288s # cd /home/zeke/go/pkg/mod/cache/vcs/036595127a116abca3f2bfd7bedafedb33d890c2b102d96361815966679432e7; git fetch --unshallow -f origin
-cd /home/zeke/go/pkg/mod/cache/vcs/036595127a116abca3f2bfd7bedafedb33d890c2b102d96361815966679432e7; git for-each-ref --format %(refname) refs/tags --merged 473cd7ce01a1113208073166464b98819526150e
-0.007s # cd /home/zeke/go/pkg/mod/cache/vcs/036595127a116abca3f2bfd7bedafedb33d890c2b102d96361815966679432e7; git for-each-ref --format %(refname) refs/tags --merged 473cd7ce01a1113208073166464b98819526150e
 cd /home/zeke/go/pkg/mod/cache/vcs/036595127a116abca3f2bfd7bedafedb33d890c2b102d96361815966679432e7; git cat-file blob 473cd7ce01a1113208073166464b98819526150e:go.mod
 0.003s # cd /home/zeke/go/pkg/mod/cache/vcs/036595127a116abca3f2bfd7bedafedb33d890c2b102d96361815966679432e7; git cat-file blob 473cd7ce01a1113208073166464b98819526150e:go.mod
-cd /home/zeke/go/pkg/mod/cache/vcs/036595127a116abca3f2bfd7bedafedb33d890c2b102d96361815966679432e7; git merge-base --is-ancestor -- v1.0.1 473cd7ce01a1113208073166464b98819526150e
-0.003s # cd /home/zeke/go/pkg/mod/cache/vcs/036595127a116abca3f2bfd7bedafedb33d890c2b102d96361815966679432e7; git merge-base --is-ancestor -- v1.0.1 473cd7ce01a1113208073166464b98819526150e
 cd /home/zeke/go/pkg/mod/cache/vcs/036595127a116abca3f2bfd7bedafedb33d890c2b102d96361815966679432e7; git cat-file blob 473cd7ce01a1113208073166464b98819526150e:go.mod
 0.002s # cd /home/zeke/go/pkg/mod/cache/vcs/036595127a116abca3f2bfd7bedafedb33d890c2b102d96361815966679432e7; git cat-file blob 473cd7ce01a1113208073166464b98819526150e:go.mod
 cd /home/zeke/go/pkg/mod/cache/vcs/036595127a116abca3f2bfd7bedafedb33d890c2b102d96361815966679432e7; git cat-file blob 473cd7ce01a1113208073166464b98819526150e:go.mod
 0.001s # cd /home/zeke/go/pkg/mod/cache/vcs/036595127a116abca3f2bfd7bedafedb33d890c2b102d96361815966679432e7; git cat-file blob 473cd7ce01a1113208073166464b98819526150e:go.mod
-github.com/pierrec/lz4 v1.0.2-0.20190131084431-473cd7ce01a1
+github.com/pierrec/lz4 v0.0.0-20190131084431-473cd7ce01a1

From the log I found that git fetch --unshallow -f origin is not executed when it failed. Which could happen on line 558 below:

go/src/cmd/go/internal/modfetch/codehost/git.go

Lines 549 to 570 in a094849

    
           func (r *gitRepo) fetchRefsLocked() error { 
        
           	if r.fetchLevel < fetchAll { 
        
           		// NOTE: To work around a bug affecting Git clients up to at least 2.23.0 
        
           		// (2019-08-16), we must first expand the set of local refs, and only then 
        
           		// unshallow the repository as a separate fetch operation. (See 
        
           		// golang.org/issue/34266 and 
        
           		// https://github.com/git/git/blob/4c86140027f4a0d2caaa3ab4bd8bfc5ce3c11c8a/transport.c#L1303-L1309.) 
        
           		if _, err := Run(r.dir, "git", "fetch", "-f", r.remote, "refs/heads/*:refs/heads/*", "refs/tags/*:refs/tags/*"); err != nil { 
        
           			return err 
        
           		} 
        
           		if _, err := os.Stat(filepath.Join(r.dir, "shallow")); err == nil { 
        
           			if _, err := Run(r.dir, "git", "fetch", "--unshallow", "-f", r.remote); err != nil { 
        
           				return err 
        
           			} 
        
           		} 
        
           		r.fetchLevel = fetchAll 
        
           	} 
        
           	return nil 
        
           }

And finally I found that the error is ignored on line 610 below:

go/src/cmd/go/internal/modfetch/coderepo.go

Lines 609 to 614 in a094849

    
           if pseudoBase == "" { 
        
           	tag, _ := r.code.RecentTag(info.Name, tagPrefix, tagAllowed) 
        
           	if tag != "" { 
        
           		pseudoBase, _ = tagToVersion(tag) 
        
           	} 
        
           }

I can reproduce the issue by modifying src/cmd/go/internal/modfetch/codehost/git.go like this:

diff --git a/src/cmd/go/internal/modfetch/codehost/git.go b/src/cmd/go/internal/modfetch/codehost/git.go
index ac2dc2348e..921f7d32cf 100644
--- a/src/cmd/go/internal/modfetch/codehost/git.go
+++ b/src/cmd/go/internal/modfetch/codehost/git.go
@@ -554,9 +554,7 @@ func (r *gitRepo) fetchRefsLocked() error {
        // golang.org/issue/34266 and
        // https://github.com/git/git/blob/4c86140027f4a0d2caaa3ab4bd8bfc5ce3c11c8a/transport.c#L1303-L1309.)
 
-       if _, err := Run(r.dir, "git", "fetch", "-f", r.remote, "refs/heads/*:refs/heads/*", "refs/tags/*:refs/tags/*"); err != nil {
-           return err
-       }
+       return fmt.Errorf("Hardcoded error")
 
        if _, err := os.Stat(filepath.Join(r.dir, "shallow")); err == nil {
            if _, err := Run(r.dir, "git", "fetch", "--unshallow", "-f", r.remote); err != nil {

https://go.dev/cl/426079 has been sent to fix this issue.

bcmills · 2022-08-29T14:25:57Z

@gopherbot, please backport to Go 1.18 and 1.19. This can cause the go command to resolve unexpected pseudo-versions for commits and branches, and the fix is very small.

gopherbot · 2022-08-29T14:26:24Z

Backport issue(s) opened: #54733 (for 1.18), #54734 (for 1.19).

Remember to create the cherry-pick CL(s) as soon as the patch is submitted to master, according to https://go.dev/wiki/MinorReleases.

gopherbot · 2022-08-29T19:44:14Z

Change https://go.dev/cl/426494 mentions this issue: [release-branch.go1.19] cmd/go/internal/modfetch: report error on failing to derive pseudo version from recent tag

gopherbot · 2022-08-29T19:44:48Z

Change https://go.dev/cl/426514 mentions this issue: [release-branch.go1.18] cmd/go/internal/modfetch: report error on failing to derive pseudo version from recent tag

gopherbot · 2022-08-29T20:33:03Z

Change https://go.dev/cl/426495 mentions this issue: cmd/go/internal/modfetch: distinguish "unsupported" errors from RecentTag

…tTag CL 426079 started checking errors from RecentTag. Unfortunately, we forgot to run "-longtest" SlowBots, and it turns out to have broken non-short tests for non-git VCS implementations, because those don't implement the RecentTag method. Updates #53935. Change-Id: I5935f2f4b3f684515e99e8bf70a840154c36249f Reviewed-on: https://go-review.googlesource.com/c/go/+/426495 Run-TryBot: Bryan Mills <[email protected]> Reviewed-by: David Chase <[email protected]> Reviewed-by: Dmitri Shuralyov <[email protected]> Auto-Submit: Bryan Mills <[email protected]> TryBot-Result: Gopher Robot <[email protected]>

bcmills added NeedsInvestigation Someone must examine and confirm this is a valid issue and not a duplicate of an existing one. modules labels Jul 18, 2022

bcmills added this to the Backlog milestone Jul 18, 2022

bcmills mentioned this issue Jul 18, 2022

build dashboard triage log #52653

Closed

bcmills changed the title ~~cmd/go: TestScript/mod_get_direct failures with v0.0.0- pseudo-versions~~ cmd/go: git test failures with v0.0.0- pseudo-versions Aug 22, 2022

ZekeLu mentioned this issue Aug 27, 2022

cmd/go/internal/modfetch: report error on failing to derive pseudo version from recent tag #54701

Closed

bcmills modified the milestones: Backlog, Go1.20 Aug 29, 2022

bcmills assigned ZekeLu Aug 29, 2022

This was referenced Aug 29, 2022

cmd/go: git fetch errors dropped when producing pseudo-versions for commits [1.18 backport] #54733

Closed

cmd/go: git fetch errors dropped when producing pseudo-versions for commits [1.19 backport] #54734

Closed

bcmills changed the title ~~cmd/go: git test failures with v0.0.0- pseudo-versions~~ cmd/go: git fetch errors dropped when producing pseudo-versions for commits Aug 29, 2022

gopherbot closed this as completed in 5cfeaed Aug 29, 2022

dmitshur added NeedsFix The path to resolution is known, but the work has not been done. and removed NeedsInvestigation Someone must examine and confirm this is a valid issue and not a duplicate of an existing one. labels Aug 29, 2022

bcmills mentioned this issue Sep 6, 2022

cmd/go: TestScript/mod_invalid_version failures on linux-386-longtest due to missing ancestor tags #52796

Closed

golang locked and limited conversation to collaborators Aug 29, 2023

gopherbot added the FrozenDueToAge label Aug 29, 2023

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

cmd/go: git fetch errors dropped when producing pseudo-versions for commits #53935

cmd/go: git fetch errors dropped when producing pseudo-versions for commits #53935

bcmills commented Jul 18, 2022

bcmills commented Jul 18, 2022

bcmills commented Aug 22, 2022

gopherbot commented Aug 27, 2022

ZekeLu commented Aug 27, 2022 •

edited

Loading

bcmills commented Aug 29, 2022

gopherbot commented Aug 29, 2022

gopherbot commented Aug 29, 2022

gopherbot commented Aug 29, 2022

gopherbot commented Aug 29, 2022

cmd/go: git fetch errors dropped when producing pseudo-versions for commits #53935

cmd/go: git fetch errors dropped when producing pseudo-versions for commits #53935

Comments

bcmills commented Jul 18, 2022

bcmills commented Jul 18, 2022

bcmills commented Aug 22, 2022

gopherbot commented Aug 27, 2022

ZekeLu commented Aug 27, 2022 • edited Loading

bcmills commented Aug 29, 2022

gopherbot commented Aug 29, 2022

gopherbot commented Aug 29, 2022

gopherbot commented Aug 29, 2022

gopherbot commented Aug 29, 2022

ZekeLu commented Aug 27, 2022 •

edited

Loading