Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Problem with unstable go-sqlite3 version - 2.0.3 #975

Closed
Shriram-RP opened this issue Oct 22, 2021 · 40 comments
Closed

Problem with unstable go-sqlite3 version - 2.0.3 #975

Shriram-RP opened this issue Oct 22, 2021 · 40 comments

Comments

@Shriram-RP
Copy link

Shriram-RP commented Oct 22, 2021

Hi,

We at the platform team of razorpay internally use bulk-insert which internally uses github.com/jinzhu/gorm v1.9.12 and creates an indirect dependency on github.com/mattn/go-sqlite3 v2.0.3.

Reference :- https://github.com/t-tiger/gorm-bulk-insert/blob/master/go.mod#L14

Now, this setup was working fine for us till last week and suddenly we had a failure in our docker build for github actions citing
github.com/mattn/[email protected]+incompatible: unknown revision v2.0.3

We also saw one of your issues where you mentioned versions 2.0+ were unstable - issue

Did you stop supporting 2.0.3 version suddenly in the last 7-10 days. We were wondering what caused the breakage of the build from our side. It would be helpful if you could throw some light on this issue.

Thanks,
Shriram

@rittneje
Copy link
Collaborator

What version of Go are you using? Also, what is the output of go env?

@ghzmhrm
Copy link

ghzmhrm commented Oct 24, 2021

I have the same problem with @Shriram-RP, but everything is going well after changing the GOPROXY env from direct to proxy.golang.org

Before changing the GOPROXY to proxy.golang.org

foo@bar golang-dir % go mod tidy
go: github.com/mattn/[email protected]+incompatible: reading github.com/mattn/go-sqlite3/go.mod at revision v2.0.1: unknown revision v2.0.1
go: downloading github.com/mattn/go-sqlite3 v2.0.1+incompatible
go: github.com/mattn/[email protected]+incompatible: reading github.com/mattn/go-sqlite3/go.mod at revision v2.0.1: unknown revision v2.0.1

Changing the GOPROXY to proxy.golang.org

foo@bar golang-dir % export GOPROXY=https://proxy.golang.org

After changing the GOPROXY to proxy.golang.org

foo@bar golang-dir % go mod tidy
go: downloading github.com/mattn/go-sqlite3 v2.0.1+incompatible

@Shriram-RP
Copy link
Author

Shriram-RP commented Oct 25, 2021

What version of Go are you using? Also, what is the output of go env?

Version - 1.17

go env output :-

GO111MODULE=""
GOARCH="arm64"
GOBIN=""
GOCACHE="/Users/{name}/Library/Caches/go-build"
GOENV="/Users/{name}/Library/Application Support/go/env"
GOEXE=""
GOEXPERIMENT=""
GOFLAGS=""
GOHOSTARCH="arm64"
GOHOSTOS="darwin"
GOINSECURE=""
GOMODCACHE="/Users/{name}/go/pkg/mod"
GONOPROXY=""
GONOSUMDB=""
GOOS="darwin"
GOPATH="/Users/{name}/go"
GOPRIVATE=""
GOPROXY="https://proxy.golang.org,direct"
GOROOT="/opt/homebrew/Cellar/go/1.17/libexec"
GOSUMDB="sum.golang.org"
GOTMPDIR=""
GOTOOLDIR="/opt/homebrew/Cellar/go/1.17/libexec/pkg/tool/darwin_arm64"
GOVCS=""
GOVERSION="go1.17"
GCCGO="gccgo"
AR="ar"
CC="clang"
CXX="clang++"
CGO_ENABLED="1"
GOMOD="/Users/{name}go/src/github.com/razorpay/reminders/go.mod"
CGO_CFLAGS="-g -O2"
CGO_CPPFLAGS=""
CGO_CXXFLAGS="-g -O2"
CGO_FFLAGS="-g -O2"
CGO_LDFLAGS="-g -O2"
PKG_CONFIG="pkg-config"
GOGCCFLAGS="-fPIC -arch arm64 -pthread -fno-caret-diagnostics -Qunused-arguments -fmessage-length=0 -fdebug-prefix-map=/var/folders/qs/9hmssmm50p7djrwxsww6nfkm0000gq/T/go-build1013432823=/tmp/go-build -gno-record-gcc-switches -fno-common"

@shaswa
Copy link

shaswa commented Oct 31, 2021

One way to fix this is adding the below code to the go.mod file
exclude github.com/mattn/go-sqlite3 v2.0.3+incompatible

@ghost
Copy link

ghost commented Nov 3, 2021

any fix ?

@mattn
Copy link
Owner

mattn commented Nov 3, 2021

This is not an issue of go-sqlite3. v2 was an accident as written in README.md

NOTE: The increase to v2 was an accident. There were no major changes or features.

@mattn mattn closed this as completed Nov 3, 2021
@ghost
Copy link

ghost commented Nov 3, 2021

@mattn I wonder if this would be a reason to use a retraction and just retag v2: https://golang.org/ref/mod#go-mod-file-retract

The impact of this is a bit broad. Large projects seem to have been impacted by this as well as small random odds and ends. Notice that tekton/cli was affected when I was trying to import it into my project. Also searching my cache I see gorm managed to get a commit with a v2 tag.

Can't imagine what else this might have landed in. A retraction seems like it would prevent tons of people from having to stuff exclude and replace into their module files, if I understand the mechanics of it correctly.


[protosam@nullhost]$ go mod init placebo
go: creating new go.mod: module placebo
[protosam@nullhost]$ go get github.com/tektoncd/[email protected]
go get: github.com/tektoncd/[email protected] updating to
	github.com/tektoncd/[email protected] requires
	github.com/tektoncd/hub/[email protected] requires
	github.com/go-testfixtures/testfixtures/[email protected] requires
	github.com/mattn/[email protected]+incompatible: reading github.com/mattn/go-sqlite3/go.mod at revision v2.0.2: unknown revision v2.0.2
[protosam@nullhost]$ grep -ri 'mattn/go-sqlite3 v2' ~/go/pkg/mod/cache
~/go/pkg/mod/cache/download/github.com/go-testfixtures/testfixtures/v3/@v/v3.2.0.mod:	github.com/mattn/go-sqlite3 v2.0.2+incompatible
~/go/pkg/mod/cache/download/github.com/jinzhu/gorm/@v/v1.9.12.mod:	github.com/mattn/go-sqlite3 v2.0.1+incompatible

@mattn
Copy link
Owner

mattn commented Nov 3, 2021

If once you clean module cache on your environment, vertion 1.14 shold be used in next time.

@ghost
Copy link

ghost commented Nov 3, 2021

That's not the case. People will be stumbling onto this for a while. go-sqlite3 is a very wide spread dependency and gets added into other projects during both manual and automated tasks.

Commits like this and this are actively referenced in projects today.

There are 39,000 files that come up with I search to get an idea of how wide spread this is.

@rittneje
Copy link
Collaborator

rittneje commented Nov 3, 2021

@protosam From what I understand, retracting the v2 tags won't resolve any of the existing issues. It just stops the issue from continuing to grow. If I have a dependency on some other module that in turn has a dependency on one of the broken v2 tags of this library, then go mod will still attempt to download that version. And unless you are using proxy.golang.org (or similar), it will fail. The only time retraction matters is when someone is explicitly upgrading their dependencies.

In other words, even if a retraction is published, someone would still have to go to every one of the broken dependents and fix their go.mod to reference a valid release. Then all affected developers have to go and update to the newer versions of these modules.

@mattn I'm guessing the reason this just become an issue recently is because the v2 tags were deleted from this repo. Is that correct? If so, would you be able to re-create them for the same commits? That way people using "direct" at least won't get failed builds. (This is orthogonal to retracting the v2 releases.)

@mattn mattn reopened this Nov 3, 2021
@mattn
Copy link
Owner

mattn commented Nov 3, 2021

Anyone know which commit should i tag ?

@ghost
Copy link

ghost commented Nov 3, 2021

@mattn I'm looking into this. I expect to find it in a go.sum file somewhere.

@ghost
Copy link

ghost commented Nov 3, 2021

Google had them cached in sum.golang.org and also they're still listed at pkg.go.dev.

v2.0.0+incompatible
v2.0.1+incompatible
v2.0.2+incompatible
v2.0.3+incompatible

I'm actually a bit confused on the serialization. Saying what I got in case someone else is able to understand deserializing this faster than I can.

@mattn
Copy link
Owner

mattn commented Nov 3, 2021

Should all tags be restored?

@mattn
Copy link
Owner

mattn commented Nov 3, 2021

I restored older v2.0.X tags. How goes good?

@mattn
Copy link
Owner

mattn commented Nov 3, 2021

So you (who used v2.0.X) should modify go.mod to switch v1.4.X.

require github.com/mattn/go-sqlite3 v1.14.9

@ghost
Copy link

ghost commented Nov 3, 2021

The thousands of forks and archive.org are pretty useful, these commits look good to me and below I've tested them. They work as I expect (they validated against sum.golang.org).

I did some additional testing to see how the retraction will work. I don't think it works against legacy versioning. Ideally if it worked, we would be capable of getting a message out to the end users so they know they're using bad versions and later these versions can be gracefully claimed for reuse.

No idea where to go with that discovery though. =/

So v2.0.0, v2.0.1, v2.0.2, and v2.0.3 work because the +incompatible tag and there is not a go.mod file (legacy versioning).

Tags v2.0.4, v2.0.5, and v2.0.6 do not seem to be needed. These have go.mod versioning and yield errors that people would have to do more work than necessary to work around. They also don't show up in sum.golang.org and are not listed on pkg.go.dev. Seems likely that no one would miss these tags. This leads me to believe they've not been in use.


Started with an empty project and tested getting a module that has a lower dependent that requires v2.0.2.

[protosam@nullhost]$ go mod init placebo
go: creating new go.mod: module placebo

[protosam@nullhost]$ go get github.com/tektoncd/[email protected]
go get: added github.com/tektoncd/cli v0.21.0

[protosam@nullhost]$ go mod verify
all modules verified

Below I tested against v2.0.0 - v2.0.6

[protosam@nullhost]$ go mod download github.com/mattn/[email protected]+incompatible
[protosam@nullhost]$ go mod tidy

[protosam@nullhost]$ go get github.com/mattn/[email protected]+incompatible
go get: upgraded github.com/mattn/go-sqlite3 v1.14.9 => v2.0.0+incompatible

[protosam@nullhost]$ go get github.com/mattn/[email protected]+incompatible
go get: upgraded github.com/mattn/go-sqlite3 v2.0.0+incompatible => v2.0.1+incompatible

[protosam@nullhost]$ go get github.com/mattn/[email protected]+incompatible
go get: upgraded github.com/mattn/go-sqlite3 v2.0.1+incompatible => v2.0.2+incompatible

[protosam@nullhost]$ go get github.com/mattn/[email protected]+incompatible
go: downloading github.com/mattn/go-sqlite3 v2.0.3+incompatible
go get: upgraded github.com/mattn/go-sqlite3 v2.0.2+incompatible => v2.0.3+incompatible

[protosam@nullhost]$ go get github.com/mattn/[email protected]+incompatible
go get: github.com/mattn/[email protected]+incompatible: invalid version: +incompatible suffix not allowed: module contains a go.mod file, so semantic import versioning is required

[protosam@nullhost]$ go get github.com/mattn/[email protected]
go get: github.com/mattn/[email protected]: invalid version: module contains a go.mod file, so major version must be compatible: should be v0 or v1, not v2

[protosam@nullhost]$ go get github.com/mattn/[email protected]
go get: github.com/mattn/[email protected]: invalid version: module contains a go.mod file, so major version must be compatible: should be v0 or v1, not v2

[protosam@nullhost]$ go get github.com/mattn/[email protected]
go get: github.com/mattn/[email protected]: invalid version: module contains a go.mod file, so major version must be compatible: should be v0 or v1, not v2

@sagikazarmark
Copy link

sagikazarmark commented Jan 11, 2022

@mattn I'd probably pin this issue while it's affecting people otherwise they are going to open new ones as they rarely look at slightly older issues.

I'd probably reach out to the Go team at this point to see if they have any ideas to stop the spread and fix the problem (eg. remove these entries from sumdb? Dunno if that's possible/good idea)

Retracting would probably mean that you can't use these tags in the future.

@mattn
Copy link
Owner

mattn commented Jan 11, 2022

This issue is already reported to Go team. golang/go#35732

@sagikazarmark
Copy link

@mattn as far as I can tell that issue was closed with working as intended. I was rather suggesting to contact them about this specific case, hoping that they could remove the offending entries from the sumdb/proxy.

@itmecho
Copy link

itmecho commented May 6, 2022

One way to fix this is adding the below code to the go.mod file exclude github.com/mattn/go-sqlite3 v2.0.3+incompatible

Thank you! This fixed my problem. When I ran go get github.com/mattn/[email protected], it downgraded another library I'm using (ory/fosite) from 0.40.2 to 0.35.1. I then added this line to my go.mod and ran go get github.com/ory/[email protected]. This upgraded the package without upgrading go-sqlite3 to the bad version.

ondrejbudai added a commit to ondrejbudai/osbuild-composer that referenced this issue Aug 28, 2022
Version 5.22 introduced a new option to /etc/containers/policy.json called
keyPaths, see

containers/image#1609

EL9 immediately took advantage of this new feature and started using it, see
https://gitlab.com/redhat/centos-stream/rpms/containers-common/-/commit/04645c4a84442da3324eea8f6538a5768e69919a

This quickly became an issue in our code: The go library (containers/image)
parses the configuration file very strictly and refuses to create a client
when policy.json with an unknown key is present on the filesystem. As we
used 5.21.1 that doesn't know the new key, our unit tests started to
failing when containers-common was present.

Reproducer:
podman run --pull=always --rm -it centos:stream9
dnf install -y dnf-plugins-core
dnf config-manager --set-enabled crb
dnf install -y gpgme-devel libassuan-devel krb5-devel golang git-core
git clone https://github.com/osbuild/osbuild-composer
cd osbuild-composer

# install the new containers-common and run the test
dnf install -y https://kojihub.stream.centos.org/kojifiles/packages/containers-common/1/44.el9/x86_64/containers-common-1-44.el9.x86_64.rpm
go test -count 1 ./...

# this returns:
--- FAIL: TestClientResolve (0.00s)
    client_test.go:31:
        	Error Trace:	client_test.go:31
        	Error:      	Received unexpected error:
        	            	Unknown key "keyPaths"
        	            	invalid policy in "/etc/containers/policy.json"
        	            	github.com/containers/image/v5/signature.NewPolicyFromFile
        	            		/osbuild-composer/vendor/github.com/containers/image/v5/signature/policy_config.go:88
        	            	github.com/osbuild/osbuild-composer/internal/container.NewClient
        	            		/osbuild-composer/internal/container/client.go:123
        	            	github.com/osbuild/osbuild-composer/internal/container_test.TestClientResolve
        	            		/osbuild-composer/internal/container/client_test.go:29
        	            	testing.tRunner
        	            		/usr/lib/golang/src/testing/testing.go:1439
        	            	runtime.goexit
        	            		/usr/lib/golang/src/runtime/asm_amd64.s:1571
        	Test:       	TestClientResolve
    client_test.go:32:
        	Error Trace:	client_test.go:32
        	Error:      	Expected value not to be nil.
        	Test:       	TestClientResolve

 When run with an older containers-common, it succeeds:
 dnf install -y https://kojihub.stream.centos.org/kojifiles/packages/containers-common/1/40.el9/x86_64/containers-common-1-40.el9.x86_64.rpm
 go test -count 1 ./...
 PASS

To sum it up, I had to upgrade github.com/containers/image/v5 to v5.22.0.
Unfortunately, this wasn't so simple, see

go get github.com/containers/image/v5@latest
go: github.com/containers/image/[email protected] requires
	github.com/letsencrypt/[email protected] requires
	github.com/honeycombio/[email protected] requires
	github.com/gobuffalo/pop/[email protected] requires
	github.com/mattn/[email protected]+incompatible: reading github.com/mattn/go-sqlite3/go.mod at revision v2.0.3: unknown revision v2.0.3

It turns out that github.com/mattn/[email protected]+incompatible has been
recently retracted mattn/go-sqlite3#998 and this
broke a ton of packages depending on it. I was able to fix it by adding

exclude github.com/mattn/go-sqlite3 v2.0.3+incompatible

to our go.mod, see
mattn/go-sqlite3#975 (comment)

After adding it,
go get github.com/containers/image/v5@latest
succeeded and tools/prepare-source.sh took care of the rest.

Signed-off-by: Ondřej Budai <[email protected]>
ondrejbudai added a commit to ondrejbudai/osbuild-composer that referenced this issue Aug 28, 2022
Version 5.22 introduced a new option to /etc/containers/policy.json called
keyPaths, see

containers/image#1609

EL9 immediately took advantage of this new feature and started using it, see
https://gitlab.com/redhat/centos-stream/rpms/containers-common/-/commit/04645c4a84442da3324eea8f6538a5768e69919a

This quickly became an issue in our code: The go library (containers/image)
parses the configuration file very strictly and refuses to create a client
when policy.json with an unknown key is present on the filesystem. As we
used 5.21.1 that doesn't know the new key, our unit tests started to
failing when containers-common was present.

Reproducer:
podman run --pull=always --rm -it centos:stream9
dnf install -y dnf-plugins-core
dnf config-manager --set-enabled crb
dnf install -y gpgme-devel libassuan-devel krb5-devel golang git-core
git clone https://github.com/osbuild/osbuild-composer
cd osbuild-composer

# install the new containers-common and run the test
dnf install -y https://kojihub.stream.centos.org/kojifiles/packages/containers-common/1/44.el9/x86_64/containers-common-1-44.el9.x86_64.rpm
go test -count 1 ./...

# this returns:
--- FAIL: TestClientResolve (0.00s)
    client_test.go:31:
        	Error Trace:	client_test.go:31
        	Error:      	Received unexpected error:
        	            	Unknown key "keyPaths"
        	            	invalid policy in "/etc/containers/policy.json"
        	            	github.com/containers/image/v5/signature.NewPolicyFromFile
        	            		/osbuild-composer/vendor/github.com/containers/image/v5/signature/policy_config.go:88
        	            	github.com/osbuild/osbuild-composer/internal/container.NewClient
        	            		/osbuild-composer/internal/container/client.go:123
        	            	github.com/osbuild/osbuild-composer/internal/container_test.TestClientResolve
        	            		/osbuild-composer/internal/container/client_test.go:29
        	            	testing.tRunner
        	            		/usr/lib/golang/src/testing/testing.go:1439
        	            	runtime.goexit
        	            		/usr/lib/golang/src/runtime/asm_amd64.s:1571
        	Test:       	TestClientResolve
    client_test.go:32:
        	Error Trace:	client_test.go:32
        	Error:      	Expected value not to be nil.
        	Test:       	TestClientResolve

 When run with an older containers-common, it succeeds:
 dnf install -y https://kojihub.stream.centos.org/kojifiles/packages/containers-common/1/40.el9/x86_64/containers-common-1-40.el9.x86_64.rpm
 go test -count 1 ./...
 PASS

To sum it up, I had to upgrade github.com/containers/image/v5 to v5.22.0.
Unfortunately, this wasn't so simple, see

go get github.com/containers/image/v5@latest
go: github.com/containers/image/[email protected] requires
	github.com/letsencrypt/[email protected] requires
	github.com/honeycombio/[email protected] requires
	github.com/gobuffalo/pop/[email protected] requires
	github.com/mattn/[email protected]+incompatible: reading github.com/mattn/go-sqlite3/go.mod at revision v2.0.3: unknown revision v2.0.3

It turns out that github.com/mattn/[email protected]+incompatible has been
recently retracted mattn/go-sqlite3#998 and this
broke a ton of packages depending on it. I was able to fix it by adding

exclude github.com/mattn/go-sqlite3 v2.0.3+incompatible

to our go.mod, see
mattn/go-sqlite3#975 (comment)

After adding it,
go get github.com/containers/image/v5@latest
succeeded and tools/prepare-source.sh took care of the rest.

Signed-off-by: Ondřej Budai <[email protected]>
thozza pushed a commit to osbuild/osbuild-composer that referenced this issue Aug 29, 2022
Version 5.22 introduced a new option to /etc/containers/policy.json called
keyPaths, see

containers/image#1609

EL9 immediately took advantage of this new feature and started using it, see
https://gitlab.com/redhat/centos-stream/rpms/containers-common/-/commit/04645c4a84442da3324eea8f6538a5768e69919a

This quickly became an issue in our code: The go library (containers/image)
parses the configuration file very strictly and refuses to create a client
when policy.json with an unknown key is present on the filesystem. As we
used 5.21.1 that doesn't know the new key, our unit tests started to
failing when containers-common was present.

Reproducer:
podman run --pull=always --rm -it centos:stream9
dnf install -y dnf-plugins-core
dnf config-manager --set-enabled crb
dnf install -y gpgme-devel libassuan-devel krb5-devel golang git-core
git clone https://github.com/osbuild/osbuild-composer
cd osbuild-composer

# install the new containers-common and run the test
dnf install -y https://kojihub.stream.centos.org/kojifiles/packages/containers-common/1/44.el9/x86_64/containers-common-1-44.el9.x86_64.rpm
go test -count 1 ./...

# this returns:
--- FAIL: TestClientResolve (0.00s)
    client_test.go:31:
        	Error Trace:	client_test.go:31
        	Error:      	Received unexpected error:
        	            	Unknown key "keyPaths"
        	            	invalid policy in "/etc/containers/policy.json"
        	            	github.com/containers/image/v5/signature.NewPolicyFromFile
        	            		/osbuild-composer/vendor/github.com/containers/image/v5/signature/policy_config.go:88
        	            	github.com/osbuild/osbuild-composer/internal/container.NewClient
        	            		/osbuild-composer/internal/container/client.go:123
        	            	github.com/osbuild/osbuild-composer/internal/container_test.TestClientResolve
        	            		/osbuild-composer/internal/container/client_test.go:29
        	            	testing.tRunner
        	            		/usr/lib/golang/src/testing/testing.go:1439
        	            	runtime.goexit
        	            		/usr/lib/golang/src/runtime/asm_amd64.s:1571
        	Test:       	TestClientResolve
    client_test.go:32:
        	Error Trace:	client_test.go:32
        	Error:      	Expected value not to be nil.
        	Test:       	TestClientResolve

 When run with an older containers-common, it succeeds:
 dnf install -y https://kojihub.stream.centos.org/kojifiles/packages/containers-common/1/40.el9/x86_64/containers-common-1-40.el9.x86_64.rpm
 go test -count 1 ./...
 PASS

To sum it up, I had to upgrade github.com/containers/image/v5 to v5.22.0.
Unfortunately, this wasn't so simple, see

go get github.com/containers/image/v5@latest
go: github.com/containers/image/[email protected] requires
	github.com/letsencrypt/[email protected] requires
	github.com/honeycombio/[email protected] requires
	github.com/gobuffalo/pop/[email protected] requires
	github.com/mattn/[email protected]+incompatible: reading github.com/mattn/go-sqlite3/go.mod at revision v2.0.3: unknown revision v2.0.3

It turns out that github.com/mattn/[email protected]+incompatible has been
recently retracted mattn/go-sqlite3#998 and this
broke a ton of packages depending on it. I was able to fix it by adding

exclude github.com/mattn/go-sqlite3 v2.0.3+incompatible

to our go.mod, see
mattn/go-sqlite3#975 (comment)

After adding it,
go get github.com/containers/image/v5@latest
succeeded and tools/prepare-source.sh took care of the rest.

Signed-off-by: Ondřej Budai <[email protected]>
@kvtb
Copy link

kvtb commented Sep 12, 2022

I restored older v2.0.X tags. How goes good?

The tags are disappeared again :(

@mattn
Copy link
Owner

mattn commented Sep 12, 2022

The 2.0.X was retracted.

https://pkg.go.dev/github.com/mattn/go-sqlite3?tab=versions

You may have already had go-sqlite3 version 2.0.X. Please update manually to 1.4.X.

@mattn mattn closed this as completed Sep 12, 2022
@rittneje
Copy link
Collaborator

@mattn Even though you retracted those versions, those tags should never be deleted, so that people can continue to download their dependencies, including for older versions of their projects.

@mattn
Copy link
Owner

mattn commented Sep 13, 2022

@rittneje Yes. My understanding is that once people manually revert to 1.4.X, they will not be upgraded thereafter since 2.X was retracted. I believe this is a problem that cannot be recovered automatically.

@rittneje
Copy link
Collaborator

My point is that deleting those v2 tags breaks people's builds so we should not do that. We can only strongly discourage their use going forward (which the retraction helps).

@mattn
Copy link
Owner

mattn commented Sep 13, 2022

I have no plans to release v2 any more.

@rittneje
Copy link
Collaborator

But those tags were already released. So there are other libraries and projects that reference them. By deleting them, those versions cannot be built properly anymore (without using a Go module proxy), which was the original stated issue. We can tell people not to use v2 in new versions of their code anymore, but we should not cause the older/existing versions to break.

@mattn
Copy link
Owner

mattn commented Sep 13, 2022

v2 was an accidental release because the Go mod incorrectly identified the branch name as a tag. #998 is a PR to add retract as a way to fix these problems.

@rittneje
Copy link
Collaborator

rittneje commented Sep 13, 2022

Retract doesn't really "fix" the problem per se. In fact, the problem is not really fixable in the way you are hoping. Once a version has been released, it should not be deleted. The retraction fixes some latent issues with fetching/upgrading via go get github.com/mattn/go-sqlite3@latest. However, if a project already references, say, v2.0.3 in its go.sum file, the retraction does not (and should not) interfere with that.

Typically people are using the default Go module proxy at proxy.golang.org. By design, this proxy does not allow versions to be deleted, so that you have eternal build reproducibility. This is an important feature for most people, as they expect all previous versions of their code to always work.

However, in the event that you are not using a proxy (GOPROXY=direct) then it instead relies on the GitHub tag still existing. It is for this reason that the tags of this repo should be preserved.

My understanding is there were two issues in the history of this repo. The first is the existence of a branch named v2.0.0, which, due to an unfortunate bug in go mod, was misinterpreted as a release version. The second is a bunch of actual tags (v2.0.1, v2.0.2, etc.) that were released in error.

@mattn
Copy link
Owner

mattn commented Sep 13, 2022

Yes, so I added note in README. https://github.com/mattn/go-sqlite3#go-sqlite3

I have not released v2 in other projects because I do not break compatibility. I also don't want to make it a hassle for the user. Fortunately, as far as I can see, many people still have v1. Because the bump to v2 is not automatic. So I decided to save the v1 users instead of the v2 users. Do you still think we should release v2.1.X or v3?

BTW, go get with fresh go.mod always get 1.4.X.

image

@rittneje
Copy link
Collaborator

Do you still think we should release v2.1.X or v3?

I think you should continue releasing v1.X.Y, as you have been. (In the event you do decide to make breaking changes, going to v3 is probably best to have a clean break from v2.) However, the v2 tags that were already published before should continue to exist indefinitely, you just won't be releasing any new v2.X.Y tags.

BTW, go get with fresh go.mod always get 1.4.X.

Yes this is due to fixes in go mod itself as well as the retraction you did. But if you specifically mention v2.0.3 in your go.mod then it works despite the retraction.

@mattn
Copy link
Owner

mattn commented Sep 13, 2022

I figured that leaving the v2 tag could be confusing to new users.

Yes this is due to fixes in go mod itself as well as the retraction you did. But if you specifically mention v2.0.3 in your go.mod then it works despite the retraction.

Yes, it's same with my understanding. Thanks. BTW, I don't touch v2 tags recently.

@kvtb
Copy link

kvtb commented Sep 14, 2022

The 2.0.X was retracted.

https://pkg.go.dev/github.com/mattn/go-sqlite3?tab=versions

You may have already had go-sqlite3 version 2.0.X. Please update manually to 1.4.X.

Indeed, I can update my code, but there are tons of 3rd party libs which require go-sqlite v2.x.x
Should I fork them all ?

@kvtb
Copy link

kvtb commented Sep 14, 2022

I see that v2 was released by mistake and having legacy v2 and ongoing v1 is confusing.
But it seems that in order to fix that mistake you should keep v1 and v2 frozen and continue development as v3

@ntsd
Copy link

ntsd commented Sep 21, 2022

I got the same problem but after I did go env -w GOPROXY="https://proxy.golang.org,direct" now it can download.

@drgrib
Copy link

drgrib commented Dec 27, 2022

@mattn Deleting the v2.0.3 broke code that I hadn't needed to touch for years. Now when I try to use v1.14.9 I get this error for that code that worked fine before:

Binary was compiled with 'CGO_ENABLED=0', go-sqlite3 requires cgo to work. This is a stub

None of the logic I wrote works anymore and it used to. I just get empty values when I query now. How do I fix this?
https://github.com/drgrib/alfred-bear/blob/master/db/db.go#L172

I have CGO_ENABLED=1 in my .bash_profile. Can we get clearer instructions on how to fix #855? The README is not helpful, at least for someone on a Mac.

@rittneje
Copy link
Collaborator

@drgrib Are you getting an error, or an empty result set? How specifically are you compiling and executing your code? Have you tried with the latest release (1.14.16)? Also, what does go env say?

#855 is not fixable as the Go toolchain does not support that mode of operation. You must compile with CGO_ENABLED=1 every time.

@drgrib
Copy link

drgrib commented Dec 27, 2022

@rittneje I guess I'm technically not getting an error, just that message when I run, along with empty result sets.

My bad too. I had CGO_ENABLED=1 in my .bash_profile rather than export CGO_ENABLED=1. It does work now. Thanks for following up.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

10 participants