Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

chore: adopt testcontainers-go for Postgres, MySQL, MSSQL and MongoDB #1515

Open
wants to merge 26 commits into
base: main
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
26 commits
Select commit Hold shift + click to select a range
4ed3649
chore: use testcontainers-go in mongodb store
mdelapenya Aug 29, 2024
0e060e1
chore: use testcontainers-go in mssql store
mdelapenya Aug 29, 2024
2f17493
chore: use testcontainers-go in mysql store
mdelapenya Aug 29, 2024
2f5263e
chore: use testcontainers-go in postgres store
mdelapenya Aug 29, 2024
689759c
fix: wrong image name for mongo
mdelapenya Aug 29, 2024
ff25433
fix: proper Go version matrix
mdelapenya Aug 29, 2024
fc01fc5
fix: remove toolchain in go mod
mdelapenya Aug 29, 2024
4ad4b03
chor: consistent Go version
mdelapenya Aug 29, 2024
b53f02b
fix: remove toolchain
mdelapenya Aug 29, 2024
de8f798
chore: consistent Go versions matrix
mdelapenya Aug 29, 2024
471a6ac
fix: remove more postgres options
mdelapenya Sep 16, 2024
c80f126
chore: adjust Go matrix versions to use 1.20 as the minimum version
mdelapenya Sep 16, 2024
1d6c0c3
chore: set Go 1.21 for modules
mdelapenya Sep 16, 2024
4d7a172
chore: use Ubuntu 22.04 base image for MSSQL
mdelapenya Sep 18, 2024
75c8525
chore: close store for previous modules
mdelapenya Sep 18, 2024
1bb4d92
chore: close store
mdelapenya Sep 18, 2024
f962d64
chore: terminate container on end
mdelapenya Sep 18, 2024
1c23d53
chore: bump MSSQL image to not crash in the GH workers
mdelapenya Sep 19, 2024
7bbad9a
fix: swap values in mysql constants
mdelapenya Sep 19, 2024
efbc89f
fix: do not close store twice in tests
mdelapenya Sep 19, 2024
153c837
fix: reorder
mdelapenya Sep 19, 2024
43a47ef
fix: do not close twice in mysql test
mdelapenya Sep 19, 2024
3cd89db
fix: terminate second mysql container
mdelapenya Sep 19, 2024
60be628
chore: bump couchbase image
mdelapenya Sep 19, 2024
a4849c8
Revert "fix: terminate second mysql container"
mdelapenya Sep 19, 2024
b3a2e40
chore: bump testcontainers-go to v0.34.0
mdelapenya Oct 23, 2024
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
58 changes: 5 additions & 53 deletions .github/workflows/benchmark.yml
Original file line number Diff line number Diff line change
Expand Up @@ -34,49 +34,6 @@ jobs:
image: 'memcached:latest'
ports:
- '11211:11211'
mongo:
image: 'mongo:latest'
ports:
- '27017:27017'
mssql:
image: 'mcmoe/mssqldocker:latest'
ports:
- '1433:1433'
env:
ACCEPT_EULA: Y
SA_PASSWORD: MsSql!1234
MSSQL_DB: master
MSSQL_USER: sa
MSSQL_PASSWORD: MsSql!1234
options: >-
--health-cmd "/opt/mssql-tools/bin/sqlcmd -U sa -P $SA_PASSWORD -Q 'select 1' -b -o /dev/null"
--health-interval 1s
--health-timeout 30s
--health-start-period 10s
--health-retries 20
mysql:
image: 'mysql:latest'
env:
MYSQL_DATABASE: fiber
MYSQL_USER: username
MYSQL_PASSWORD: password
MYSQL_ROOT_PASSWORD: password
ports:
- '3306:3306'
options: >-
--health-cmd "mysqladmin ping" --health-interval 10s --health-timeout
5s --health-retries 5
postgres:
image: 'postgres:latest'
ports:
- '5432:5432'
env:
POSTGRES_DB: fiber
POSTGRES_USER: username
POSTGRES_PASSWORD: "pass#w%rd"
options: >-
--health-cmd pg_isready --health-interval 10s --health-timeout 5s
--health-retries 5

steps:
- name: Fetch Repository
Expand Down Expand Up @@ -152,18 +109,13 @@ jobs:
done
shell: bash
env:
MSSQL_DATABASE: master
MSSQL_USERNAME: sa
MSSQL_PASSWORD: MsSql!1234
MYSQL_USERNAME: username
MYSQL_PASSWORD: password
MYSQL_DATABASE: fiber
POSTGRES_DATABASE: fiber
POSTGRES_USERNAME: username
POSTGRES_PASSWORD: "pass#w%rd"
TEST_CLICKHOUSE_IMAGE: "clickhouse/clickhouse-server:23-alpine"
TEST_COUCHBASE_IMAGE: "couchbase:enterprise-7.1.1"
TEST_COUCHBASE_IMAGE: "couchbase:enterprise-7.6.3"
TEST_MINIO_IMAGE: "docker.io/minio/minio:RELEASE.2024-08-17T01-24-54Z"
TEST_MONGODB_IMAGE: "docker.io/mongo:7"
TEST_MSSQL_IMAGE: "mcr.microsoft.com/mssql/server:2022-CU14-ubuntu-22.04"
TEST_MYSQL_IMAGE: "docker.io/mysql:9"
TEST_POSTGRES_IMAGE: "docker.io/postgres:16-alpine"

- name: Get Previous Benchmark Results
uses: actions/cache@v4
Expand Down
2 changes: 1 addition & 1 deletion .github/workflows/test-couchbase.yml
Original file line number Diff line number Diff line change
Expand Up @@ -27,5 +27,5 @@ jobs:
go-version: '${{ matrix.go-version }}'
- name: Run Test
env:
TEST_COUCHBASE_IMAGE: couchbase:enterprise-7.1.1
TEST_COUCHBASE_IMAGE: couchbase:enterprise-7.6.3
run: cd ./couchbase && go test ./... -v -race
9 changes: 3 additions & 6 deletions .github/workflows/test-mongodb.yml
Original file line number Diff line number Diff line change
Expand Up @@ -12,17 +12,12 @@ name: "Tests Mongodb"
jobs:
Tests:
runs-on: ubuntu-latest
services:
mongo:
image: 'mongo:latest'
ports:
- '27017:27017'
strategy:
matrix:
go-version:
- 1.19.x
- 1.20.x
- 1.21.x
- 1.22.x
mdelapenya marked this conversation as resolved.
Show resolved Hide resolved
steps:
- name: Fetch Repository
uses: actions/checkout@v4
Expand All @@ -31,5 +26,7 @@ jobs:
with:
go-version: '${{ matrix.go-version }}'
- name: Run Test
env:
TEST_MONGODB_IMAGE: docker.io/mongo:7
run: cd ./mongodb && go test ./... -v -race

25 changes: 3 additions & 22 deletions .github/workflows/test-mssql.yml
Original file line number Diff line number Diff line change
Expand Up @@ -12,29 +12,12 @@ name: "Tests MSSQL"
jobs:
Tests:
runs-on: ubuntu-latest
services:
mssql:
image: 'mcmoe/mssqldocker:latest'
ports:
- '1433:1433'
env:
ACCEPT_EULA: Y
SA_PASSWORD: MsSql!1234
MSSQL_DB: master
MSSQL_USER: sa
MSSQL_PASSWORD: MsSql!1234
options: >-
--health-cmd "/opt/mssql-tools/bin/sqlcmd -U sa -P $SA_PASSWORD -Q 'select 1' -b -o /dev/null"
--health-interval 1s
--health-timeout 30s
--health-start-period 10s
--health-retries 20
strategy:
matrix:
go-version:
- 1.19.x
- 1.20.x
- 1.21.x
- 1.22.x
steps:
- name: Fetch Repository
uses: actions/checkout@v4
Expand All @@ -43,8 +26,6 @@ jobs:
with:
go-version: '${{ matrix.go-version }}'
- name: Run Test
run: cd ./mssql && go test ./... -v -race
env:
MSSQL_DATABASE: master
MSSQL_USERNAME: sa
MSSQL_PASSWORD: MsSql!1234
TEST_MSSQL_IMAGE: "mcr.microsoft.com/mssql/server:2022-CU14-ubuntu-22.04"
run: cd ./mssql && go test ./... -v -race
21 changes: 3 additions & 18 deletions .github/workflows/test-mysql.yml
Original file line number Diff line number Diff line change
Expand Up @@ -12,25 +12,12 @@ name: "Tests MySQL"
jobs:
Tests:
runs-on: ubuntu-latest
services:
mysql:
image: 'mysql:latest'
env:
MYSQL_DATABASE: fiber
MYSQL_USER: username
MYSQL_PASSWORD: password
MYSQL_ROOT_PASSWORD: password
ports:
- '3306:3306'
options: >-
--health-cmd "mysqladmin ping" --health-interval 10s --health-timeout
5s --health-retries 5
strategy:
matrix:
go-version:
- 1.19.x
- 1.20.x
- 1.21.x
- 1.22.x
mdelapenya marked this conversation as resolved.
Show resolved Hide resolved
steps:
- name: Fetch Repository
uses: actions/checkout@v4
Expand All @@ -39,8 +26,6 @@ jobs:
with:
go-version: '${{ matrix.go-version }}'
- name: Run Test
run: cd ./mysql && go test ./... -v -race
env:
MYSQL_USERNAME: username
MYSQL_PASSWORD: password
MYSQL_DATABASE: fiber
TEST_MYSQL_IMAGE: docker.io/mysql:9
run: cd ./mysql && go test ./... -v -race
18 changes: 2 additions & 16 deletions .github/workflows/test-postgres.yml
Original file line number Diff line number Diff line change
Expand Up @@ -12,18 +12,6 @@ name: "Tests Postgres"
jobs:
Tests:
runs-on: ubuntu-latest
services:
postgres:
image: 'postgres:latest'
ports:
- '5432:5432'
env:
POSTGRES_DB: fiber
POSTGRES_USER: username
POSTGRES_PASSWORD: "pass#w%rd"
options: >-
--health-cmd pg_isready --health-interval 10s --health-timeout 5s
--health-retries 5
strategy:
matrix:
go-version:
Expand All @@ -38,8 +26,6 @@ jobs:
with:
go-version: '${{ matrix.go-version }}'
- name: Run Test
run: cd ./postgres && go test ./... -v -race
env:
POSTGRES_DATABASE: fiber
POSTGRES_USERNAME: username
POSTGRES_PASSWORD: "pass#w%rd"
TEST_POSTGRES_IMAGE: "docker.io/postgres:16-alpine"
run: cd ./postgres && go test ./... -v -race
21 changes: 19 additions & 2 deletions clickhouse/clickhouse_test.go
Original file line number Diff line number Diff line change
Expand Up @@ -42,6 +42,11 @@ func getTestConnection(t testing.TB, cfg Config) (*Storage, error) {
if err != nil {
return nil, err
}
t.Cleanup(func() {
if c != nil {
require.NoError(t, c.Terminate(ctx))
}
})

hostPort, err := c.ConnectionHost(ctx)
if err != nil {
Expand All @@ -67,13 +72,14 @@ func getTestConnection(t testing.TB, cfg Config) (*Storage, error) {
}

func Test_Connection(t *testing.T) {
_, err := getTestConnection(t, Config{
client, err := getTestConnection(t, Config{
Engine: Memory,
Table: "test_table",
Clean: true,
})

require.NoError(t, err)

defer client.Close()
}

func Test_Set(t *testing.T) {
Expand Down Expand Up @@ -191,6 +197,17 @@ func Test_Reset(t *testing.T) {
assert.Equal(t, []byte{}, value)
}

func TestClose_ShouldReturn_NoError(t *testing.T) {
client, err := getTestConnection(t, Config{
Engine: Memory,
Table: "test_table",
Clean: true,
})
require.NoError(t, err)

require.NoError(t, client.Close())
}

func Benchmark_Clickhouse_Set(b *testing.B) {
b.ReportAllocs()
b.ResetTimer()
Expand Down
8 changes: 4 additions & 4 deletions clickhouse/go.mod
Original file line number Diff line number Diff line change
@@ -1,11 +1,11 @@
module github.com/gofiber/storage/clickhouse

go 1.21
go 1.22

require (
github.com/ClickHouse/clickhouse-go/v2 v2.26.0
github.com/stretchr/testify v1.9.0
github.com/testcontainers/testcontainers-go/modules/clickhouse v0.33.0
github.com/testcontainers/testcontainers-go/modules/clickhouse v0.34.0
)

require (
Expand All @@ -18,7 +18,7 @@ require (
github.com/containerd/containerd v1.7.18 // indirect
github.com/containerd/log v0.1.0 // indirect
github.com/containerd/platforms v0.2.1 // indirect
github.com/cpuguy83/dockercfg v0.3.1 // indirect
github.com/cpuguy83/dockercfg v0.3.2 // indirect
github.com/davecgh/go-spew v1.1.1 // indirect
github.com/distribution/reference v0.6.0 // indirect
github.com/docker/docker v27.1.1+incompatible // indirect
Expand Down Expand Up @@ -53,7 +53,7 @@ require (
github.com/shoenig/go-m1cpu v0.1.6 // indirect
github.com/shopspring/decimal v1.4.0 // indirect
github.com/sirupsen/logrus v1.9.3 // indirect
github.com/testcontainers/testcontainers-go v0.33.0 // indirect
github.com/testcontainers/testcontainers-go v0.34.0 // indirect
github.com/tklauser/go-sysconf v0.3.12 // indirect
github.com/tklauser/numcpus v0.6.1 // indirect
github.com/yusufpapurcu/wmi v1.2.3 // indirect
Expand Down
14 changes: 8 additions & 6 deletions clickhouse/go.sum
Original file line number Diff line number Diff line change
Expand Up @@ -20,8 +20,8 @@ github.com/containerd/log v0.1.0 h1:TCJt7ioM2cr/tfR8GPbGf9/VRAX8D2B4PjzCpfX540I=
github.com/containerd/log v0.1.0/go.mod h1:VRRf09a7mHDIRezVKTRCrOq78v577GXq3bSa3EhrzVo=
github.com/containerd/platforms v0.2.1 h1:zvwtM3rz2YHPQsF2CHYM8+KtB5dvhISiXh5ZpSBQv6A=
github.com/containerd/platforms v0.2.1/go.mod h1:XHCb+2/hzowdiut9rkudds9bE5yJ7npe7dG/wG+uFPw=
github.com/cpuguy83/dockercfg v0.3.1 h1:/FpZ+JaygUR/lZP2NlFI2DVfrOEMAIKP5wWEJdoYe9E=
github.com/cpuguy83/dockercfg v0.3.1/go.mod h1:sugsbF4//dDlL/i+S+rtpIWp+5h0BHJHfjj5/jFyUJc=
github.com/cpuguy83/dockercfg v0.3.2 h1:DlJTyZGBDlXqUZ2Dk2Q3xHs/FtnooJJVaad2S9GKorA=
github.com/cpuguy83/dockercfg v0.3.2/go.mod h1:sugsbF4//dDlL/i+S+rtpIWp+5h0BHJHfjj5/jFyUJc=
github.com/creack/pty v1.1.18 h1:n56/Zwd5o6whRC5PMGretI4IdRLlmBXYNjScPaBgsbY=
github.com/creack/pty v1.1.18/go.mod h1:MOBLtS5ELjhRRrroQr9kyvTxUAFNvYEK993ew/Vr4O4=
github.com/davecgh/go-spew v1.1.0/go.mod h1:J7Y8YcW2NihsgmVo/mv3lAwl/skON4iLHjSsI+c5H38=
Expand Down Expand Up @@ -122,17 +122,19 @@ github.com/sirupsen/logrus v1.9.3/go.mod h1:naHLuLoDiP4jHNo9R0sCBMtWGeIprob74mVs
github.com/stretchr/objx v0.1.0/go.mod h1:HFkY916IF+rwdDfMAkV7OtwuqBVzrE8GR6GFx+wExME=
github.com/stretchr/objx v0.4.0/go.mod h1:YvHI0jy2hoMjB+UWwv71VJQ9isScKT/TqJzVSSt89Yw=
github.com/stretchr/objx v0.5.0/go.mod h1:Yh+to48EsGEfYuaHDzXPcE3xhTkx73EhmCGUpEOglKo=
github.com/stretchr/objx v0.5.2 h1:xuMeJ0Sdp5ZMRXx/aWO6RZxdr3beISkG5/G/aIRr3pY=
github.com/stretchr/objx v0.5.2/go.mod h1:FRsXN1f5AsAjCGJKqEizvkpNtU+EGNCLh3NxZ/8L+MA=
github.com/stretchr/testify v1.6.1/go.mod h1:6Fq8oRcR53rry900zMqJjRRixrwX3KX962/h/Wwjteg=
github.com/stretchr/testify v1.7.0/go.mod h1:6Fq8oRcR53rry900zMqJjRRixrwX3KX962/h/Wwjteg=
github.com/stretchr/testify v1.7.1/go.mod h1:6Fq8oRcR53rry900zMqJjRRixrwX3KX962/h/Wwjteg=
github.com/stretchr/testify v1.8.0/go.mod h1:yNjHg4UonilssWZ8iaSj1OCr/vHnekPRkoO+kdMU+MU=
github.com/stretchr/testify v1.8.4/go.mod h1:sz/lmYIOXD/1dqDmKjjqLyZ2RngseejIcXlSw2iwfAo=
github.com/stretchr/testify v1.9.0 h1:HtqpIVDClZ4nwg75+f6Lvsy/wHu+3BoSGCbBAcpTsTg=
github.com/stretchr/testify v1.9.0/go.mod h1:r2ic/lqez/lEtzL7wO/rwa5dbSLXVDPFyf8C91i36aY=
github.com/testcontainers/testcontainers-go v0.33.0 h1:zJS9PfXYT5O0ZFXM2xxXfk4J5UMw/kRiISng037Gxdw=
github.com/testcontainers/testcontainers-go v0.33.0/go.mod h1:W80YpTa8D5C3Yy16icheD01UTDu+LmXIA2Keo+jWtT8=
github.com/testcontainers/testcontainers-go/modules/clickhouse v0.33.0 h1:YbB5DBkpgY+GlGPFqTSV1hzWPm3ZHirEyooZrj+ZXK0=
github.com/testcontainers/testcontainers-go/modules/clickhouse v0.33.0/go.mod h1:qJuMPl9yWIWasmdBILM2uDk1Ny1kdeigcKMJ6A8PZz0=
github.com/testcontainers/testcontainers-go v0.34.0 h1:5fbgF0vIN5u+nD3IWabQwRybuB4GY8G2HHgCkbMzMHo=
github.com/testcontainers/testcontainers-go v0.34.0/go.mod h1:6P/kMkQe8yqPHfPWNulFGdFHTD8HB2vLq/231xY2iPQ=
github.com/testcontainers/testcontainers-go/modules/clickhouse v0.34.0 h1:1CO1aV5kFXWvFCTxgIVjcMXaXwO8cznw58F+V8Kp+Fg=
github.com/testcontainers/testcontainers-go/modules/clickhouse v0.34.0/go.mod h1:TTecjXxuNwIyNfSKo6X071LoUkMIa3+unU5yam8Db+s=
github.com/tidwall/pretty v1.0.0/go.mod h1:XNkn88O1ChpSDQmQeStsy+sBenx6DDtFZJxhVysOjyk=
github.com/tklauser/go-sysconf v0.3.12 h1:0QaGUFOdQaIVdPgfITYzaTegZvdCjmYO52cSFAEVmqU=
github.com/tklauser/go-sysconf v0.3.12/go.mod h1:Ho14jnntGE1fpdOqQEEaiKRpvIavV0hSfmBq8nJbHYI=
Expand Down
Loading
Loading