used uber atomic bool instead standard in lock/unlock db #580

prinkov · 2021-06-12T19:04:06Z

No description provided.

coveralls · 2021-06-12T19:36:30Z

Pull Request Test Coverage Report for Build 466

151 of 218 (69.27%) changed or added relevant lines in 16 files are covered.
5 unchanged lines in 5 files lost coverage.
Overall coverage decreased (-0.3%) to 57.141%

Changes Missing Coverage	Covered Lines	Changed/Added Lines	%
database/clickhouse/clickhouse.go	0	1	0.0%
database/ql/ql.go	2	3	66.67%
database/sqlcipher/sqlcipher.go	2	3	66.67%
database/sqlite/sqlite.go	2	3	66.67%
database/sqlite3/sqlite3.go	2	3	66.67%
database/cassandra/cassandra.go	2	4	50.0%
database/firebird/firebird.go	2	4	50.0%
database/redshift/redshift.go	2	4	50.0%
database/stub/stub.go	2	4	50.0%
database/mongodb/mongodb.go	38	44	86.36%

Files with Coverage Reduction	New Missed Lines	%
database/cockroachdb/cockroachdb.go	1	61.57%
database/ql/ql.go	1	62.2%
database/sqlcipher/sqlcipher.go	1	66.1%
database/sqlite3/sqlite3.go	1	66.1%
database/sqlite/sqlite.go	1	66.1%

Totals
Change from base Build 420:	-0.3%
Covered Lines:	3577
Relevant Lines:	6260

💛 - Coveralls

dhui

Thanks for the PR! 🙏
I've been meaning to make this change for a while but haven't gotten around to it.

dhui · 2021-06-14T06:35:56Z

database/clickhouse/clickhouse.go

@@ -285,7 +285,7 @@ func (ch *ClickHouse) Lock() error {
 }
 func (ch *ClickHouse) Unlock() error {
 	if !ch.isLocked.CAS(true, false) {
-		return database.ErrLocked
+		return database.ErrNotLocked


Good catch!

yeah, this is my bug))

dhui · 2021-06-14T06:40:09Z

database/pgx/pgx.go

@@ -220,7 +221,7 @@ func (p *Postgres) Close() error {

 // https://www.postgresql.org/docs/9.6/static/explicit-locking.html#ADVISORY-LOCKS
 func (p *Postgres) Lock() error {
-	if p.isLocked {
+	if p.isLocked.Load() {


We should replace this with CAS, release the local if the postgres lock fails, and remove the CAS call below for a proper local lock implementation

A lot of thanks for review. fixed it

dhui · 2021-06-14T06:43:47Z

database/postgres/postgres.go

@@ -214,7 +215,7 @@ func (p *Postgres) Close() error {

 // https://www.postgresql.org/docs/9.6/static/explicit-locking.html#ADVISORY-LOCKS
 func (p *Postgres) Lock() error {
-	if p.isLocked {
+	if p.isLocked.Load() {


Same here about fixing the local lock optimization implementation

dhui · 2021-06-14T06:44:54Z

database/sqlserver/sqlserver.go

@@ -154,7 +155,7 @@ func (ss *SQLServer) Close() error {

 // Lock creates an advisory local on the database to prevent multiple migrations from running at the same time.
 func (ss *SQLServer) Lock() error {
-	if ss.isLocked {
+	if ss.isLocked.Load() {


Same here about fixing the local lock optimization implementation

thnx,fixed it

dhui · 2021-06-14T06:45:30Z

database/sqlserver/sqlserver.go

@@ -181,7 +184,7 @@ func (ss *SQLServer) Lock() error {

 // Unlock froms the migration lock from the database
 func (ss *SQLServer) Unlock() error {
-	if !ss.isLocked {
+	if !ss.isLocked.Load() {


Same here about fixing the local lock optimization implementation

thnx,fixed it

dhui · 2021-06-16T04:07:33Z

database/mysql/mysql.go

-		}
-		return nil
+	if !success {
+		m.isLocked.Store(false)


~~Add a comment here about how removing the local lock in this situation is desired since the local lock is an optimization and it's not trying to reflect the state of the db lock.~~

Actually, let's add our own CAS wrapper to automatically restore the lock state on error (in database/util.go) so we don't need to remember to do it:

func casRestoreOnErr(lock *atomic.Bool, o, n bool, f func() error) error { if !lock.CAS(o, n) { return ErrLocked } if err := f(); err != nil { // Automatically unlock lock.Store(o) return err } return nil }

Also, can you add tests for this wrapper?

Thank you. Of course I will do it

Done, review pls

dhui

Thanks again for addressing my feedback! Just a few minor fixes left!

database/util_test.go

dhui

Thanks for addressing my feedback!
Sorry for the delay, I must have missed a notification that there were more commits to review...

dhui · 2021-06-23T18:29:23Z

database/mongodb/mongodb.go

 	return database.CasRestoreOnErr(&m.isLocked, false, true, database.ErrLocked, func() error {
-		if !m.config.Locking.Enabled {


I think we can always try to acquire the local lock before acquiring the db lock even if locking is disabled. IIRC, the locking enabled config was added for backwards compatibility and to avoid an extra network hop for existing clients

ok, thanks. Done

dhui

Thanks again for the PR and addressing all of the feedback!

used uber atomic bool instead standard in lock/unlock db

9814da9

dhui requested changes Jun 14, 2021

View reviewed changes

fixing local lock optimization implementation

55902af

prinkov force-pushed the master branch from e61aaf1 to 55902af Compare June 14, 2021 18:17

prinkov added 3 commits June 15, 2021 22:35

Merge branch 'master' into master

1e867eb

fixing mysql local lock optimization implementation

1598b1e

fixing cockroachdb local lock optimization implementation

93b3798

dhui requested changes Jun 16, 2021

View reviewed changes

prinkov added 3 commits June 16, 2021 12:26

CAS wrapper to automatically restore

c1da4a8

added test for cas with restore

312d98d

added atomic lock to sqlite & mongo

0b282a5

prinkov force-pushed the master branch from a844cdf to 0b282a5 Compare June 16, 2021 10:09

added atomic lock to sqlite & mongo

c4fd509

prinkov force-pushed the master branch from 0b282a5 to c4fd509 Compare June 16, 2021 10:18

Merge branch 'master' of https://github.com/prinkov/migrate

946b672

dhui requested changes Jun 16, 2021

View reviewed changes

database/util_test.go Show resolved Hide resolved

database/util_test.go Outdated Show resolved Hide resolved

database/util_test.go Outdated Show resolved Hide resolved

database/util_test.go Show resolved Hide resolved

fix/refactor test cas restore function

71bfde6

prinkov force-pushed the master branch from 752a4ac to 71bfde6 Compare June 16, 2021 20:38

a.prinkov and others added 2 commits June 17, 2021 11:04

Merge branch 'master' of https://github.com/prinkov/migrate

8120a9a

disable lock for mongo

93fac42

dhui requested changes Jun 23, 2021

View reviewed changes

mongo local lock before acquiring the db lock

11e2c9d

dhui approved these changes Jun 23, 2021

View reviewed changes

dhui merged commit 3dfb0ff into golang-migrate:master Jun 23, 2021

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

used uber atomic bool instead standard in lock/unlock db #580

used uber atomic bool instead standard in lock/unlock db #580

prinkov commented Jun 12, 2021

coveralls commented Jun 12, 2021 •

edited

Loading

dhui left a comment

dhui Jun 14, 2021

prinkov Jun 14, 2021

dhui Jun 14, 2021

prinkov Jun 14, 2021

dhui Jun 14, 2021

prinkov Jun 14, 2021

dhui Jun 14, 2021

prinkov Jun 14, 2021

dhui Jun 14, 2021

prinkov Jun 14, 2021

dhui Jun 16, 2021

prinkov Jun 16, 2021

prinkov Jun 16, 2021

dhui left a comment

dhui left a comment

dhui Jun 23, 2021

prinkov Jun 23, 2021

dhui left a comment

		return database.CasRestoreOnErr(&m.isLocked, false, true, database.ErrLocked, func() error {
		if !m.config.Locking.Enabled {

used uber atomic bool instead standard in lock/unlock db #580

used uber atomic bool instead standard in lock/unlock db #580

Conversation

prinkov commented Jun 12, 2021

coveralls commented Jun 12, 2021 • edited Loading

Pull Request Test Coverage Report for Build 466

💛 - Coveralls

dhui left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

dhui left a comment

Choose a reason for hiding this comment

dhui left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

dhui left a comment

Choose a reason for hiding this comment

coveralls commented Jun 12, 2021 •

edited

Loading