fix: broken tx retries for cluster clients after #697 #709

rueian · 2024-12-22T14:30:39Z

No description provided.

rueian · 2024-12-22T14:46:28Z

cluster.go

 			return nil
 		}
-		count.m[p]++
+		count.m[cc]++


make naming consistent.

rueian · 2024-12-22T14:48:37Z

cluster.go

 				break
 			}
 		}
 		if last == cmds.InitSlot {
 			return nil
 		}
+	} else if init {
+		cc := c.pslots[last]
+		count.m[cc] += inits


We don't count cmd.InitSlot commands to c.pslots[last] in the code above this, so add the count back here.

wyxloading · 2024-12-23T02:57:18Z

Find one extreme case:

MULTI
MULTI
GET foo
EXEC

We will get [4]RedisResult with all successful results. And resp[3] (which means EXEC result) will get a len == 3 array, that's not accurate right. The second MULTI should get a error ERR MULTI calls can not be nested and the transaction can EXEC though.

EDIT: maybe we should find the nearest successful MULTI command, but not the nearest MULTI command, if we want to find the whole transaction willing to retry ?

rueian · 2024-12-23T03:25:33Z

The second MULTI should get a error ERR MULTI calls can not be nested and the transaction can EXEC though.

I just tried the double MULTI case but got an EXECABORT when doing EXEC. I think we shouldn't retry the transaction if the nearest MULTI command fails with a Redis error.

wyxloading · 2024-12-23T03:44:20Z

I think we shouldn't retry the transaction if the nearest MULTI command fails with a Redis error.

make sense though.

BTW i got different response from redis:7.4 cluster

cmd:

MULTI
MULTI
PTTL foo
EXEC

response

OK
ERR MULTI calls can not be nested
QUEUED
1) (integer) -2

EDIT:
If we sent the second MULTI command after PTTL foo, EXEC with same result.
If key foo belong to slot in state migrating, it will get ASK error, seems like it will fit in the code that find out a transaction and retry logic

rueian · 2024-12-23T05:28:09Z

cluster.go

+				for mi = i; mi >= 0 && !isMulti(commands[mi]) && !isExec(commands[mi]); mi-- {
+				}
+				for ei = i; ei < len(commands) && !isMulti(commands[ei]) && !isExec(commands[ei]); ei++ {
+				}


Both mi and ei cursor will stop at either MULTI or EXEC to avoid crossing tx boundaries.

rueian · 2024-12-23T05:48:10Z

I think we shouldn't retry the transaction if the nearest MULTI command fails with a Redis error.

make sense though.

BTW i got different response from redis:7.4 cluster

cmd:
MULTI
MULTI
PTTL foo
EXEC
response
OK
ERR MULTI calls can not be nested
QUEUED
1) (integer) -2
EDIT: If we sent the second MULTI command after PTTL foo, EXEC with same result. If key foo belong to slot in state migrating, it will get ASK error, seems like it will fit in the code that find out a transaction and retry logic

I tested these cases on Valkey 8, and they all failed with EXECABORT. But anyway, I think we should just not retry on these wired cases.

wyxloading · 2024-12-23T12:44:34Z

LGTM mostly, but i have to test this PR again tomorrow

Signed-off-by: Rueian <[email protected]>

rueian · 2024-12-23T15:48:39Z

cluster.go

+				}
+				for ei = i; ei < len(commands) && !isMulti(commands[ei]) && !isExec(commands[ei]); ei++ {
+				}
+				if mi >= 0 && mi < ei && ei < len(commands) && isMulti(commands[mi]) && isExec(commands[ei]) && resps[mi].val.string == ok { // a transaction is found.


If the nearest MULTI didn't succeed, we don't retry the tx.

wyxloading · 2024-12-23T12:43:03Z

cluster.go

+				}
+				for ei = i; ei < len(commands) && !isMulti(commands[ei]) && !isExec(commands[ei]); ei++ {
+				}
+				if mi >= 0 && mi < ei && ei < len(commands) && isMulti(commands[mi]) && isExec(commands[ei]) && resps[mi].val.string == "QUEUED" { // a transaction is found.


resps[mi].val.string should be OK or just verify there is no error ?

Yes, it should be OK.

wyxloading · 2024-12-24T06:22:01Z

cluster.go

+					continue // the transaction has been added to the retries, go to the next cmd.
+				}
+			}
+			if hasInit && (mi < i && i < ei && mi >= 0 && isMulti(commands[mi]) || i > ei && ei >= 0 && isMulti(commands[ei])) {


isMulti(commands[ei]) maybe a typo ? should be isExec(commands[ei])

It is not a typo. If commands[ei] is a MULTI then we should not retry the command on i. However, I found that it is actually a dead code, i will never be larger than ei at this point, so the condition is removed now.

wyxloading

LGTM 👍

wyxloading · 2024-12-24T08:27:32Z

IMO we should publish a release for it.

Signed-off-by: Rueian <[email protected]>

rueian · 2024-12-24T10:37:22Z

IMO we should publish a release for it.

Yeap, but we just forget handling ASKING correctly. I have made a new commit for it.

rueian mentioned this pull request Dec 22, 2024

DoXCache invocation will block until slot state change from migrating to stable / moved #703

Closed

rueian commented Dec 22, 2024

View reviewed changes

rueian force-pushed the fix-cluster-tx-retry branch 3 times, most recently from dbe3db5 to 6bfd890 Compare December 23, 2024 02:26

rueian changed the title ~~fix: broken tx retris for cluster clients after #697~~ fix: broken tx retries for cluster clients after #697 Dec 23, 2024

rueian force-pushed the fix-cluster-tx-retry branch from 6bfd890 to f61ee99 Compare December 23, 2024 05:25

rueian commented Dec 23, 2024

View reviewed changes

rueian force-pushed the fix-cluster-tx-retry branch from f61ee99 to b4c543d Compare December 23, 2024 05:47

fix: broken tx retries for cluster clients after #697

fbdca04

Signed-off-by: Rueian <[email protected]>

rueian force-pushed the fix-cluster-tx-retry branch from b4c543d to fbdca04 Compare December 23, 2024 15:48

rueian commented Dec 23, 2024

View reviewed changes

wyxloading reviewed Dec 24, 2024

View reviewed changes

rueian force-pushed the fix-cluster-tx-retry branch from 443734d to aafed64 Compare December 24, 2024 06:29

rueian marked this pull request as ready for review December 24, 2024 07:01

rueian assigned wyxloading Dec 24, 2024

rueian requested a review from wyxloading December 24, 2024 07:49

wyxloading approved these changes Dec 24, 2024

View reviewed changes

fix: broken tx retris for cluster clients after #697

e5cfe35

Signed-off-by: Rueian <[email protected]>

rueian force-pushed the fix-cluster-tx-retry branch from aafed64 to e5cfe35 Compare December 24, 2024 08:37

fix: broken tx retris for cluster clients after #697

060a937

Signed-off-by: Rueian <[email protected]>

rueian force-pushed the fix-cluster-tx-retry branch from 9792175 to 060a937 Compare December 24, 2024 09:41

rueian merged commit 99daad7 into main Dec 24, 2024
60 checks passed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

fix: broken tx retries for cluster clients after #697 #709

fix: broken tx retries for cluster clients after #697 #709

rueian commented Dec 22, 2024

rueian Dec 22, 2024

rueian Dec 22, 2024

wyxloading commented Dec 23, 2024 •

edited

Loading

rueian commented Dec 23, 2024

wyxloading commented Dec 23, 2024 •

edited

Loading

rueian Dec 23, 2024

rueian commented Dec 23, 2024

wyxloading commented Dec 23, 2024

rueian Dec 23, 2024

wyxloading Dec 23, 2024

rueian Dec 24, 2024

wyxloading Dec 24, 2024

rueian Dec 24, 2024

wyxloading left a comment

wyxloading commented Dec 24, 2024

rueian commented Dec 24, 2024

fix: broken tx retries for cluster clients after #697 #709

fix: broken tx retries for cluster clients after #697 #709

Conversation

rueian commented Dec 22, 2024

Choose a reason for hiding this comment

Choose a reason for hiding this comment

wyxloading commented Dec 23, 2024 • edited Loading

rueian commented Dec 23, 2024

wyxloading commented Dec 23, 2024 • edited Loading

Choose a reason for hiding this comment

rueian commented Dec 23, 2024

wyxloading commented Dec 23, 2024

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

wyxloading left a comment

Choose a reason for hiding this comment

wyxloading commented Dec 24, 2024

rueian commented Dec 24, 2024

wyxloading commented Dec 23, 2024 •

edited

Loading

wyxloading commented Dec 23, 2024 •

edited

Loading