Conversation
Review ChecklistHello reviewers! 👋 Please follow this checklist when reviewing this Pull Request. General
Tests
Documentation
New flags
If a workflow is added or modified:
Backward compatibility
|
Codecov ReportAttention: Patch coverage is
Additional details and impacted files@@ Coverage Diff @@
## main #17826 +/- ##
==========================================
- Coverage 67.45% 67.42% -0.03%
==========================================
Files 1592 1592
Lines 258167 258554 +387
==========================================
+ Hits 174143 174338 +195
- Misses 84024 84216 +192 ☔ View full report in Codecov by Sentry. |
766be25 to
39089b4
Compare
This is currently missing and leads to incorrect types to be returned for `LEAST` & `GREATEST` as comparison functions. There's a little mismatch here in behavior compared to MySQL which I argue is actually a bug in MySQL. In MySQL, a temporal type always has the binary collation: ``` mysql> select NOW(6), collation(NOW(6)); +----------------------------+-------------------+ | NOW(6) | collation(NOW(6)) | +----------------------------+-------------------+ | 2025-02-19 15:33:21.732301 | binary | +----------------------------+-------------------+ 1 row in set (0.00 sec) ``` On MySQL 8.4, this results in: ``` mysql> select GREATEST(NOW(6), NOW(6)), collation(GREATEST(NOW(6), NOW(6))); +----------------------------+-------------------------------------+ | GREATEST(NOW(6), NOW(6)) | collation(GREATEST(NOW(6), NOW(6))) | +----------------------------+-------------------------------------+ | 2025-02-19 15:35:00.921308 | latin1_swedish_ci | +----------------------------+-------------------------------------+ 1 row in set (0.00 sec) ``` But on MySQL 8.0, it returns: ``` mysql> select GREATEST(NOW(6), NOW(6)), collation(GREATEST(NOW(6), NOW(6))); +----------------------------+-------------------------------------+ | GREATEST(NOW(6), NOW(6)) | collation(GREATEST(NOW(6), NOW(6))) | +----------------------------+-------------------------------------+ | 2025-02-19 15:35:00.921308 | utf8mb4_0900_ai_ci | +----------------------------+-------------------------------------+ 1 row in set (0.00 sec) ``` Neither of these collations make sense, because it really should not change the collation and return `binary` still. That is what Vitess still does with the changes here (hence the addition to the test framework to allow skipping the collation check). I'll also report the issue upstream to make it behave correctly there as well. Signed-off-by: Dirkjan Bussink <d.bussink@gmail.com>
39089b4 to
a693845
Compare
| env := evalengine.EmptyExpressionEnv(venv) | ||
|
|
||
| tc.Run(func(query string, row []sqltypes.Value) { | ||
| tc.Run(func(query string, row []sqltypes.Value, skipCollationCheck bool) { |
There was a problem hiding this comment.
shouldn't we use skipCollationCheck here?
There was a problem hiding this comment.
@systay This currently doesn't do collations checks anyway, so I don't think it really matters.
There was a problem hiding this comment.
@systay This did remind me that we actually should validate the collation. We still ignore this flag though, since our evalengine and compiler should always match.
The ignore collation flag is for where explicitly don't 100% match MySQL (due to bugs, arguably broken behavior etc).
Signed-off-by: Dirkjan Bussink <d.bussink@gmail.com>
| "vitess.io/vitess/go/mysql/collations" | ||
| "vitess.io/vitess/go/mysql/collations/charset" | ||
| "vitess.io/vitess/go/mysql/collations/colldata" | ||
| datetime2 "vitess.io/vitess/go/mysql/datetime" |
There was a problem hiding this comment.
nitpick: another solution would be to use the variable name datetimes in the functions, and keep import named datetime
Signed-off-by: Dirkjan Bussink <d.bussink@gmail.com>
| } | ||
|
|
||
| func compareRemoteExprEnv(t *testing.T, collationEnv *collations.Environment, env *evalengine.ExpressionEnv, conn *mysql.Conn, expr string, fields []*querypb.Field, cmp *testcases.Comparison) { | ||
| func compareRemoteExprEnv(t *testing.T, collationEnv *collations.Environment, env *evalengine.ExpressionEnv, conn *mysql.Conn, expr string, fields []*querypb.Field, cmp *testcases.Comparison, skipCollationCheck bool) { |
There was a problem hiding this comment.
skipCollationCheck is needed because of the issue in MySQL described in mysql/mysql-server#602.
Signed-off-by: Dirkjan Bussink <d.bussink@gmail.com>
[release-19.0] Implement temporal comparisons (vitessio#17826) (vitessio#17852)
* [release-21.0] Bump to `v21.0.4-SNAPSHOT` after the `v21.0.3` release (vitessio#17766) Signed-off-by: Rohit Nayak <rohit@planetscale.com> Co-authored-by: Rohit Nayak <rohit@planetscale.com> * [release-21.0] smartconnpool: Better handling for idle expiration (vitessio#17757) (vitessio#17781) Signed-off-by: Vicent Marti <vmg@strn.cat> Co-authored-by: vitess-bot[bot] <108069721+vitess-bot[bot]@users.noreply.github.com> * [release-21.0] Fail assignment expressions with the correct message (vitessio#17752) (vitessio#17776) Signed-off-by: Manan Gupta <manan@planetscale.com> Co-authored-by: Manan Gupta <35839558+GuptaManan100@users.noreply.github.com> Co-authored-by: Manan Gupta <manan@planetscale.com> * [release-21.0] Multi-tenant workflow SwitchWrites: Don't add denied tables on cancelMigration() (vitessio#17782) (vitessio#17797) Signed-off-by: Rohit Nayak <rohit@planetscale.com> Co-authored-by: vitess-bot[bot] <108069721+vitess-bot[bot]@users.noreply.github.com> * [release-21.0] VReplication Atomic Copy Workflows: fix bugs around concurrent inserts (vitessio#17772) (vitessio#17793) Signed-off-by: Rohit Nayak <rohit@planetscale.com> Co-authored-by: vitess-bot[bot] <108069721+vitess-bot[bot]@users.noreply.github.com> * [release-21.0] Upgrade the Golang version to `go1.23.6` (vitessio#17699) Signed-off-by: GitHub <noreply@github.com> Signed-off-by: Florent Poinsard <florent.poinsard@outlook.fr> Co-authored-by: frouioui <frouioui@users.noreply.github.com> Co-authored-by: Florent Poinsard <florent.poinsard@outlook.fr> * [release-21.0] Fix a potential connection pool leak. (vitessio#17807) (vitessio#17814) Signed-off-by: Arthur Schreiber <arthurschreiber@github.com> Co-authored-by: vitess-bot[bot] <108069721+vitess-bot[bot]@users.noreply.github.com> * backport: support subqueries inside subqueries when merging (Release 21.0) (vitessio#17811) Signed-off-by: Andres Taylor <andres@planetscale.com> * [release-21.0] Fix vtcombo parsing flags incorrectly (vitessio#17743) (vitessio#17820) Signed-off-by: Dirkjan Bussink <d.bussink@gmail.com> Co-authored-by: Dirkjan Bussink <d.bussink@gmail.com> * [release-21.0] pool: reopen connection closed by idle timeout (vitessio#17818) (vitessio#17829) * [release-21.0] Implement temporal comparisons (vitessio#17826) (vitessio#17854) Signed-off-by: Dirkjan Bussink <d.bussink@gmail.com> Co-authored-by: vitess-bot[bot] <108069721+vitess-bot[bot]@users.noreply.github.com> * [release-21.0] evalengine: normalize types during compilation (vitessio#17887) (vitessio#17896) Signed-off-by: Vicent Marti <vmg@strn.cat> Signed-off-by: Andres Taylor <andres@planetscale.com> Co-authored-by: vitess-bot[bot] <108069721+vitess-bot[bot]@users.noreply.github.com> Co-authored-by: Vicent Marti <vmg@strn.cat> * [release-21.0] Fix: Separate Lock for Keyspace to Update Controller Mapping in Schema Tracking (vitessio#17873) (vitessio#17885) Signed-off-by: Harshit Gangal <harshit@planetscale.com> Co-authored-by: vitess-bot[bot] <108069721+vitess-bot[bot]@users.noreply.github.com> Co-authored-by: Harshit Gangal <harshit@planetscale.com> * [release-21.0] Upgrade the Golang version to `go1.23.7` (vitessio#17901) Signed-off-by: GitHub <noreply@github.com> Signed-off-by: Florent Poinsard <florent.poinsard@outlook.fr> Co-authored-by: frouioui <frouioui@users.noreply.github.com> Co-authored-by: Florent Poinsard <florent.poinsard@outlook.fr> * [release-21.0] fix: race on storing schema engine last changed time (vitessio#17914) (vitessio#17917) Signed-off-by: Harshit Gangal <harshit@planetscale.com> Signed-off-by: Shlomi Noach <2607934+shlomi-noach@users.noreply.github.com> Co-authored-by: Harshit Gangal <harshit@planetscale.com> Co-authored-by: Shlomi Noach <2607934+shlomi-noach@users.noreply.github.com> * [release-21.0] [VTAdmin] Insert into schema cache if exists already and not expired (vitessio#17908) (vitessio#17924) Signed-off-by: Frances Thai <notfelineit@gmail.com> Co-authored-by: vitess-bot[bot] <108069721+vitess-bot[bot]@users.noreply.github.com> * [release-21.0] Bump golang.org/x/net from 0.34.0 to 0.36.0 (vitessio#17958) (vitessio#17960) Signed-off-by: Dirkjan Bussink <d.bussink@gmail.com> Co-authored-by: vitess-bot[bot] <108069721+vitess-bot[bot]@users.noreply.github.com> * [release-21.0] fix flaky test on mysqlshell backup engine (vitessio#17981) Signed-off-by: Renan Rangel <rrangel@slack-corp.com> Signed-off-by: Florent Poinsard <florent.poinsard@outlook.fr> Co-authored-by: Renan Rangel <rvrangel@users.noreply.github.com> * [release-21.0] DML test fix for duplicate column value (vitessio#17980) Signed-off-by: Harshit Gangal <harshit@planetscale.com> * [release-21.0] Fix DISTINCT on ENUM/SET columns by making enums/set hashable (vitessio#17936) (vitessio#17991) Signed-off-by: Gene Parmesan Thomas <201852096+gopoto@users.noreply.github.com> Co-authored-by: vitess-bot[bot] <108069721+vitess-bot[bot]@users.noreply.github.com> * [release-21.0] Fix tablet selection in `vtctld BackupShard` (vitessio#18002) (vitessio#18025) Signed-off-by: Florent Poinsard <florent.poinsard@outlook.fr> Co-authored-by: vitess-bot[bot] <108069721+vitess-bot[bot]@users.noreply.github.com> * [release-21.0] Set proper join vars type for the RHS field query in OLAP (vitessio#18028) (vitessio#18038) Signed-off-by: Florent Poinsard <florent.poinsard@outlook.fr> Signed-off-by: Andres Taylor <andres@planetscale.com> Co-authored-by: vitess-bot[bot] <108069721+vitess-bot[bot]@users.noreply.github.com> Co-authored-by: Andres Taylor <andres@planetscale.com> * [release-21.0] Use release branches for upgrade downgrade tests (vitessio#18029) (vitessio#18035) Signed-off-by: Manan Gupta <manan@planetscale.com> Co-authored-by: vitess-bot[bot] <108069721+vitess-bot[bot]@users.noreply.github.com> * [release-21.0] Test: Increase query timeout to fix flaky test 'TestQueryTimeoutWithShardTargeting' (vitessio#18016) (vitessio#18040) Signed-off-by: Harshit Gangal <harshit@planetscale.com> Co-authored-by: vitess-bot[bot] <108069721+vitess-bot[bot]@users.noreply.github.com> * [release-21.0] fix: App and Dba Pool metrics (vitessio#18048) (vitessio#18084) Signed-off-by: Harshit Gangal <harshit@planetscale.com> Co-authored-by: vitess-bot[bot] <108069721+vitess-bot[bot]@users.noreply.github.com> * [release-21.0] Upgrade the Golang version to `go1.23.8` (vitessio#18092) Signed-off-by: GitHub <noreply@github.com> Signed-off-by: Florent Poinsard <florent.poinsard@outlook.fr> Co-authored-by: frouioui <frouioui@users.noreply.github.com> Co-authored-by: Florent Poinsard <florent.poinsard@outlook.fr> * [release-21.0] bugfix: allow window functions when possible to push down (vitessio#18103) (vitessio#18105) Signed-off-by: Andres Taylor <andres@planetscale.com> Co-authored-by: vitess-bot[bot] <108069721+vitess-bot[bot]@users.noreply.github.com> Co-authored-by: Andres Taylor <andres@planetscale.com> * [release-21.0] VDiff: Fix logic for reconciling extra rows (vitessio#17950) (vitessio#18072) Signed-off-by: Rohit Nayak <rohit@planetscale.com> Co-authored-by: vitess-bot[bot] <108069721+vitess-bot[bot]@users.noreply.github.com> * [release-21.0] VStream API: Reset stopPos in catchup (vitessio#18119) (vitessio#18122) Signed-off-by: Noble Mittal <noblemittal@outlook.com> Co-authored-by: vitess-bot[bot] <108069721+vitess-bot[bot]@users.noreply.github.com> * [release-21.0] Fix `Reshard Cancel` behavior (vitessio#18020) (vitessio#18080) Signed-off-by: Arthur Schreiber <arthurschreiber@github.com> Co-authored-by: Arthur Schreiber <arthurschreiber@github.com> * Fix backup shard copy paste error (vitessio#18100) Signed-off-by: Dirkjan Bussink <d.bussink@gmail.com> * [release-21.0] go/vt/vtgate: take routing rules into account for traffic mirroring (vitessio#17953) (vitessio#17994) Signed-off-by: Max Englander <max@planetscale.com> Co-authored-by: vitess-bot[bot] <108069721+vitess-bot[bot]@users.noreply.github.com> Co-authored-by: Max Englander <max@planetscale.com> * [release-21.0] Bugfix: Missing data when running vtgate outer joins (vitessio#18036) (vitessio#18044) Signed-off-by: Andres Taylor <andres@planetscale.com> Co-authored-by: vitess-bot[bot] <108069721+vitess-bot[bot]@users.noreply.github.com> * [release-21.0] Filter out tablets with unknown replication lag when electing a new primary (vitessio#18004) (vitessio#18075) Signed-off-by: Florent Poinsard <florent.poinsard@outlook.fr> Co-authored-by: Florent Poinsard <35779988+frouioui@users.noreply.github.com> * [release-21.0] Fix: Ensure Consistent Lookup Vindex Handles Duplicate Rows in Single Query (vitessio#17974) (vitessio#18078) Signed-off-by: Harshit Gangal <harshit@planetscale.com> Co-authored-by: Harshit Gangal <harshit@planetscale.com> * [release-21.0] Code Freeze for `v21.0.4` (vitessio#18135) Signed-off-by: Florent Poinsard <florent.poinsard@outlook.fr> * [release-21.0] Release of `v21.0.4` (vitessio#18136) Signed-off-by: Florent Poinsard <florent.poinsard@outlook.fr> * test fix Signed-off-by: Tim Vaillancourt <tim@timvaillancourt.com> * Revert "test fix" This reverts commit 55aec5c. --------- Signed-off-by: Rohit Nayak <rohit@planetscale.com> Signed-off-by: Vicent Marti <vmg@strn.cat> Signed-off-by: Manan Gupta <manan@planetscale.com> Signed-off-by: GitHub <noreply@github.com> Signed-off-by: Florent Poinsard <florent.poinsard@outlook.fr> Signed-off-by: Arthur Schreiber <arthurschreiber@github.com> Signed-off-by: Andres Taylor <andres@planetscale.com> Signed-off-by: Dirkjan Bussink <d.bussink@gmail.com> Signed-off-by: Harshit Gangal <harshit@planetscale.com> Signed-off-by: Shlomi Noach <2607934+shlomi-noach@users.noreply.github.com> Signed-off-by: Frances Thai <notfelineit@gmail.com> Signed-off-by: Renan Rangel <rrangel@slack-corp.com> Signed-off-by: Gene Parmesan Thomas <201852096+gopoto@users.noreply.github.com> Signed-off-by: Noble Mittal <noblemittal@outlook.com> Signed-off-by: Max Englander <max@planetscale.com> Signed-off-by: Tim Vaillancourt <tim@timvaillancourt.com> Co-authored-by: vitess-bot <139342327+vitess-bot@users.noreply.github.com> Co-authored-by: Rohit Nayak <rohit@planetscale.com> Co-authored-by: vitess-bot[bot] <108069721+vitess-bot[bot]@users.noreply.github.com> Co-authored-by: Manan Gupta <35839558+GuptaManan100@users.noreply.github.com> Co-authored-by: Manan Gupta <manan@planetscale.com> Co-authored-by: frouioui <frouioui@users.noreply.github.com> Co-authored-by: Florent Poinsard <florent.poinsard@outlook.fr> Co-authored-by: Andrés Taylor <andres@planetscale.com> Co-authored-by: Matt Lord <mattalord@gmail.com> Co-authored-by: Dirkjan Bussink <d.bussink@gmail.com> Co-authored-by: Harshit Gangal <harshit@planetscale.com> Co-authored-by: Vicent Marti <vmg@strn.cat> Co-authored-by: Shlomi Noach <2607934+shlomi-noach@users.noreply.github.com> Co-authored-by: Florent Poinsard <35779988+frouioui@users.noreply.github.com> Co-authored-by: Renan Rangel <rvrangel@users.noreply.github.com> Co-authored-by: Arthur Schreiber <arthurschreiber@github.com> Co-authored-by: Max Englander <max@planetscale.com>
…sio#17852) Signed-off-by: Dirkjan Bussink <d.bussink@gmail.com> Co-authored-by: vitess-bot[bot] <108069721+vitess-bot[bot]@users.noreply.github.com>
…ons (#768) * [release-19.0] Implement temporal comparisons (vitessio#17826) (vitessio#17852) Signed-off-by: Dirkjan Bussink <d.bussink@gmail.com> Co-authored-by: vitess-bot[bot] <108069721+vitess-bot[bot]@users.noreply.github.com> * [release-19.0] Fix week number for date_format evalengine function (vitessio#17432) (vitessio#17454) Signed-off-by: Dirkjan Bussink <d.bussink@gmail.com> Signed-off-by: Manan Gupta <manan@planetscale.com> Co-authored-by: Manan Gupta <35839558+GuptaManan100@users.noreply.github.com> Co-authored-by: Manan Gupta <manan@planetscale.com> --------- Signed-off-by: Dirkjan Bussink <d.bussink@gmail.com> Signed-off-by: Manan Gupta <manan@planetscale.com> Co-authored-by: vitess-bot[bot] <108069721+vitess-bot[bot]@users.noreply.github.com> Co-authored-by: Dirkjan Bussink <d.bussink@gmail.com> Co-authored-by: Manan Gupta <35839558+GuptaManan100@users.noreply.github.com> Co-authored-by: Manan Gupta <manan@planetscale.com>
This is currently missing and leads to incorrect types to be returned for
LEAST&GREATESTas comparison functions.There's a little mismatch here in behavior compared to MySQL which I argue is actually a bug in MySQL. In MySQL, a temporal type always has the binary collation:
On MySQL 8.4, this results in:
But on MySQL 8.0, it returns:
Neither of these collations make sense, because it really should not change the collation and return
binarystill. That is what Vitess still does with the changes here (hence the addition to the test framework to allow skipping the collation check).I'll also report the issue upstream to make it behave correctly there as well.
Marked this also for backporting since using these types today leads to a panic on queries.
Related Issue(s)
Fixes #17750
Checklist