[Feature] Support assign sort key of primary key model by schema change reorder. #13642

Linkerist · 2022-11-18T01:52:43Z

What type of PR is this：

Which issues of this PR fixes ：

Problem Summary(Required) ：

do not support assign sort key of primary key model by schema change reorder now:

MySQL [test]> CREATE TABLE test (
    ->   `k1` int,
    ->   `k2` int,
    ->   `k3` int,
    ->   `v1` int,
    ->   `v2` int,
    ->   `v3` int,
    ->   `v4` int,
    ->   `v5` int
    -> ) ENGINE=OLAP
    -> PRIMARY KEY(`k1`, `k2`, `k3`)
    -> COMMENT "OLAP"
    -> DISTRIBUTED BY HASH(`k1`, `k2`, `k3`) BUCKETS 1
    -> ORDER BY(`v2`, `v3`)
    -> PROPERTIES ("replication_num" = "1");
Query OK, 0 rows affected (0.11 sec)

MySQL [test]> ALTER TABLE test ORDER BY (v3, v2);
Primary key table do not support reorder column

should be:

MySQL [test]> select v2, v3 from test;
+------+------+
| v2   | v3   |
+------+------+
|    1 |    5 |
|    2 |    4 |
|    3 |    3 |
|    4 |    2 |
|    5 |    1 |
+------+------+
5 rows in set (0.02 sec)

MySQL [test]> show create table test;
+-------+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
| Table | Create Table                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                |
+-------+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
| test  | CREATE TABLE `test` (
  `k1` int(11) NOT NULL COMMENT "",
  `k2` int(11) NOT NULL COMMENT "",
  `k3` int(11) NOT NULL COMMENT "",
  `v1` int(11) NULL COMMENT "",
  `v2` int(11) NULL COMMENT "",
  `v3` int(11) NULL COMMENT "",
  `v4` int(11) NULL COMMENT "",
  `v5` int(11) NULL COMMENT ""
) ENGINE=OLAP 
PRIMARY KEY(`k1`, `k2`, `k3`)
COMMENT "OLAP"
DISTRIBUTED BY HASH(`k1`, `k2`, `k3`) BUCKETS 1 
ORDER BY(`v2`, `v3`)
PROPERTIES (
"replication_num" = "1",
"in_memory" = "false",
"storage_format" = "DEFAULT",
"enable_persistent_index" = "false",
"compression" = "LZ4"
); |
+-------+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
1 row in set (0.01 sec)

MySQL [test]> ALTER TABLE test ORDER BY (v3, v2);
Query OK, 0 rows affected (0.03 sec)

MySQL [test]> select v3, v2 from test;
+------+------+
| v3   | v2   |
+------+------+
|    1 |    5 |
|    2 |    4 |
|    3 |    3 |
|    4 |    2 |
|    5 |    1 |
+------+------+
5 rows in set (0.01 sec)

MySQL [test]> show create table test;
+-------+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
| Table | Create Table                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                |
+-------+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
| test  | CREATE TABLE `test` (
  `k1` int(11) NOT NULL COMMENT "",
  `k2` int(11) NOT NULL COMMENT "",
  `k3` int(11) NOT NULL COMMENT "",
  `v1` int(11) NULL COMMENT "",
  `v2` int(11) NULL COMMENT "",
  `v3` int(11) NULL COMMENT "",
  `v4` int(11) NULL COMMENT "",
  `v5` int(11) NULL COMMENT ""
) ENGINE=OLAP 
PRIMARY KEY(`k1`, `k2`, `k3`)
COMMENT "OLAP"
DISTRIBUTED BY HASH(`k1`, `k2`, `k3`) BUCKETS 1 
ORDER BY(`v3`, `v2`)
PROPERTIES (
"replication_num" = "1",
"in_memory" = "false",
"storage_format" = "DEFAULT",
"enable_persistent_index" = "false",
"compression" = "LZ4"
); |
+-------+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
1 row in set (0.01 sec)

Checklist:

I have added test cases for my bug fix or my new feature
This pr will affect users' behaviors
This pr needs user documentation (for new or modified features or behaviors)
- I have added documentation for my new feature or new function

github-actions · 2022-11-18T02:13:50Z

clang-tidy review says "All clean, LGTM! 👍"

be/src/bench/CMakeLists.txt

be/src/storage/rowset/rowset_writer.cpp

github-actions · 2022-11-21T04:11:01Z

clang-tidy review says "All clean, LGTM! 👍"

github-actions · 2022-11-21T11:13:30Z

clang-tidy review says "All clean, LGTM! 👍"

be/src/storage/rowset/rowset_writer.cpp

be/test/exprs/vectorized/math_functions_test.cpp

decster · 2022-11-23T08:47:56Z

fe/fe-core/src/main/java/com/starrocks/alter/SchemaChangeHandler.java

-                    throw new DdlException("Primary key table do not support reorder column");
+                    List<Integer> sortKeyIdxes = new ArrayList<>();
+                    processReorderColumnOfPrimaryKey((ReorderColumnsClause) alterClause, olapTable, indexSchemaMap, sortKeyIdxes);
+                    return createJobForProcessReorderColumnOfPrimaryKey(db.getId(), olapTable,
+                            indexSchemaMap, sortKeyIdxes);


what if a user made a mistake and use the old usage of alter table order by, for example:

create table table0 (a,b,c,d,e) primary key(a), order by(c,d) alter table order by (d, c, b, c, e)

Could we provide a notice of potential misusage if the user provides all the columns in the table in alter table order by for primary key table?

old usage 1. need all columns. 2. all keys need be the prefix of all columns. 3. column can not be duplicate appear.
if write all columns, we can not tell the differences between make all columns to be sort key on purpose and making a mistake to use the old usage.
maybe the furthest the judgement can do is provide a notice when satisfy 1, 2, since that in the new usage 2 is needless.

…ema change reorder.

…d more strict NotSupport logic branch.

Linkerist · 2022-12-02T11:59:55Z

run starrocks_be_unittest

github-actions · 2022-12-02T12:53:50Z

clang-tidy review says "All clean, LGTM! 👍"

sonarcloud · 2022-12-07T14:45:41Z

Kudos, SonarCloud Quality Gate passed!

0 Bugs
0 Vulnerabilities
0 Security Hotspots
1 Code Smell

0.0% Coverage
1.3% Duplication

wanpengfei-git · 2022-12-07T14:56:18Z

[FE PR Coverage Check]

😞 fail : 13 / 43 (30.23%)

file detail

	path	covered_line	new_line	coverage	not_covered_line_detail
🔵	com/starrocks/alter/SchemaChangeHandler.java	9	35	25.71%	[504, 505, 507, 508, 509, 510, 513, 514, 518, 520, 1044, 1045, 1049, 1052, 1053, 1054, 1055, 1056, 1057, 1059, 1060, 1061, 1063, 1064, 1065, 1212]
🔵	com/starrocks/alter/SchemaChangeJobV2.java	2	6	33.33%	[201, 314, 315, 316]
🔵	com/starrocks/alter/AlterJobV2Builder.java	2	2	100.00%	[]

github-actions · 2022-12-07T15:44:11Z

clang-tidy review says "All clean, LGTM! 👍"

wanpengfei-git · 2022-12-08T03:21:18Z

run starrocks_admit_test

sonarcloud · 2022-12-08T04:51:10Z

Kudos, SonarCloud Quality Gate passed!

0 Bugs
0 Vulnerabilities
0 Security Hotspots
0 Code Smells

No Coverage information
2.2% Duplication

github-actions · 2022-12-08T04:54:18Z

clang-tidy review says "All clean, LGTM! 👍"

… key model by schema change reorder. (StarRocks#13642) `` MySQL [test]> CREATE TABLE test ( -> `k1` int, -> `k2` int, -> `k3` int, -> `v1` int, -> `v2` int, -> `v3` int, -> `v4` int, -> `v5` int -> ) ENGINE=OLAP -> PRIMARY KEY(`k1`, `k2`, `k3`) -> COMMENT "OLAP" -> DISTRIBUTED BY HASH(`k1`, `k2`, `k3`) BUCKETS 1 -> ORDER BY(`v2`, `v3`) -> PROPERTIES ("replication_num" = "1"); Query OK, 0 rows affected (0.11 sec) MySQL [test]> ALTER TABLE test ORDER BY (v3, v2); Primary key table do not support reorder column ``` should be: ``` MySQL [test]> select v2, v3 from test; +------+------+ | v2 | v3 | +------+------+ | 1 | 5 | | 2 | 4 | | 3 | 3 | | 4 | 2 | | 5 | 1 | +------+------+ 5 rows in set (0.02 sec) MySQL [test]> show create table test; +-------+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ | Table | Create Table | +-------+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ | test | CREATE TABLE `test` ( `k1` int(11) NOT NULL COMMENT "", `k2` int(11) NOT NULL COMMENT "", `k3` int(11) NOT NULL COMMENT "", `v1` int(11) NULL COMMENT "", `v2` int(11) NULL COMMENT "", `v3` int(11) NULL COMMENT "", `v4` int(11) NULL COMMENT "", `v5` int(11) NULL COMMENT "" ) ENGINE=OLAP PRIMARY KEY(`k1`, `k2`, `k3`) COMMENT "OLAP" DISTRIBUTED BY HASH(`k1`, `k2`, `k3`) BUCKETS 1 ORDER BY(`v2`, `v3`) PROPERTIES ( "replication_num" = "1", "in_memory" = "false", "storage_format" = "DEFAULT", "enable_persistent_index" = "false", "compression" = "LZ4" ); | +-------+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ 1 row in set (0.01 sec) MySQL [test]> ALTER TABLE test ORDER BY (v3, v2); Query OK, 0 rows affected (0.03 sec) MySQL [test]> select v3, v2 from test; +------+------+ | v3 | v2 | +------+------+ | 1 | 5 | | 2 | 4 | | 3 | 3 | | 4 | 2 | | 5 | 1 | +------+------+ 5 rows in set (0.01 sec) MySQL [test]> show create table test; +-------+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ | Table | Create Table | +-------+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ | test | CREATE TABLE `test` ( `k1` int(11) NOT NULL COMMENT "", `k2` int(11) NOT NULL COMMENT "", `k3` int(11) NOT NULL COMMENT "", `v1` int(11) NULL COMMENT "", `v2` int(11) NULL COMMENT "", `v3` int(11) NULL COMMENT "", `v4` int(11) NULL COMMENT "", `v5` int(11) NULL COMMENT "" ) ENGINE=OLAP PRIMARY KEY(`k1`, `k2`, `k3`) COMMENT "OLAP" DISTRIBUTED BY HASH(`k1`, `k2`, `k3`) BUCKETS 1 ORDER BY(`v3`, `v2`) PROPERTIES ( "replication_num" = "1", "in_memory" = "false", "storage_format" = "DEFAULT", "enable_persistent_index" = "false", "compression" = "LZ4" ); | +-------+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ 1 row in set (0.01 sec) ```

…ge reorder. (#13642) (#14930) * [cherry-pick][branch-2.5][Feature] Support assign sort key of primary key model by schema change reorder. (#13642) `` MySQL [test]> CREATE TABLE test ( -> `k1` int, -> `k2` int, -> `k3` int, -> `v1` int, -> `v2` int, -> `v3` int, -> `v4` int, -> `v5` int -> ) ENGINE=OLAP -> PRIMARY KEY(`k1`, `k2`, `k3`) -> COMMENT "OLAP" -> DISTRIBUTED BY HASH(`k1`, `k2`, `k3`) BUCKETS 1 -> ORDER BY(`v2`, `v3`) -> PROPERTIES ("replication_num" = "1"); Query OK, 0 rows affected (0.11 sec) MySQL [test]> ALTER TABLE test ORDER BY (v3, v2); Primary key table do not support reorder column ``` should be: ``` MySQL [test]> select v2, v3 from test; +------+------+ | v2 | v3 | +------+------+ | 1 | 5 | | 2 | 4 | | 3 | 3 | | 4 | 2 | | 5 | 1 | +------+------+ 5 rows in set (0.02 sec) MySQL [test]> show create table test; +-------+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ | Table | Create Table | +-------+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ | test | CREATE TABLE `test` ( `k1` int(11) NOT NULL COMMENT "", `k2` int(11) NOT NULL COMMENT "", `k3` int(11) NOT NULL COMMENT "", `v1` int(11) NULL COMMENT "", `v2` int(11) NULL COMMENT "", `v3` int(11) NULL COMMENT "", `v4` int(11) NULL COMMENT "", `v5` int(11) NULL COMMENT "" ) ENGINE=OLAP PRIMARY KEY(`k1`, `k2`, `k3`) COMMENT "OLAP" DISTRIBUTED BY HASH(`k1`, `k2`, `k3`) BUCKETS 1 ORDER BY(`v2`, `v3`) PROPERTIES ( "replication_num" = "1", "in_memory" = "false", "storage_format" = "DEFAULT", "enable_persistent_index" = "false", "compression" = "LZ4" ); | +-------+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ 1 row in set (0.01 sec) MySQL [test]> ALTER TABLE test ORDER BY (v3, v2); Query OK, 0 rows affected (0.03 sec) MySQL [test]> select v3, v2 from test; +------+------+ | v3 | v2 | +------+------+ | 1 | 5 | | 2 | 4 | | 3 | 3 | | 4 | 2 | | 5 | 1 | +------+------+ 5 rows in set (0.01 sec) MySQL [test]> show create table test; +-------+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ | Table | Create Table | +-------+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ | test | CREATE TABLE `test` ( `k1` int(11) NOT NULL COMMENT "", `k2` int(11) NOT NULL COMMENT "", `k3` int(11) NOT NULL COMMENT "", `v1` int(11) NULL COMMENT "", `v2` int(11) NULL COMMENT "", `v3` int(11) NULL COMMENT "", `v4` int(11) NULL COMMENT "", `v5` int(11) NULL COMMENT "" ) ENGINE=OLAP PRIMARY KEY(`k1`, `k2`, `k3`) COMMENT "OLAP" DISTRIBUTED BY HASH(`k1`, `k2`, `k3`) BUCKETS 1 ORDER BY(`v3`, `v2`) PROPERTIES ( "replication_num" = "1", "in_memory" = "false", "storage_format" = "DEFAULT", "enable_persistent_index" = "false", "compression" = "LZ4" ); | +-------+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ 1 row in set (0.01 sec) ``` * resolve conflict.

…ge reorder. (#13642) `` MySQL [test]> CREATE TABLE test ( -> `k1` int, -> `k2` int, -> `k3` int, -> `v1` int, -> `v2` int, -> `v3` int, -> `v4` int, -> `v5` int -> ) ENGINE=OLAP -> PRIMARY KEY(`k1`, `k2`, `k3`) -> COMMENT "OLAP" -> DISTRIBUTED BY HASH(`k1`, `k2`, `k3`) BUCKETS 1 -> ORDER BY(`v2`, `v3`) -> PROPERTIES ("replication_num" = "1"); Query OK, 0 rows affected (0.11 sec) MySQL [test]> ALTER TABLE test ORDER BY (v3, v2); Primary key table do not support reorder column ``` should be: ``` MySQL [test]> select v2, v3 from test; +------+------+ | v2 | v3 | +------+------+ | 1 | 5 | | 2 | 4 | | 3 | 3 | | 4 | 2 | | 5 | 1 | +------+------+ 5 rows in set (0.02 sec) MySQL [test]> show create table test; +-------+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ | Table | Create Table | +-------+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ | test | CREATE TABLE `test` ( `k1` int(11) NOT NULL COMMENT "", `k2` int(11) NOT NULL COMMENT "", `k3` int(11) NOT NULL COMMENT "", `v1` int(11) NULL COMMENT "", `v2` int(11) NULL COMMENT "", `v3` int(11) NULL COMMENT "", `v4` int(11) NULL COMMENT "", `v5` int(11) NULL COMMENT "" ) ENGINE=OLAP PRIMARY KEY(`k1`, `k2`, `k3`) COMMENT "OLAP" DISTRIBUTED BY HASH(`k1`, `k2`, `k3`) BUCKETS 1 ORDER BY(`v2`, `v3`) PROPERTIES ( "replication_num" = "1", "in_memory" = "false", "storage_format" = "DEFAULT", "enable_persistent_index" = "false", "compression" = "LZ4" ); | +-------+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ 1 row in set (0.01 sec) MySQL [test]> ALTER TABLE test ORDER BY (v3, v2); Query OK, 0 rows affected (0.03 sec) MySQL [test]> select v3, v2 from test; +------+------+ | v3 | v2 | +------+------+ | 1 | 5 | | 2 | 4 | | 3 | 3 | | 4 | 2 | | 5 | 1 | +------+------+ 5 rows in set (0.01 sec) MySQL [test]> show create table test; +-------+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ | Table | Create Table | +-------+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ | test | CREATE TABLE `test` ( `k1` int(11) NOT NULL COMMENT "", `k2` int(11) NOT NULL COMMENT "", `k3` int(11) NOT NULL COMMENT "", `v1` int(11) NULL COMMENT "", `v2` int(11) NULL COMMENT "", `v3` int(11) NULL COMMENT "", `v4` int(11) NULL COMMENT "", `v5` int(11) NULL COMMENT "" ) ENGINE=OLAP PRIMARY KEY(`k1`, `k2`, `k3`) COMMENT "OLAP" DISTRIBUTED BY HASH(`k1`, `k2`, `k3`) BUCKETS 1 ORDER BY(`v3`, `v2`) PROPERTIES ( "replication_num" = "1", "in_memory" = "false", "storage_format" = "DEFAULT", "enable_persistent_index" = "false", "compression" = "LZ4" ); | +-------+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ 1 row in set (0.01 sec) ``` (cherry picked from commit d25e71b) # Conflicts: # be/src/storage/tablet_updates.cpp # be/test/exprs/vectorized/math_functions_test.cpp # be/test/storage/tablet_updates_test.cpp

mergify · 2022-12-13T06:22:30Z

backport branch-2.5

✅ Backports have been created

#15140 [Feature] Support assign sort key of primary key model by schema change reorder. (backport #13642) has been created for branch branch-2.5

…e with separate primary key and sort key (#20035) If we create a primary key table with a separate primary key and sort key, BE maybe crash if we do a schema change in the table(add column/drop column) The reason is that we use column id as the sort key index in the tablet, and we only support reordering the sort key in this pr(#13642) which assumes the number of columns in the origin tablet and new tablet during schema change is the same. However, if we add/drop a column, the sort key index in the new tablet will be different from the origin tablet and we will execute reorder which may cause BE crash. Signed-off-by: zhangqiang <[email protected]>

…e with separate primary key and sort key (#20035) If we create a primary key table with a separate primary key and sort key, BE maybe crash if we do a schema change in the table(add column/drop column) The reason is that we use column id as the sort key index in the tablet, and we only support reordering the sort key in this pr(#13642) which assumes the number of columns in the origin tablet and new tablet during schema change is the same. However, if we add/drop a column, the sort key index in the new tablet will be different from the origin tablet and we will execute reorder which may cause BE crash. Signed-off-by: zhangqiang <[email protected]> (cherry picked from commit 82f9d31)

…e with separate primary key and sort key (StarRocks#20035) If we create a primary key table with a separate primary key and sort key, BE maybe crash if we do a schema change in the table(add column/drop column) The reason is that we use column id as the sort key index in the tablet, and we only support reordering the sort key in this pr(StarRocks#13642) which assumes the number of columns in the origin tablet and new tablet during schema change is the same. However, if we add/drop a column, the sort key index in the new tablet will be different from the origin tablet and we will execute reorder which may cause BE crash. Signed-off-by: zhangqiang <[email protected]>

wanpengfei-git added the be-build label Nov 18, 2022

chaoyli reviewed Nov 19, 2022

View reviewed changes

be/src/bench/CMakeLists.txt Outdated Show resolved Hide resolved

be/src/storage/rowset/rowset_writer.cpp Show resolved Hide resolved

github-actions bot removed the be-build label Nov 21, 2022

wanpengfei-git added the be-build label Nov 21, 2022

github-actions bot removed the be-build label Nov 21, 2022

wanpengfei-git added the be-build label Nov 21, 2022

decster reviewed Nov 23, 2022

View reviewed changes

github-actions bot removed the be-build label Nov 29, 2022

Linkerist force-pushed the feature_support_schema_change_reorder_primary_key_model_sort_key branch from bdda272 to fd21ee4 Compare December 2, 2022 04:07

Linkerist added 5 commits December 2, 2022 15:03

[Feature] Support assign sort key of primary key model by reorder sch…

5b899f7

…ema change reorder.

uncomment bench binary.

f488bbe

add some UT.

91cbe80

restore the irrelevant test file formatted by clang-format.sh, and ad…

3b93671

…d more strict NotSupport logic branch.

change some code due to main branch rebase compile error.

0f89d07

Linkerist force-pushed the feature_support_schema_change_reorder_primary_key_model_sort_key branch from fd21ee4 to 0f89d07 Compare December 2, 2022 08:05

wanpengfei-git added the be-build label Dec 2, 2022

decster previously approved these changes Dec 5, 2022

View reviewed changes

setIndexMeta should add sort key idxes.

1436696

Linkerist dismissed decster’s stale review via 1436696 December 7, 2022 14:36

github-actions bot removed the be-build label Dec 7, 2022

wanpengfei-git added the be-build label Dec 7, 2022

decster approved these changes Dec 8, 2022

View reviewed changes

chaoyli approved these changes Dec 8, 2022

View reviewed changes

chaoyli enabled auto-merge (squash) December 8, 2022 03:20

wanpengfei-git added the Approved Ready to merge label Dec 8, 2022

chaoyli merged commit d25e71b into StarRocks:main Dec 8, 2022

github-actions bot removed Approved Ready to merge be-build labels Dec 8, 2022

wanpengfei-git added the be-build label Dec 8, 2022

Linkerist deleted the feature_support_schema_change_reorder_primary_key_model_sort_key branch December 8, 2022 05:15

mergify bot mentioned this pull request Dec 13, 2022

[Feature] Support assign sort key of primary key model by schema change reorder. (backport #13642) #15140

Closed

sevev mentioned this pull request Mar 22, 2023

[BugFix] Be maybe crash when we do schema change for primary key table with separate primary key and sort key #20035

Merged

16 tasks

sevev mentioned this pull request Dec 7, 2023

[BugFix][Cherry-pick][Branch-2.5] Be maybe crash when we do schema change for primary key table with separate primary key and sort key (backport #20035) #36636

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[Feature] Support assign sort key of primary key model by schema change reorder. #13642

[Feature] Support assign sort key of primary key model by schema change reorder. #13642

Linkerist commented Nov 18, 2022 •

edited by chaoyli

Loading

github-actions bot commented Nov 18, 2022

github-actions bot commented Nov 21, 2022

github-actions bot commented Nov 21, 2022

decster Nov 23, 2022

Linkerist Nov 29, 2022

Linkerist commented Dec 2, 2022

github-actions bot commented Dec 2, 2022

sonarcloud bot commented Dec 7, 2022

wanpengfei-git commented Dec 7, 2022

github-actions bot commented Dec 7, 2022

wanpengfei-git commented Dec 8, 2022

sonarcloud bot commented Dec 8, 2022

github-actions bot commented Dec 8, 2022

mergify bot commented Dec 13, 2022

[Feature] Support assign sort key of primary key model by schema change reorder. #13642

[Feature] Support assign sort key of primary key model by schema change reorder. #13642

Conversation

Linkerist commented Nov 18, 2022 • edited by chaoyli Loading

What type of PR is this：

Which issues of this PR fixes ：

Problem Summary(Required) ：

Checklist:

github-actions bot commented Nov 18, 2022

github-actions bot commented Nov 21, 2022

github-actions bot commented Nov 21, 2022

decster Nov 23, 2022

Choose a reason for hiding this comment

Linkerist Nov 29, 2022

Choose a reason for hiding this comment

Linkerist commented Dec 2, 2022

github-actions bot commented Dec 2, 2022

sonarcloud bot commented Dec 7, 2022

wanpengfei-git commented Dec 7, 2022

[FE PR Coverage Check]

file detail

github-actions bot commented Dec 7, 2022

wanpengfei-git commented Dec 8, 2022

sonarcloud bot commented Dec 8, 2022

github-actions bot commented Dec 8, 2022

mergify bot commented Dec 13, 2022

✅ Backports have been created

Linkerist commented Nov 18, 2022 •

edited by chaoyli

Loading