Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat(ddl): support alter table cast tuple type column #17310

Merged
merged 7 commits into from
Jan 20, 2025

Conversation

b41sh
Copy link
Member

@b41sh b41sh commented Jan 16, 2025

I hereby agree to the terms of the CLA available at: https://docs.databend.com/dev/policies/cla/

Summary

Add a new function arrays_zip to build Array(Tuple) for multiple Array arguments, for example:

root@0.0.0.0:48000/default> SELECT arrays_zip([1,2,3], ['a','b','c'], 10);

SELECT
  arrays_zip([1,2,3], ['a','b','c'], 10)

┌─────────────────────────────────────────────┐
│ arrays_zip([1, 2, 3], ['a', 'b', 'c'], 10) │
│      Array(Tuple(UInt8, String, UInt8))     │
├─────────────────────────────────────────────┤
│ [(1,'a',10),(2,'b',10),(3,'c',10)]          │
└─────────────────────────────────────────────┘
1 row read in 0.096 sec. Processed 1 row, 1 B (10.42 rows/s, 10 B/s)

Alter table modify column allow convert Tuple and Array(Tuple) with different leaf column numbers. As Tuple type can't auto cast to different Tuple type, we read tuple leaf columns and build a new tuple with tuple and arrays_zip function.

for example:

root@0.0.0.0:48000/default> create table t1(
  a int,
  b Array(Tuple(Float64 NULL, Float64 NULL, String NULL) NULL) NULL
);

root@0.0.0.0:48000/default> insert into t1 values(1, null),(2, [(1,2,'x'),(3,4,'y')]),(3, [(5,null,null),(6,null,'z')]);

root@0.0.0.0:48000/default> select * from t1;
┌────────────────────────────────────────────────────────────────────────────────────────────────────────────┐
│        a        │                                             b                                            │
│ Nullable(Int32) │ Nullable(Array(Nullable(Tuple(Nullable(Float64), Nullable(Float64), Nullable(String))))) │
├─────────────────┼──────────────────────────────────────────────────────────────────────────────────────────┤
│               1NULL                                                                                     │
│               2 │ [(1,2,'x'),(3,4,'y')]                                                                    │
│               3 │ [(5,NULL,NULL),(6,NULL,'z')]                                                             │
└────────────────────────────────────────────────────────────────────────────────────────────────────────────┘
3 rows read in 0.040 sec. Processed 3 rows, 162 B (75 rows/s, 3.96 KiB/s)

root@0.0.0.0:48000/default> alter table t1 MODIFY COLUMN b Array(Tuple(Float64 NULL, Float64 NULL, String NULL, String NULL) NULL) NULL;

root@0.0.0.0:48000/default> select * from t1;

┌──────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────┐
│        a        │                                                      b                                                     │
│ Nullable(Int32) │ Nullable(Array(Nullable(Tuple(Nullable(Float64), Nullable(Float64), Nullable(String), Nullable(String))))) │
├─────────────────┼────────────────────────────────────────────────────────────────────────────────────────────────────────────┤
│               1NULL                                                                                                       │
│               2 │ [(1,2,'x',NULL),(3,4,'y',NULL)]                                                                            │
│               3 │ [(5,NULL,NULL,NULL),(6,NULL,'z',NULL)]                                                                     │
└──────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────┘
3 rows read in 0.047 sec. Processed 3 rows, 211 B (63.83 rows/s, 4.38 KiB/s)

in this example, we use the following SQL to read values from the orginal table and insert the values into the modified table

SELECT a, if(is_not_null(b), arrays_zip(array_transform(b, x -> x.1), array_transform(b, x -> x.2), array_transform(b, x -> x.3), NULL), NULL) FROM t1;

array_transform function convert the Array Tuple inner column to an Array, and if the original inner column not exist, use the default value instead. arrays_zip function use those arguments to construct a new Array Tuple column

  • fixes: #[Link the issue here]

Tests

  • Unit Test
  • Logic Test
  • Benchmark Test
  • No Test - Explain why

Type of change

  • Bug Fix (non-breaking change which fixes an issue)
  • New Feature (non-breaking change which adds functionality)
  • Breaking Change (fix or feature that could cause existing functionality not to work as expected)
  • Documentation Update
  • Refactoring
  • Performance Improvement
  • Other (please describe):

This change is Reviewable

@b41sh b41sh requested review from sundy-li and dantengsky January 16, 2025 15:28
@github-actions github-actions bot added the pr-feature this PR introduces a new feature to the codebase label Jan 16, 2025
@sundy-li
Copy link
Member

sundy-li commented Jan 17, 2025

Better rename it into array_zip https://docs.databricks.com/en/sql/language-manual/functions/arrays_zip.html

@sundy-li sundy-li enabled auto-merge January 20, 2025 03:17
@sundy-li sundy-li added this pull request to the merge queue Jan 20, 2025
@github-merge-queue github-merge-queue bot removed this pull request from the merge queue due to failed status checks Jan 20, 2025
@b41sh b41sh added this pull request to the merge queue Jan 20, 2025
@BohuTANG BohuTANG removed this pull request from the merge queue due to a manual request Jan 20, 2025
@BohuTANG BohuTANG merged commit 7720353 into databendlabs:main Jan 20, 2025
70 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
pr-feature this PR introduces a new feature to the codebase
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants