-
Notifications
You must be signed in to change notification settings - Fork 2.9k
Docs: Flink can add/drop/modify columns #13617
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Conversation
Per https://issues.apache.org/jira/browse/FLINK-21634, these features are now available. e.g. ```sql Flink SQL> SET 'execution.checkpointing.interval' = '30s'; [INFO] Execute statement succeeded. Flink SQL> CREATE TABLE test01 (col1 INT) > WITH ( > 'connector' = 'iceberg', > 'catalog-name' = 'foo', > 'catalog-database' = 'my_glue_db', > 'warehouse' = 's3://rmoff-lakehouse/00/', > 'catalog-impl' = 'org.apache.iceberg.aws.glue.GlueCatalog', > 'ioImpl' = 'org.apache.iceberg.aws.s3.S3FileIO'); [INFO] Execute statement succeeded. -- Add a column Flink SQL> ALTER TABLE test01 ADD test_col INT; [INFO] Execute statement succeeded. Flink SQL> DESCRIBE test01; +----------+------+------+-----+--------+-----------+ | name | type | null | key | extras | watermark | +----------+------+------+-----+--------+-----------+ | col1 | INT | TRUE | | | | | test_col | INT | TRUE | | | | +----------+------+------+-----+--------+-----------+ 2 rows in set -- Drop a column Flink SQL> ALTER TABLE test01 DROP test_col; [INFO] Execute statement succeeded. Flink SQL> DESCRIBE test01; +------+------+------+-----+--------+-----------+ | name | type | null | key | extras | watermark | +------+------+------+-----+--------+-----------+ | col1 | INT | TRUE | | | | +------+------+------+-----+--------+-----------+ 1 row in set -- Rename a column Flink SQL> ALTER TABLE test01 RENAME test_col TO foo; [INFO] Execute statement succeeded. Flink SQL> DESCRIBE test01; +------+------+------+-----+--------+-----------+ | name | type | null | key | extras | watermark | +------+------+------+-----+--------+-----------+ | col1 | INT | TRUE | | | | | foo | INT | TRUE | | | | +------+------+------+-----+--------+-----------+ 2 rows in set -- Modify a column Flink SQL> ALTER TABLE test01 MODIFY col1 STRING; [INFO] Execute statement succeeded. ```
mxm
left a comment
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Thanks @rmoff for spotting this important change!
docs/docs/flink.md
Outdated
|
|
||
| * Don't support creating iceberg table with hidden partitioning. [Discussion](http://mail-archives.apache.org/mod_mbox/flink-dev/202008.mbox/%3cCABi+2jQCo3MsOa4+ywaxV5J-Z8TGKNZDX-pQLYB-dG+dVUMiMw@mail.gmail.com%3e) in flink mail list. | ||
| * Don't support creating iceberg table with computed column. | ||
| * Don't support creating iceberg table with watermark. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Side note: This is also not true anymore: #12191
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
shall we remove that in this PR too? or open a new one?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
While we're on it, we can simply remove this one as well. What do you think?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
yes, makes sense. So the two remaining features to list as not implemented are:
- Don't support creating iceberg table with hidden partitioning. Discussion in flink mail list.
- Don't support creating iceberg table with computed column.
is that right?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Yes @rmoff, those are the 2 last missing features mentioned here.
Could you please update the doc accordingly?
Thanks,
Peter
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
is that right?
Correct.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Fixed d60a9a8
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@pvary could you review/merge please? thanks.
Per https://issues.apache.org/jira/browse/FLINK-21634, these features are now available.
e.g.