Skip to content

Conversation

xiangyuf
Copy link
Contributor

@xiangyuf xiangyuf commented Oct 13, 2025

What is the purpose of the change

Remove targetColumns from the digest generation of SinkReuser since it is deprecated now.

Brief change log

Remove the targetColumns from the digest generation of SinkReuser.

Verifying this change

This change added tests and can be verified as follows:

  • Added unit test in SinkReuseTestBase#testSinkReuseWithPartialColumnsNotSupportsTargetColumnWriting
  • Added unit test in SinkReuseTestBase#testSinkReuseWithPartialColumnsAndSupportsTargetColumnWriting

Does this pull request potentially affect one of the following parts:

  • Dependencies (does it add or upgrade a dependency): no
  • The public API, i.e., is any changed class annotated with @Public(Evolving): no
  • The serializers: no
  • The runtime per-record code paths (performance sensitive): no
  • Anything that affects deployment or recovery: JobManager (and its components), Checkpointing, Kubernetes/Yarn, ZooKeeper: no
  • The S3 file system connector: no

Documentation

  • Does this pull request introduce a new feature? yes
  • If yes, how is the feature documented? FLIP-506

@flinkbot
Copy link
Collaborator

flinkbot commented Oct 13, 2025

CI report:

Bot commands The @flinkbot bot supports the following commands:
  • @flinkbot run azure re-run the last Azure build

@xiangyuf xiangyuf force-pushed the FLINK-38510 branch 2 times, most recently from 73ff406 to 8cf78a2 Compare October 13, 2025 15:10
@xiangyuf xiangyuf changed the title [FLINK-38510][table-planner] Remove the targetColumns from the digest generation of SinkReuser [FLINK-38510][table-planner] Remove targetColumns from the digest generation of SinkReuser Oct 13, 2025
@xiangyuf
Copy link
Contributor Author

@yunfengzhou-hub @xuyangzhong PTAL when u have time~

@xiangyuf
Copy link
Contributor Author

@flinkbot run azure

1 similar comment
@yunfengzhou-hub
Copy link
Contributor

@flinkbot run azure

@yunfengzhou-hub
Copy link
Contributor

Thanks for the PR. LGTM

Copy link
Contributor

@xuyangzhong xuyangzhong left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for driving this. I have left some comments.

public void testSinkReuseWithPartialColumns() {
public void testSinkReuseWithPartialColumnsNotSupportsTargetColumnWriting() {
StatementSet statementSet = util.tableEnv().createStatementSet();
/// sink1 has not implemented the {@link SupportsTargetColumnWriting} sink ability
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit use // instead of using ///

@Test
public void testSinkReuseWithPartialColumnsAndSupportsTargetColumnWriting() {
StatementSet statementSet = util.tableEnv().createStatementSet();
/// sink2 has implemented the {@link SupportsTargetColumnWriting} sink ability
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ditto

</Resource>
<Resource name="optimized exec plan">
<![CDATA[
Sink(table=[default_catalog.default_database.sink1], targetColumns=[[0]], fields=[x, EXPR$1])
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The plan is a bit strange with the description targetColumns=[[0]] in Sink.

I think we can either:

  1. In the PhysicalSink, check whether SupportsTargetColumnWriting is present before printing targetColumns, and avoid printing targetColumns if it's not.
  2. In the PhysicalSink, reference SupportsTargetColumnWriting when printing targetColumns, regardless of the targetColumns themselves.
  3. Complete the printing of targetColumns based on https://issues.apache.org/jira/browse/FLINK-37708.

As of now, since targetColumns has not been removed from PhysicalSink, I tend to prefer option 1. What do you think, or do you have any other suggestions?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants