Skip to content

MINOR: Fix bug where StoreBuilder added after connecting store with processor - #5737

Closed
bbejeck wants to merge 1 commit into
apache:trunkfrom
bbejeck:MINOR_fix_order_of_adding_stores
Closed

MINOR: Fix bug where StoreBuilder added after connecting store with processor#5737
bbejeck wants to merge 1 commit into
apache:trunkfrom
bbejeck:MINOR_fix_order_of_adding_stores

Conversation

@bbejeck

@bbejeck bbejeck commented Oct 3, 2018

Copy link
Copy Markdown
Member

Fixes bug found in #5731 where we add a state store to the processor before adding the state store to the InternalTopologyBuilder

I've added a test which fails without this fix.

Committer Checklist (excluded from commit message)

  • Verify design and implementation
  • Verify test coverage and CI build status
  • Verify documentation (including upgrade notes)

@bbejeck

bbejeck commented Oct 3, 2018

Copy link
Copy Markdown
Member Author

ping @guozhangwang @mjsax @vvcephei

@mjsax mjsax added the streams label Oct 3, 2018

@mjsax mjsax left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@guozhangwang

Copy link
Copy Markdown
Contributor

@bbejeck My understanding is that, generally:

  1. storeBuilder is for adding the "embedded" store to a stateful processor, like the materialized store for aggregations an joins, and here for suppressions.
  2. storeNames is used for connecting any additional stores to the underlying processors.

And inside addStateStore we will still call the function to connect the storeBuilder.name() with processorName, so we should never have a storeBuilder AND its storeName added into the storeNames anyways.

The unit test you added, is calling on processorNode which is an internal node class, hence users should never be able to call add

new StatefulProcessorNode<>("node", processorParameters, new String[]{"store"}, storeBuilder, false);

And as long as we make sure 1) and 2) are mutual exclusive I think this should be fine?

@bbejeck

bbejeck commented Oct 4, 2018

Copy link
Copy Markdown
Member Author

@guozhangwang

That is correct, as I think about it more as I stated in the slack channel, in practice we'll only get either store names or a store builder passed in. Should I just close this PR?

@guozhangwang

Copy link
Copy Markdown
Contributor

Should I just close this PR?

Yeah, if you agree with my reasoning I think this is not an issue then.

@bbejeck

bbejeck commented Oct 4, 2018

Copy link
Copy Markdown
Member Author

Yes I do agree, closing this PR

@bbejeck bbejeck closed this Oct 4, 2018
guozhangwang pushed a commit to guozhangwang/kafka that referenced this pull request Oct 5, 2018
In recent PRs, we have been confused about the proper usage of
StatefulProcessorNode (apache#5731 , apache#5737 )

This change disambiguates it.

Reviewers: Matthias J. Sax <matthias@confluent.io>, Bill Bejeck <bill@confluent.io>, Guozhang Wang <guozhang@confluent.io>
guozhangwang pushed a commit that referenced this pull request Oct 5, 2018
In recent PRs, we have been confused about the proper usage of
StatefulProcessorNode (#5731 , #5737 )

This change disambiguates it.

Reviewers: Matthias J. Sax <matthias@confluent.io>, Bill Bejeck <bill@confluent.io>, Guozhang Wang <guozhang@confluent.io>
pengxiaolong pushed a commit to pengxiaolong/kafka that referenced this pull request Jun 14, 2019
In recent PRs, we have been confused about the proper usage of
StatefulProcessorNode (apache#5731 , apache#5737 )

This change disambiguates it.

Reviewers: Matthias J. Sax <matthias@confluent.io>, Bill Bejeck <bill@confluent.io>, Guozhang Wang <guozhang@confluent.io>
@bbejeck
bbejeck deleted the MINOR_fix_order_of_adding_stores branch July 10, 2024 13:56
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants