Skip to content

MINOR: Prohibit setting StreamsConfig commit.interval.ms to a negative value - #5809

Merged
mjsax merged 4 commits into
apache:trunkfrom
occho:streams-prohibit-negative-commit-interval
Oct 23, 2018
Merged

MINOR: Prohibit setting StreamsConfig commit.interval.ms to a negative value#5809
mjsax merged 4 commits into
apache:trunkfrom
occho:streams-prohibit-negative-commit-interval

Conversation

@occho

@occho occho commented Oct 17, 2018

Copy link
Copy Markdown
Contributor

Prohibit setting StreamsConfig commit.interval.ms to a negative value to avoid possible ambiguity of what it indicates.

So far, setting the property to a negative value can be used to turn off periodic offset-commit by StreamThread. With this change, that will no longer work.
Instead of using a negative value, LONG.MAX_VALUE can be used to achieve the almost same thing.

Committer Checklist (excluded from commit message)

  • Verify design and implementation
  • Verify test coverage and CI build status
  • Verify documentation (including upgrade notes)

@occho

occho commented Oct 17, 2018

Copy link
Copy Markdown
Contributor Author

We might be able to remove the check interval >= 0, but I'm not sure whether we can assume their value always come from StreamsConfig. So, I did not update the code.

if (commitTimeMs >= 0 && now - lastCommitMs > commitTimeMs) {

if (flushInterval >= 0 && now >= lastFlush + flushInterval) {

@occho

occho commented Oct 17, 2018

Copy link
Copy Markdown
Contributor Author

@guozhangwang Could you review this?

@guozhangwang guozhangwang left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM. This is from a discussion on the accepted range of this config, cc @bbejeck @vvcephei @mjsax for another review.

Although it changes the public config's range, I think this is really a "bug" that we should fix rather than changing the semantics, so we can skip the KIP process.

@mjsax

mjsax commented Oct 18, 2018

Copy link
Copy Markdown
Member

It's a judgement call. I am fine with the change. We recently updated KafkaStreams#close() similarly.

@occho I think it's save the remove the check >= 0 in both classes. Can you update the code accordingly?

@occho

occho commented Oct 18, 2018

Copy link
Copy Markdown
Contributor Author

@occho I think it's save the remove the check >= 0 in both classes. Can you update the code accordingly?

Sure. Thank you for your advice.
Updated.

@bbejeck bbejeck left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Overall looks good to me. I think we should have some unit tests demonstrating the fix i.e an Exception results when trying to set commit.interval.ms to a negative value

@occho

occho commented Oct 19, 2018

Copy link
Copy Markdown
Contributor Author

Added a test.

@occho

occho commented Oct 19, 2018

Copy link
Copy Markdown
Contributor Author

Ah, wait, I need more fix.

@occho

occho commented Oct 19, 2018

Copy link
Copy Markdown
Contributor Author

Updated and rebased onto the latest trunk.

@bbejeck bbejeck left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks @occho just one very minor nit, otherwise LGTM

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit: maybe keep this test but pass in 0 instead?

@occho occho Oct 21, 2018

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

From my understanding, the test name is wrong and should be like shouldNotFlushWhenFlushIntervalIsNegative.
The code that shouldNotFlushWhenFlushIntervalIsZero tests is about this line:

if (flushInterval >= 0 && now >= lastFlush + flushInterval) {

With the code on trunk, when flushInterval is set to negative, stateMaintainer.flushState() is not called.
But, since this spec is removed, I think it is okay to remove the test.

WDYT?

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ack, thanks for the explanation. Sounds good to me.

@bbejeck

bbejeck commented Oct 21, 2018

Copy link
Copy Markdown
Member

retest this please

@mjsax mjsax left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM.

Thanks for the PR @occho!

@mjsax
mjsax merged commit e9bbfde into apache:trunk Oct 23, 2018
@occho

occho commented Oct 23, 2018

Copy link
Copy Markdown
Contributor Author

Thank you for reviewing. :)

@occho
occho deleted the streams-prohibit-negative-commit-interval branch November 12, 2018 07:19
pengxiaolong pushed a commit to pengxiaolong/kafka that referenced this pull request Jun 14, 2019
…e value (apache#5809)

Reviewers: Guozhang Wang <guozhang@confluent.io>, Matthias J. Sax <matthias@confluent.io>, Bill Bejeck <bill@confluent.io>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants