Skip to content

MINOR: Replace null with an actual value for timestamp field in InsertField SMT unit tests#8649

Merged
kkonstantine merged 2 commits into
apache:trunkfrom
wicknicks:insertfield-ts-ut
May 12, 2020
Merged

MINOR: Replace null with an actual value for timestamp field in InsertField SMT unit tests#8649
kkonstantine merged 2 commits into
apache:trunkfrom
wicknicks:insertfield-ts-ut

Conversation

@wicknicks

@wicknicks wicknicks commented May 12, 2020

Copy link
Copy Markdown
Contributor

Signed-off-by: Arjun Satish arjun@confluent.io

PR adds a unit test to check the behavior of the InsertField SMT when trying to insert a field that takes its value from the Kafka record timestamp. All other tests currently pass a null value for the record timestamp.

Committer Checklist (excluded from commit message)

  • Verify design and implementation
  • Verify test coverage and CI build status
  • Verify documentation (including upgrade notes)

Signed-off-by: Arjun Satish <arjun@confluent.io>
@wicknicks

Copy link
Copy Markdown
Contributor Author

@rhauch @kkonstantine @C0urante small addition here. reviews appreciated. thanks in advance!

@C0urante

C0urante commented May 12, 2020

Copy link
Copy Markdown
Contributor

The code looks good; I'm just a little curious about why we'd want to do a separate test here instead of altering the existing tests to all (or nearly all) use actual timestamp values instead of null?

@wicknicks

Copy link
Copy Markdown
Contributor Author

The other tests seemed to be testing for the empty timestamp case. Hence the new test, also helps keep the new test precise to a usecase. I'd prefer to just add another case for the schema case, if that is the main concern that made you thin that way, @C0urante. WDYT?

@C0urante

C0urante commented May 12, 2020

Copy link
Copy Markdown
Contributor

I think changing the empty timestamp cases to use a real timestamp instead might be fine here. The ? or ! suffix logic is the same for the timestamp field as it is with the partition, offset, topic, and static fields.
If we want to explicitly test for the optionality of a field based on that suffix that might call for a separate test on its own, but it doesn't seem like there's anything special about the logic or the code for the timestamp field that would necessitate an entirely separate test for it.

Signed-off-by: Arjun Satish <arjun@confluent.io>
@wicknicks

Copy link
Copy Markdown
Contributor Author

@C0urante updated the test. it should cover both schema and schemaeless cases in existing tests. let me know if that works.

@C0urante C0urante left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM, thanks Arjun!

@kkonstantine

Copy link
Copy Markdown
Contributor

@wicknicks I understand that the current tests were taking a shortcut by not filling out a the timestamp field with an actual value. But other than observing this omission, is there another motivation behind this change that we'd need to take into account and include in the description of this PR?

@wicknicks

Copy link
Copy Markdown
Contributor Author

@kkonstantine a Connector was not setting the field correctly in a local setup, and I was digging into why this is the case. to check if the SMT is correct, I looked at the tests, and they were all checking nulls, as you pointed out. Hence, the update, so we have clarity on what the expected input is, and what the output should be.

@kkonstantine kkonstantine left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sounds good. Thanks for improving these tests.

LGTM

@kkonstantine

Copy link
Copy Markdown
Contributor

jdk14 green, jdk 11 failed on 1 flaky test, jdk 8 failed to run, but that's ok.

@kkonstantine kkonstantine changed the title MINOR: Add a unit test for timestamp field in InsertField SMT MINOR: Replace null with an actual value for timestamp field in InsertField SMT unit tests May 12, 2020
@kkonstantine
kkonstantine merged commit 9bc96d5 into apache:trunk May 12, 2020
@wicknicks

Copy link
Copy Markdown
Contributor Author

Thanks, @kkonstantine and @C0urante!

Kvicii added a commit to Kvicii/kafka that referenced this pull request May 14, 2020
* 'trunk' of github.com:apache/kafka:
  MINOR: add option to rebuild source for system tests (apache#6656)
  KAFKA-9850 Move KStream#repartition operator validation during Topolo… (apache#8550)
  MINOR: Add a duplicate() method to Message classes (apache#8556)
  KAFKA-9966: add internal assignment listener to stabilize eos-beta upgrade test (apache#8648)
  MINOR: Replace null with an actual value for timestamp field in InsertField SMT unit tests (apache#8649)
  MINOR: Fix ProcessorContext JavaDocs and stream-time computation (apache#8603)
  MINOR: improve tests for TopologyTestDriver (apache#8631)
  KAFKA-9821: consolidate Streams rebalance triggering mechanisms (apache#8596)
  KAFKA-9669; Loosen validation of inner offsets for older message formats (apache#8647)
  KAFKA-8770: KIP-557: Drop idempotent KTable source updates (apache#8254)
  MINOR: Remove allow concurrent test (apache#8641)
jwijgerd pushed a commit to buxapp/kafka that referenced this pull request May 14, 2020
…tField SMT unit tests (apache#8649)

Add actual values instead of just passing null in unit tests that check the behavior of the InsertField SMT when trying to insert a field that takes its value from the Kafka record timestamp.

Reviewers: Chris Egerton <chrise@confluent.io>, Konstantine Karantasis <konstantine@confluent.io>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants