Skip to content

Conversation

@MaxGekk
Copy link
Member

@MaxGekk MaxGekk commented Jun 22, 2020

What changes were proposed in this pull request?

Call the replace() method from UTF8String to remove the GMT string from the input of DateTimeUtils.cleanLegacyTimestampStr. It removes all GMT substrings.

Why are the changes needed?

Simpler code improves maintainability

Does this PR introduce any user-facing change?

Should not

How was this patch tested?

By existing test suites JsonSuite and UnivocityParserSuite.

@SparkQA
Copy link

SparkQA commented Jun 22, 2020

Test build #124358 has finished for PR 28892 at commit c1e4d0c.

  • This patch passes all tests.
  • This patch merges cleanly.
  • This patch adds no public classes.

@SparkQA
Copy link

SparkQA commented Jun 22, 2020

Test build #124359 has finished for PR 28892 at commit bcea391.

  • This patch passes all tests.
  • This patch merges cleanly.
  • This patch adds no public classes.

@MaxGekk
Copy link
Member Author

MaxGekk commented Jun 22, 2020

@srowen @dongjoon-hyun @cloud-fan @HyukjinKwon Could you review the little refactoring.

Copy link
Contributor

@Fokko Fokko left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Love it

}
}
// the "GMT" string. For example, it returns 2000-01-01T00:00+01:00 for 2000-01-01T00:00GMT+01:00.
def cleanLegacyTimestampStr(s: UTF8String): UTF8String = s.replace(gmtUtf8, UTF8String.EMPTY_UTF8)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

doesn't the java.lang.String have the replace method?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It has but look at how it is implemented via regexp. @JoshRosen implemented more effective replace in UTF8String #24707. That's why I took it. I hope it seems reasonable.

@cloud-fan
Copy link
Contributor

thanks, merging to master!

@cloud-fan cloud-fan closed this in 979a8eb Jun 23, 2020
Copy link
Member

@HyukjinKwon HyukjinKwon left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Late LGTM too

@MaxGekk MaxGekk deleted the simplify-cleanLegacyTimestampStr branch December 11, 2020 20:26
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants