[SPARK-34180][SQL] Revert SPARK-33888#31270
Conversation
…as TimestampType, it should be physical Int in millis" This reverts commit 0b647fe.
|
What do you mean by "start time point"? The TIME type has no date part and should return seconds(with certain precision) of the hour:minute:second fields. |
Yes it has no date part. So in MsSqlServer, the date part is assumed |
|
@cloud-fan |
|
Kubernetes integration test starting |
|
@sarutak can you take a look at https://github.com/apache/spark/pull/30902/files#r561601546 ? I think the test is wrong. |
|
@cloud-fan O.K, I'll close this PR and open another PR to fix the wrong assertion. Thanks. |
|
Kubernetes integration test status success |
What changes were proposed in this pull request?
This PR reverts SPARK-33888 (#30902) .
That PR has the two serious problems.
PostgreSQLIntegrationSuiteto fail.sql.Types.TIMEis mapped tosql.types.IntegerTypeso we'll get millisecond value. But the start time point is different betweenMsSqlServerand other RDBMSs. ForMsSqlServer, it starts from1900-01-01 00:00:00while1970-01-01 00:00:00for other RDBMS.More about problem-2, the following assertion passes before SPARK-33888.
But after SPARK-33888, the following modified assertion fails because the millisecond value is considered starting from
1970-01-01 00:00:00.Why are the changes needed?
To fix the serious issues.
Does this PR introduce any user-facing change?
No.
How was this patch tested?
This is just reverting a previous change.