Skip to content

Conversation

@alexjo2144
Copy link
Member

@alexjo2144 alexjo2144 commented Jan 19, 2024

Description

Could not find a reason for this behavior change in the Databricks release notes, but the incorrect behavior in the Databricks engine seems to have stopped.

Results from the last build on master

tests               | 2024-01-19 20:03:39 INFO: FAILURE     /    io.trino.tests.product.deltalake.TestDeltaLakeInsertCompatibility.testTimestampInsertCompatibility (Groups: profile_specific_tests, delta-lake-exclude-91, delta-lake-databricks, delta-lake-oss) took 11.5 seconds
tests               | 2024-01-19 20:03:39 SEVERE: Failure cause:
tests               | java.lang.AssertionError: Could not find rows:
tests               | [1, 0001-01-03 00:00:00.000]
tests               | 
tests               | actual rows:
tests               | [1, 0001-01-01 00:00:00.000]
tests               | [2, 2023-01-02 01:02:03.999]
tests               | [3, 2023-03-04 01:02:03.999]
tests               | [4, 9999-12-31 23:59:59.999]

Additional context and related issues

Release notes

(x) This is not user-visible or is docs only, and no release notes are required.
( ) Release notes are required. Please propose a release note for me.
( ) Release notes are required, with the following suggested text:

@alexjo2144
Copy link
Member Author

@ebyhr can I get a run with secrets when you get a chance?

@hashhar
Copy link
Member

hashhar commented Jan 19, 2024

/test-with-secrets 4d46aa6

assertThat(onDelta().executeQuery("SELECT id, date_format(ts, \"yyyy-MM-dd HH:mm:ss.SSS\") FROM default." + tableName))
.containsOnly(
row(1, databricksRuntimeVersion.isPresent() ? "0001-01-03 00:00:00.000" : "0001-01-01 00:00:00.000"), // Databricks returns incorrect results
row(1, expected.orElse("0001-01-01 00:00:00.000")),
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
row(1, expected.orElse("0001-01-01 00:00:00.000")),
row(1, databricksRuntimeVersion
.map(version -> version.isAtLeast(DATABRICKS_133_RUNTIME_VERSION) ? "0001-01-01 00:00:00.000" : "0001-01-03 00:00:00.000")
.orElse("0001-01-01 00:00:00.000")),

@wendigo
Copy link
Contributor

wendigo commented Jan 19, 2024

@findinpath @alexjo2144 is it ok to merge it as it is?

@ebyhr ebyhr merged commit 9b08c11 into trinodb:master Jan 19, 2024
@github-actions github-actions bot added this to the 437 milestone Jan 19, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Development

Successfully merging this pull request may close these issues.

5 participants