Skip to content

Conversation

@ebyhr
Copy link
Member

@ebyhr ebyhr commented Oct 15, 2025

Description

Databricks 11.3 LTS is going to reach EOL on Oct 19. https://docs.databricks.com/aws/en/release-notes/runtime/

Release notes

(x) This is not user-visible or is docs only, and no release notes are required.

Summary by Sourcery

Remove end-of-life Databricks 11.3 LTS from live product tests and CI, clean up obsolete version checks, and add lightweight plugin unit support for reading Databricks 11.3 data

Enhancements:

  • Drop DatabricksVersion constant and conditional branches for runtime version 11.3 in product tests
  • Omit all suite definitions, CI jobs, and test groups targeting Databricks 11.3 LTS
  • Add a static plugin resource and a TestDeltaLakeBasic unit test to verify reading region data from Databricks 11.3 LTS

CI:

  • Remove suite-delta-lake-databricks113 job and related environment variables from GitHub Actions CI

@cla-bot cla-bot bot added the cla-signed label Oct 15, 2025
@github-actions github-actions bot added the delta-lake Delta Lake connector label Oct 15, 2025
Copy link

@sourcery-ai sourcery-ai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hey there - I've reviewed your changes - here's some feedback:

  • The new TestDeltaLakeBasic test for databricks113 is no longer included in any CI suite (suite-delta-lake-databricks113 was removed), so it will never run—please re-enable or reassign it to an active suite.
  • After consolidating away the 11.3 version checks, verify that all test group annotations still map to existing suites so no tests are inadvertently skipped.
  • Consider removing or documenting the remaining databricks113 test resources if they are no longer needed now that 11.3 support is dropped.
Prompt for AI Agents
Please address the comments from this code review:

## Overall Comments
- The new TestDeltaLakeBasic test for databricks113 is no longer included in any CI suite (suite-delta-lake-databricks113 was removed), so it will never run—please re-enable or reassign it to an active suite.
- After consolidating away the 11.3 version checks, verify that all test group annotations still map to existing suites so no tests are inadvertently skipped.
- Consider removing or documenting the remaining databricks113 test resources if they are no longer needed now that 11.3 support is dropped.

Sourcery is free for open source - if you like our reviews please consider sharing them ✨
Help me be more useful! Please click 👍 or 👎 on each comment and I'll use the feedback to improve your reviews.

@trinodb trinodb deleted a comment from sourcery-ai bot Oct 15, 2025
Data generated using Databricks 11.3:

```sql
CREATE TABLE default.region (regionkey bigint, name string, comment string)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

do we still needs those files?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes because tables created by Databricks 11.3 should be readable even after the EOL.

@ebyhr ebyhr requested review from findepi and wendigo October 15, 2025 04:25
@ebyhr ebyhr merged commit 57b16da into master Oct 15, 2025
103 checks passed
@ebyhr ebyhr deleted the ebi/databricks-113 branch October 15, 2025 09:28
@github-actions github-actions bot added this to the 478 milestone Oct 15, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

cla-signed delta-lake Delta Lake connector

Development

Successfully merging this pull request may close these issues.

3 participants