Upgrade Delta Standalone to 3.0.0

[dnskr]

This PR upgrades Delta Standalone dependency to 3.0.0 for Delta Lake Connector.

Test plan - Existing tests

== RELEASE NOTES ==

Delta Lake Changes
* Upgrade Delta Standalone to 3.0.0

[agrawalreetika]

Hi @dnskr,
Wanted to confirm does this upgrade also fix _last_checkpoint file check-related exception which was mentioned here - #18898

[dnskr]

Hi @dnskr, Wanted to confirm does this upgrade also fix _last_checkpoint file check-related exception which was mentioned here - #18898

I don't think so. I'm not sure I can check it, because unfortunately I'm quite limited in Presto development and debugging by Windows machine I use.

[tdcmeehan]

@dnskr have you done any end to end tests on this upgrade?

[agrawalreetika]

Hi @dnskr, Wanted to confirm does this upgrade also fix _last_checkpoint file check-related exception which was mentioned here - #18898

@dnskr, I tried query on this upgrade. Looks like _last_checkpoint file check exception is not handled here as well. Just curious, do we know what Delta reader side improvement would get on Presto side with this upgrade?

[dnskr]

@tdcmeehan Unfortunately, I cannot build Presto or run tests locally, because I use Windows. So, I rely on passing CI/CD pipelines and existing delta tests to check that the upgrade is fine.

@agrawalreetika There is at least one performance improvement in Delta Standalone according to release notes:

The key features in this release are:

• Support for disabling Delta checkpointing during commits - For very large tables with millions of files, performing Delta checkpoints can become an expensive overhead during writes. Users can now disable this checkpointing by setting the hadoop configuration property io.delta.standalone.checkpointing.enabled to false. This is only safe and suggested to do if another job will periodically perform the checkpointing.
• Performance improvement to snapshot initialization - When a delta table is loaded at a particular version, the snapshot must contain, at a minimum, the latest protocol and metadata. This PR improves the snapshot load performance for repeated table changes.
• Support adding absolute paths to the Delta log - This now enables users to manually perform SHALLOW CLONEs and create Delta tables with external files.
• Fix in schema evolution to prevent adding non-nullable columns to existing Delta tables

@agrawalreetika @tdcmeehan I would like to be transparent about the amount of effort from my side for the changes.
I upgraded Delta Standalone just to keep the dependency up to date, without expecting any significant improvements.
During the preliminary upgrade to 3.0.0rc1 I found and reported a minor bug, which is fixed in 3.0.0.
I would be happy if someone could take the opportunity to manage Delta Lake upgrades in more careful way.

BTW, as far as I know, next Delta Standalone 3.1.0 release will be based on Delta Kernel, which might require more effort for upgrade, but also can provide more capabilities for Delta Lake connector in Presto.

[dnskr]

Close in favor of #22596