Release DataFusion 39.0.0

[alamb]

Is your feature request related to a problem or challenge?

Tracking ticket for next release, also a place to track desired inclusions

Last release was https://crates.io/crates/datafusion/38.0.0 (May 10, 2024) so next major release would be around June 10, 2024

List if items that would be good to get into this release:

• [Regression] Query using ARRAY_AGG(DISTINCT) causes panic #10486
• Support build right with HashJoin in DataFusion #9603
• Upgrade to Arrow 52 Release arrow-rs / parquet version 52.0.0 arrow-rs#5688
• fix: fix string repeat for negative numbers #10760

Additional context

37.0.0 release ticket: #10217

[andygrove]

@alamb Ideally, I would like to wait until DF upgrades to arrow-rs 52 before we create the next release. Comet is currently using forks of arrow-rs & datafusion because we are waiting on a fix that is going into arrow-rs 52.

[alamb]

I think @tustvold plans to make 52.0.0 next week -- there are no more blockers I am aware of. If he doesn't have time I will make the release candidate myself.

[waynexia]

arrow-rs 52 upgrade PR: #10765, I'll keep track on the (near) latest unreleased HEAD of arrow-rs so we can shift to the new version once it's published.

[andygrove]

I plan on starting the RC process as soon as #10765 is merged (thank you @waynexia for having this ready!)

[andygrove]

PR to prepare 39.0.0-rc1: #10828

[alamb]

Would it be possible to try and include #10802 in this release (so that we don't have to release a subsequent breaking API change next release)? It isn't a huge deal, but it would be nice

[alamb]

Released on june 10: https://crates.io/crates/datafusion/39.0.0

ticket for 40: #11077