Blog post with DataFusion Jan - June 2024

[alamb]

Is your feature request related to a problem or challenge?

We have had good luck writing up quarterly updates for DataFusion, most recently:
https://arrow.apache.org/blog/2024/01/19/datafusion-34.0.0/

(see #6780)

Describe the solution you'd like

Write a blog post

Describe alternatives you've considered

No response

Additional context

No response

[alamb]

I am starting to collect a list of things to highlight here

• New trait based APIs
• Pluggable handler for CREATE FUNCTIONL Add plugable handler for CREATE FUNCTION #9333 (thanks @milenkovicm)
• DataFusion Comet blog published: https://arrow.apache.org/blog/2024/03/06/comet-donation/
• Move of DataFusion to a top level Apache project was approved by the community: [DISCUSS] Move Apache Arrow Datafusion to a new top level Apache project #6475
• Large scale "extract scalar functions from the core" continues at good pace way [Epic] Port BuiltInFunctons to datafusion-functions-* crates #9285

Performance improvements:

• specialized group values for strings/binary Implement specialized group values for single Uft8/LargeUtf8/Binary/LargeBinary column #8827

• Meetup

• Agenda for DataFusion meetup 2024 is looking good UPDATE: VENUE IS FULL Invitation: Mar 25, 2024 DataFusion meetup before the DataCouncil March 2024 in Austin TX #8522

• DataFusion SIGMOD paper about DataFusion was Finalize SIGMOD 2024 paper ~(if accepted)~ #8373 (comment)

SQL to String features from @devinjdangelo / @backkem

• [EPIC] Support converting Exprs and LogicalPlans --> SQL Strings #9494

Maybe REcursive CTEs:

• Hardening Recursive CTEs Add support for recursive CTEs #462 with @matthewgapp and @jonahgao

[alamb]

CTE support: #9619

[Omega359]

Just a few things that come to mind:

Functions:

• New: nv1, nvl2
• Improvements: unnest, null handling improvements for lead/lag

[alamb]

WASM from @waynexia #9834

[alamb]

I am now officially out of time and excuses -- I need to write this post soon

[alamb]

Started gathering ideas apache/datafusion-site#6

[alamb]

I plan one more round of copyediting and then posting in 2 days. Please leave comments if you have any: apache/datafusion-site#6

[alamb]

Blog post is live: https://datafusion.apache.org/blog/2024/07/24/datafusion-40.0.0/

[alamb]

Filed #11631 to track the next one