Release DataFusion `40.0.0`

[alamb]

Is your feature request related to a problem or challenge?

Tracking ticket for next release, also a place to track desired inclusions

Last release was https://crates.io/crates/datafusion/39.0.0 (June 10, 2024) so next major release would be around July 10, 2024

List if items that would be good to get into this release:

• For DataFusion Comet, we would like to upgrade to a version of arrow-rs that has Fix FFI array offset handling arrow-rs#5964
• Consider renaming UserDefinedSQLPlanner to ExprPlanner #11304
• Remove any aliases in Filter::try_new rather than erroring #11307
• Support IS NULL and IS NOT NULL on Unions #11321

Additional context

39.0.0 release ticket: #10517

[samuelcolvin]

Please could you consider including #11307, and maybe even #11321?

[alamb]

Please could you consider including #11307, and maybe even #11321?

Added to the list

[alamb]

FYI there is one more PR I would like to propose -- specifically reverting #11132 which turned out to be unecessary but will cause some uncessary downstream API churn. PR coming shortly

[samuelcolvin]

😞 yes good point, that had been weighing heavy on my mind, I'll do it now.

[alamb]

No worries -- I just made #11341 FYI @samuelcolvin

[samuelcolvin]

https://github.com/datafusion-contrib/datafusion-functions-json is now up to date with datafusion 40 on main, arrow and question mark operators work and the new union behavior is tested and working.

[hveiga]

Hi folks, is there an ETA for 40.0.0 to be released? Or maybe at least a release candidate? Thank you.

[alamb]

Sorry -- we didn't update the ticket

The release was approved https://lists.apache.org/thread/hvzw5x4rlysptd6936wkss0dfy1tkb5n

And it is available on crates.io: https://crates.io/crates/datafusion/40.0.0

[alamb]

I filed #11476 to track 41

[hveiga]

Thanks @alamb !