DataFusion `38.0.0` Release

[andygrove]

Is your feature request related to a problem or challenge?

I plan on starting to prepare for the DataFusion 38 release on Monday, April 29th. Let's use this issue to track which issues/PRs need to be included.

This will be the first release as a top-level ASF project and there will likely be some additional work required.

Describe the solution you'd like

Issues/PRs to include:

• Minor: Possibility to strip datafusion error name #10186 (would be helpful for the Comet project)
• Fix docs.rs build for datafusion-proto (hopefully) #10254
• Error "entered unreachable code: NamedStructField should be rewritten in OperatorToFunction" after upgrade to 37 #10181
• [Bug Fix]: Deem hash repartition unnecessary when input and output has 1 partition #10095
• Regression with coalesce expr_fn function: can't take multiple arguments #10320
• [BUG] Panic when querying table with wrong partition columns order #9785

Describe alternatives you've considered

No response

Additional context

37 release ticket: #9682

[l1t1]

please release the Python version too.

[andygrove]

please release the Python version too.

We will need someone to make the necessary changes to the DataFusion Python subproject first, so that it compiles against the new version of DataFusion.

Is this something that you would be interested in helping with?

[alamb]

I added a few more items to the waiting list:

#10254
#10181
#10095

[alamb]

Added #9785 per #9785 (comment)

[andygrove]

Hi @alamb. I assume we still want to wait on #10181 for creating the release?

[alamb]

Hi @alamb. I assume we still want to wait on #10181 for creating the release?

I think that would be good -- are you about ready to make a release candidate? If so I will work on #10181 tomorrow

[andygrove]

Hi @alamb. I assume we still want to wait on #10181 for creating the release?

I think that would be good -- are you about ready to make a release candidate? If so I will work on #10181 tomorrow

I am ready any time.

[alamb]

Here is a PR witha proposed API #10330 that would close #10181

[alamb]

I also think #10320 should be included in 38.0.0 (as it would be a regression if we released main as is). There is a PR up to fix it here #10321

[andygrove]

I will start the process of creating the first release candidate today.

[alamb]

Vote thread: https://lists.apache.org/thread/nnhtnxt8rs65k5x59scfdc0qkmqghy3n

[andygrove]

Release has been completed