We are happy to announce the first pre-release of Polars 2.0.
Polars 2.0 will bring the streaming engine as a default and improves overall strictness of Polars.
Give it a spin and share your thoughts. We expect the final Polars 2.0 release in 3-4 weeks.
pola.rs/posts/announci…
The Polars GPU engine now runs on a new streaming backend, built with NVIDIA on their RapidsMPF library.
• Queries are no longer bound by GPU memory. Data flows through the query graph in morsels that spill to host memory when device memory gets tight.
• The same query scales
We've released Python Polars 1.44. Some of the highlights:
• Schema evolution when writing to Iceberg
sink_iceberg() gained schema_mode="merge", so appending a frame that has a new column extends the table schema instead of raising. Rows written earlier read back as null for
As datasets grow, your tools can limit what analysis is possible. That causes teams to slows down, or forces them to work with samples rather than the whole dataset.
BMLL Technologies tackled this issue by migrating their market data workloads to Polars, resulting in a 48x
