Added DataFrame::dropPartitions() transformation

[norberttech]

Change Log

Added

• DataFrame::dropPartitions() transformation

Fixed

Changed

Removed

Deprecated

Security

---

Description

[github-actions]

Flow PHP - Benchmarks

^{_{Results of the benchmarks from this PR are compared with the results from 1.x branch.}}

Extractors

+-----------------------+-------------------+------+-----+------------------+------------------+-----------------+
| benchmark             | subject           | revs | its | mem_peak         | mode             | rstdev          |
+-----------------------+-------------------+------+-----+------------------+------------------+-----------------+
| AvroExtractorBench    | bench_extract_10k | 1    | 3   | 35.132mb +0.00%  | 724.264ms +0.16% | ±0.48% -68.91%  |
| CSVExtractorBench     | bench_extract_10k | 1    | 3   | 4.817mb +0.03%   | 319.572ms +0.11% | ±2.69% +759.50% |
| JsonExtractorBench    | bench_extract_10k | 1    | 3   | 4.882mb +0.03%   | 936.064ms +0.16% | ±0.30% +103.40% |
| ParquetExtractorBench | bench_extract_10k | 1    | 3   | 239.586mb +0.00% | 1.142s +1.07%    | ±1.93% +367.38% |
| TextExtractorBench    | bench_extract_10k | 1    | 3   | 4.666mb +0.02%   | 30.849ms +0.44%  | ±2.05% +106.85% |
| XmlExtractorBench     | bench_extract_10k | 1    | 3   | 4.668mb +0.02%   | 428.187ms +0.00% | ±0.13% -95.11%  |
+-----------------------+-------------------+------+-----+------------------+------------------+-----------------+

Transformers

+-----------------------------+--------------------------+------+-----+------------------+-----------------+----------------+
| benchmark                   | subject                  | revs | its | mem_peak         | mode            | rstdev         |
+-----------------------------+--------------------------+------+-----+------------------+-----------------+----------------+
| RenameEntryTransformerBench | bench_transform_10k_rows | 1    | 3   | 110.380mb +0.00% | 66.171ms +4.16% | ±0.21% -22.69% |
+-----------------------------+--------------------------+------+-----+------------------+-----------------+----------------+

Loaders

+--------------------+----------------+------+-----+------------------+------------------+-----------------+
| benchmark          | subject        | revs | its | mem_peak         | mode             | rstdev          |
+--------------------+----------------+------+-----+------------------+------------------+-----------------+
| AvroLoaderBench    | bench_load_10k | 1    | 3   | 94.744mb +0.00%  | 452.926ms -0.55% | ±0.79% -44.55%  |
| CSVLoaderBench     | bench_load_10k | 1    | 3   | 54.851mb +0.00%  | 74.204ms +3.05%  | ±0.64% -51.86%  |
| JsonLoaderBench    | bench_load_10k | 1    | 3   | 105.337mb +0.00% | 57.800ms +2.33%  | ±1.41% -53.28%  |
| ParquetLoaderBench | bench_load_10k | 1    | 3   | 320.548mb +0.00% | 1.303s +3.60%    | ±1.20% +227.73% |
| TextLoaderBench    | bench_load_10k | 1    | 3   | 17.708mb +0.01%  | 41.120ms -1.11%  | ±0.94% +427.75% |
+--------------------+----------------+------+-----+------------------+------------------+-----------------+

Building Blocks

+-------------------------+----------------------------+------+-----+------------------+------------------+------------------+
| benchmark               | subject                    | revs | its | mem_peak         | mode             | rstdev           |
+-------------------------+----------------------------+------+-----+------------------+------------------+------------------+
| NativeEntryFactoryBench | bench_entry_factory        | 1    | 3   | 115.995mb +0.00% | 387.357ms -1.29% | ±1.04% -38.60%   |
| NativeEntryFactoryBench | bench_entry_factory        | 1    | 3   | 59.714mb +0.00%  | 193.775ms +0.35% | ±0.86% -43.63%   |
| NativeEntryFactoryBench | bench_entry_factory        | 1    | 3   | 14.839mb +0.01%  | 42.832ms +0.38%  | ±1.51% +209.61%  |
| TypeDetectorBench       | bench_type_detector        | 1    | 3   | 59.402mb +0.00%  | 337.792ms +2.09% | ±0.59% +145.01%  |
| TypeDetectorBench       | bench_type_detector        | 1    | 3   | 14.325mb +0.01%  | 66.149ms +1.19%  | ±1.22% +23.76%   |
| RowsBench               | bench_chunk_10_on_10k      | 2    | 3   | 76.460mb +0.00%  | 3.333ms +2.85%   | ±3.00% -15.21%   |
| RowsBench               | bench_diff_left_1k_on_10k  | 2    | 3   | 96.253mb +0.00%  | 188.780ms +2.54% | ±0.67% -41.21%   |
| RowsBench               | bench_diff_right_1k_on_10k | 2    | 3   | 74.778mb +0.00%  | 18.937ms +3.23%  | ±0.58% +34.35%   |
| RowsBench               | bench_drop_1k_on_10k       | 2    | 3   | 77.700mb +0.00%  | 1.846ms +9.33%   | ±0.83% -16.05%   |
| RowsBench               | bench_drop_right_1k_on_10k | 2    | 3   | 77.700mb +0.00%  | 1.809ms -1.01%   | ±2.01% -26.65%   |
| RowsBench               | bench_entries_on_10k       | 2    | 3   | 74.812mb +0.00%  | 2.539ms +3.48%   | ±0.32% -35.79%   |
| RowsBench               | bench_filter_on_10k        | 2    | 3   | 75.341mb +0.00%  | 14.608ms +2.21%  | ±0.97% +20.52%   |
| RowsBench               | bench_find_on_10k          | 2    | 3   | 75.341mb +0.00%  | 14.442ms +1.05%  | ±3.22% +257.56%  |
| RowsBench               | bench_find_one_on_10k      | 10   | 3   | 73.244mb +0.00%  | 1.806μs +6.62%   | ±2.57% -9.09%    |
| RowsBench               | bench_first_on_10k         | 10   | 3   | 73.244mb +0.00%  | 0.400μs +33.33%  | ±0.00% +0.00%    |
| RowsBench               | bench_flat_map_on_1k       | 2    | 3   | 86.799mb +0.00%  | 12.690ms +1.01%  | ±0.68% -75.26%   |
| RowsBench               | bench_map_on_10k           | 2    | 3   | 116.160mb +0.00% | 64.152ms +2.51%  | ±0.21% -69.73%   |
| RowsBench               | bench_merge_1k_on_10k      | 2    | 3   | 75.861mb +0.00%  | 1.275ms -1.82%   | ±2.54% +1037.18% |
| RowsBench               | bench_partition_by_on_10k  | 2    | 3   | 79.207mb +0.00%  | 58.455ms +3.23%  | ±1.19% +54.71%   |
| RowsBench               | bench_remove_on_10k        | 2    | 3   | 77.962mb +0.00%  | 3.954ms +4.05%   | ±1.97% -36.31%   |
| RowsBench               | bench_sort_asc_on_1k       | 2    | 3   | 73.389mb +0.00%  | 40.239ms -1.35%  | ±1.38% +99.66%   |
| RowsBench               | bench_sort_by_on_1k        | 2    | 3   | 73.390mb +0.00%  | 40.628ms +3.04%  | ±1.45% -21.65%   |
| RowsBench               | bench_sort_desc_on_1k      | 2    | 3   | 73.389mb +0.00%  | 40.080ms -1.47%  | ±1.59% +37.13%   |
| RowsBench               | bench_sort_entries_on_1k   | 2    | 3   | 75.686mb +0.00%  | 7.538ms +2.90%   | ±3.32% +507.69%  |
| RowsBench               | bench_sort_on_1k           | 2    | 3   | 73.244mb +0.00%  | 29.035ms -1.71%  | ±0.95% -47.76%   |
| RowsBench               | bench_take_1k_on_10k       | 10   | 3   | 73.244mb +0.00%  | 13.336μs +1.64%  | ±2.10% +119.50%  |
| RowsBench               | bench_take_right_1k_on_10k | 10   | 3   | 73.244mb +0.00%  | 15.880μs +1.03%  | ±1.06% +18.93%   |
| RowsBench               | bench_unique_on_1k         | 2    | 3   | 96.254mb +0.00%  | 184.938ms +0.43% | ±0.26% -68.37%   |
+-------------------------+----------------------------+------+-----+------------------+------------------+------------------+