Move handling of SaveMode to FilesystemStreams

[norberttech]

Change Log

Added

Fixed

Changed

• Moved handling SaveMode to FilesystemStreams

Removed

• Execution Plan and Processors

Deprecated

Security

---

Description

Since the only use case for Processors was to handle SaveMode and we moved handling SaveMode to FilesystemStreams this mechanism can be removed for now and replaced with a proper Executor later.

The main goal of this change was to avoid removing all existing partitions when SaveMode::Overwrite is used but instead remove only partitions that should be overwritten.

[github-actions]

Flow PHP - Benchmarks

^{_{Results of the benchmarks from this PR are compared with the results from 1.x branch.}}

Extractors

+-----------------------+-------------------+------+-----+------------------+------------------+-----------------+
| benchmark             | subject           | revs | its | mem_peak         | mode             | rstdev          |
+-----------------------+-------------------+------+-----+------------------+------------------+-----------------+
| AvroExtractorBench    | bench_extract_10k | 1    | 3   | 34.743mb +0.03%  | 706.940ms +1.52% | ±1.24% +26.38%  |
| CSVExtractorBench     | bench_extract_10k | 1    | 3   | 4.603mb +0.31%   | 297.136ms +0.58% | ±0.36% -8.16%   |
| JsonExtractorBench    | bench_extract_10k | 1    | 3   | 4.767mb +0.21%   | 948.594ms +0.99% | ±1.11% +68.07%  |
| ParquetExtractorBench | bench_extract_10k | 1    | 3   | 239.477mb +0.00% | 1.113s +0.12%    | ±0.32% -70.94%  |
| TextExtractorBench    | bench_extract_10k | 1    | 3   | 4.560mb -0.07%   | 23.720ms +1.58%  | ±0.92% -8.38%   |
| XmlExtractorBench     | bench_extract_10k | 1    | 3   | 4.560mb -0.07%   | 403.309ms -0.03% | ±2.69% +344.58% |
+-----------------------+-------------------+------+-----+------------------+------------------+-----------------+

Transformers

+-----------------------------+--------------------------+------+-----+------------------+-----------------+----------------+
| benchmark                   | subject                  | revs | its | mem_peak         | mode            | rstdev         |
+-----------------------------+--------------------------+------+-----+------------------+-----------------+----------------+
| RenameEntryTransformerBench | bench_transform_10k_rows | 1    | 3   | 110.244mb -0.00% | 62.101ms -1.18% | ±0.79% -68.63% |
+-----------------------------+--------------------------+------+-----+------------------+-----------------+----------------+

Loaders

+--------------------+----------------+------+-----+------------------+------------------+----------------+
| benchmark          | subject        | revs | its | mem_peak         | mode             | rstdev         |
+--------------------+----------------+------+-----+------------------+------------------+----------------+
| AvroLoaderBench    | bench_load_10k | 1    | 3   | 94.404mb +0.01%  | 438.528ms -0.30% | ±1.29% +12.72% |
| CSVLoaderBench     | bench_load_10k | 1    | 3   | 54.709mb +0.02%  | 70.521ms -3.06%  | ±0.52% +5.90%  |
| JsonLoaderBench    | bench_load_10k | 1    | 3   | 104.986mb +0.01% | 54.800ms -1.99%  | ±1.12% +76.10% |
| ParquetLoaderBench | bench_load_10k | 1    | 3   | 320.464mb +0.00% | 1.452s -1.76%    | ±0.17% -86.77% |
| TextLoaderBench    | bench_load_10k | 1    | 3   | 17.587mb +0.06%  | 40.694ms -3.80%  | ±0.23% -57.67% |
+--------------------+----------------+------+-----+------------------+------------------+----------------+

Building Blocks

+-------------------------+----------------------------+------+-----+------------------+------------------+------------------+
| benchmark               | subject                    | revs | its | mem_peak         | mode             | rstdev           |
+-------------------------+----------------------------+------+-----+------------------+------------------+------------------+
| RowsBench               | bench_chunk_10_on_10k      | 2    | 3   | 76.291mb +0.01%  | 2.861ms +32.60%  | ±3.25% +269.08%  |
| RowsBench               | bench_diff_left_1k_on_10k  | 2    | 3   | 96.082mb +0.01%  | 177.618ms -1.21% | ±0.51% -31.77%   |
| RowsBench               | bench_diff_right_1k_on_10k | 2    | 3   | 74.608mb +0.01%  | 17.446ms -3.76%  | ±0.80% -27.94%   |
| RowsBench               | bench_drop_1k_on_10k       | 2    | 3   | 75.430mb +2.79%  | 1.703ms -0.11%   | ±2.67% +348.53%  |
| RowsBench               | bench_drop_right_1k_on_10k | 2    | 3   | 75.430mb +2.79%  | 1.666ms +0.99%   | ±1.06% -8.21%    |
| RowsBench               | bench_entries_on_10k       | 2    | 3   | 74.643mb +0.01%  | 2.504ms -2.45%   | ±0.61% -73.42%   |
| RowsBench               | bench_filter_on_10k        | 2    | 3   | 75.172mb +0.01%  | 14.304ms +0.83%  | ±0.75% +46.82%   |
| RowsBench               | bench_find_on_10k          | 2    | 3   | 75.172mb +0.01%  | 14.161ms -1.96%  | ±0.66% -15.83%   |
| RowsBench               | bench_find_one_on_10k      | 10   | 3   | 73.075mb +0.01%  | 1.706μs 0.00%    | ±2.72% 0.00%     |
| RowsBench               | bench_first_on_10k         | 10   | 3   | 73.075mb +0.01%  | 0.400μs 0.00%    | ±0.00% 0.00%     |
| RowsBench               | bench_flat_map_on_1k       | 2    | 3   | 86.632mb +0.01%  | 12.023ms -2.05%  | ±1.91% +3011.51% |
| RowsBench               | bench_map_on_10k           | 2    | 3   | 115.991mb +0.00% | 65.430ms +1.85%  | ±1.66% +43.31%   |
| RowsBench               | bench_merge_1k_on_10k      | 2    | 3   | 75.693mb +0.01%  | 1.792ms -3.23%   | ±1.36% +8.08%    |
| RowsBench               | bench_partition_by_on_10k  | 2    | 3   | 77.961mb +0.00%  | 32.982ms -5.13%  | ±0.19% -54.25%   |
| RowsBench               | bench_remove_on_10k        | 2    | 3   | 77.793mb +0.01%  | 3.820ms -12.62%  | ±2.84% +684.92%  |
| RowsBench               | bench_sort_asc_on_1k       | 2    | 3   | 73.218mb +0.01%  | 37.885ms -1.63%  | ±0.99% +79.41%   |
| RowsBench               | bench_sort_by_on_1k        | 2    | 3   | 73.219mb +0.01%  | 38.376ms +0.36%  | ±0.24% -69.04%   |
| RowsBench               | bench_sort_desc_on_1k      | 2    | 3   | 73.218mb +0.01%  | 38.110ms +0.25%  | ±1.14% +13.53%   |
| RowsBench               | bench_sort_entries_on_1k   | 2    | 3   | 75.517mb +0.01%  | 7.357ms -0.43%   | ±0.56% -63.75%   |
| RowsBench               | bench_sort_on_1k           | 2    | 3   | 73.075mb +0.01%  | 28.694ms +1.30%  | ±1.18% -53.32%   |
| RowsBench               | bench_take_1k_on_10k       | 10   | 3   | 73.075mb +0.01%  | 13.300μs +3.90%  | ±0.61% -51.88%   |
| RowsBench               | bench_take_right_1k_on_10k | 10   | 3   | 73.075mb +0.01%  | 15.900μs +2.62%  | ±0.00% -100.00%  |
| RowsBench               | bench_unique_on_1k         | 2    | 3   | 96.083mb +0.01%  | 179.039ms -2.75% | ±0.46% -61.51%   |
| TypeDetectorBench       | bench_type_detector        | 1    | 3   | 59.171mb +0.00%  | 352.838ms +0.13% | ±0.66% +26.02%   |
| TypeDetectorBench       | bench_type_detector        | 1    | 3   | 14.094mb +0.02%  | 69.367ms -1.74%  | ±0.93% +10.21%   |
| NativeEntryFactoryBench | bench_entry_factory        | 1    | 3   | 115.835mb +0.00% | 387.201ms -1.68% | ±0.89% -46.61%   |
| NativeEntryFactoryBench | bench_entry_factory        | 1    | 3   | 59.553mb +0.01%  | 193.977ms -1.76% | ±1.78% +238.21%  |
| NativeEntryFactoryBench | bench_entry_factory        | 1    | 3   | 14.675mb +0.05%  | 41.577ms -5.64%  | ±0.85% -73.34%   |
+-------------------------+----------------------------+------+-----+------------------+------------------+------------------+