Simplified filesystem streams

[norberttech]

Change Log

Added

Fixed

• updating file by ETL through overwrite save mode
• FilesystemStreams test suite
• even when writing to a partitioned dataset, flow will respect original filename given as destination path

Changed

• Writing to partitioned datasets is no longer creating random file names, even append will create new files with suffixed name of the original file

Removed

• DataFrame::threadSafe / DataFrame::appendSafe

Deprecated

Security

---

Description

tldr;

Before, the following code would not work because of how FilesystemStreams was implemented and how it was handling overwrite() save mode.

df()
    ->read(from_csv(__DIR__ . '/dataset/file.csv'))
    ->saveMode(overwrite())
    ->drop('name')
    ->write(to_csv(__DIR__ . '/dataset/file.csv'))
    ->run();

Now save mode is first writing files on a side (with ._flow_tmp suffix) and only once everything is completed it removes old files and rename new ones.
This is supported for both, partitioned and not partitioned datasets.

[github-actions]

Flow PHP - Benchmarks

^{_{Results of the benchmarks from this PR are compared with the results from 1.x branch.}}

Extractors

+-----------------------+-------------------+------+-----+------------------+------------------+----------------+
| benchmark             | subject           | revs | its | mem_peak         | mode             | rstdev         |
+-----------------------+-------------------+------+-----+------------------+------------------+----------------+
| AvroExtractorBench    | bench_extract_10k | 1    | 3   | 35.269mb +0.04%  | 823.110ms -0.96% | ±1.30% +50.20% |
| CSVExtractorBench     | bench_extract_10k | 1    | 3   | 4.989mb +0.35%   | 341.570ms -0.12% | ±0.35% +19.65% |
| JsonExtractorBench    | bench_extract_10k | 1    | 3   | 5.139mb +0.35%   | 1.051s -0.62%    | ±0.86% +55.57% |
| ParquetExtractorBench | bench_extract_10k | 1    | 3   | 135.804mb +0.01% | 890.214ms -0.14% | ±0.53% -33.87% |
| TextExtractorBench    | bench_extract_10k | 1    | 3   | 4.900mb +0.28%   | 35.439ms +0.25%  | ±0.38% +23.32% |
| XmlExtractorBench     | bench_extract_10k | 1    | 3   | 4.902mb +0.36%   | 440.584ms +1.43% | ±3.15% +26.34% |
+-----------------------+-------------------+------+-----+------------------+------------------+----------------+

Transformers

+-----------------------------+--------------------------+------+-----+------------------+-----------------+-----------------+
| benchmark                   | subject                  | revs | its | mem_peak         | mode            | rstdev          |
+-----------------------------+--------------------------+------+-----+------------------+-----------------+-----------------+
| RenameEntryTransformerBench | bench_transform_10k_rows | 1    | 3   | 110.609mb +0.01% | 63.929ms -1.61% | ±0.47% +112.87% |
+-----------------------------+--------------------------+------+-----+------------------+-----------------+-----------------+

Loaders

+--------------------+----------------+------+-----+------------------+------------------+-----------------+
| benchmark          | subject        | revs | its | mem_peak         | mode             | rstdev          |
+--------------------+----------------+------+-----+------------------+------------------+-----------------+
| AvroLoaderBench    | bench_load_10k | 1    | 3   | 95.650mb +0.01%  | 469.250ms -1.09% | ±0.61% -19.95%  |
| CSVLoaderBench     | bench_load_10k | 1    | 3   | 54.125mb +0.04%  | 71.671ms -0.73%  | ±0.13% -81.07%  |
| JsonLoaderBench    | bench_load_10k | 1    | 3   | 106.554mb +0.02% | 53.354ms +0.31%  | ±0.76% -24.46%  |
| ParquetLoaderBench | bench_load_10k | 1    | 3   | 224.372mb +0.01% | 1.424s -0.79%    | ±0.69% +123.57% |
| TextLoaderBench    | bench_load_10k | 1    | 3   | 17.948mb +0.07%  | 39.329ms -2.83%  | ±0.59% +60.22%  |
+--------------------+----------------+------+-----+------------------+------------------+-----------------+

Building Blocks

+-------------------------+----------------------------+------+-----+------------------+------------------+-----------------+
| benchmark               | subject                    | revs | its | mem_peak         | mode             | rstdev          |
+-------------------------+----------------------------+------+-----+------------------+------------------+-----------------+
| NativeEntryFactoryBench | bench_entry_factory        | 1    | 3   | 116.715mb +0.00% | 499.100ms -0.23% | ±1.47% +16.05%  |
| NativeEntryFactoryBench | bench_entry_factory        | 1    | 3   | 60.194mb +0.01%  | 248.649ms +2.29% | ±0.99% +9.38%   |
| NativeEntryFactoryBench | bench_entry_factory        | 1    | 3   | 15.128mb +0.02%  | 53.232ms +1.96%  | ±1.77% +181.85% |
| TypeDetectorBench       | bench_type_detector        | 1    | 3   | 59.959mb +0.01%  | 435.038ms +0.18% | ±0.24% -78.35%  |
| TypeDetectorBench       | bench_type_detector        | 1    | 3   | 14.498mb +0.02%  | 85.552ms -1.44%  | ±0.81% +124.66% |
| RowsBench               | bench_chunk_10_on_10k      | 2    | 3   | 76.684mb +0.00%  | 3.394ms +0.48%   | ±0.21% -92.91%  |
| RowsBench               | bench_diff_left_1k_on_10k  | 2    | 3   | 96.410mb +0.00%  | 179.671ms +0.23% | ±0.62% -23.25%  |
| RowsBench               | bench_diff_right_1k_on_10k | 2    | 3   | 74.936mb +0.00%  | 17.901ms -2.06%  | ±0.52% +132.46% |
| RowsBench               | bench_drop_1k_on_10k       | 2    | 3   | 77.924mb +0.00%  | 1.694ms -0.33%   | ±2.56% +25.19%  |
| RowsBench               | bench_drop_right_1k_on_10k | 2    | 3   | 77.924mb +0.00%  | 1.678ms -14.69%  | ±0.95% -69.18%  |
| RowsBench               | bench_entries_on_10k       | 2    | 3   | 75.036mb +0.00%  | 2.580ms +2.41%   | ±2.72% -21.17%  |
| RowsBench               | bench_filter_on_10k        | 2    | 3   | 75.565mb +0.00%  | 15.307ms -9.29%  | ±1.24% -6.95%   |
| RowsBench               | bench_find_on_10k          | 2    | 3   | 75.565mb +0.00%  | 14.848ms -11.71% | ±0.24% -84.28%  |
| RowsBench               | bench_find_one_on_10k      | 10   | 3   | 73.469mb +0.00%  | 1.806μs 0.00%    | ±2.57% 0.00%    |
| RowsBench               | bench_first_on_10k         | 10   | 3   | 73.469mb +0.00%  | 0.400μs 0.00%    | ±0.00% 0.00%    |
| RowsBench               | bench_flat_map_on_1k       | 2    | 3   | 87.023mb +0.00%  | 13.106ms +1.13%  | ±1.46% +129.65% |
| RowsBench               | bench_map_on_10k           | 2    | 3   | 116.384mb +0.00% | 65.437ms -3.20%  | ±0.68% -47.65%  |
| RowsBench               | bench_merge_1k_on_10k      | 2    | 3   | 76.084mb +0.00%  | 1.238ms -4.58%   | ±2.81% +93.92%  |
| RowsBench               | bench_partition_by_on_10k  | 2    | 3   | 79.431mb +0.00%  | 56.838ms -1.88%  | ±1.25% -0.42%   |
| RowsBench               | bench_remove_on_10k        | 2    | 3   | 78.186mb +0.00%  | 3.825ms -0.02%   | ±2.26% +20.26%  |
| RowsBench               | bench_sort_asc_on_1k       | 2    | 3   | 73.547mb +0.00%  | 40.534ms +1.06%  | ±0.66% -71.03%  |
| RowsBench               | bench_sort_by_on_1k        | 2    | 3   | 73.547mb +0.00%  | 40.665ms -0.80%  | ±1.98% +9.97%   |
| RowsBench               | bench_sort_desc_on_1k      | 2    | 3   | 73.547mb +0.00%  | 40.591ms +1.91%  | ±1.56% +48.49%  |
| RowsBench               | bench_sort_entries_on_1k   | 2    | 3   | 75.910mb +0.00%  | 7.323ms -0.26%   | ±0.33% -26.98%  |
| RowsBench               | bench_sort_on_1k           | 2    | 3   | 73.469mb +0.00%  | 29.019ms -1.46%  | ±1.54% -30.62%  |
| RowsBench               | bench_take_1k_on_10k       | 10   | 3   | 73.469mb +0.00%  | 13.336μs +0.18%  | ±2.10% +197.04% |
| RowsBench               | bench_take_right_1k_on_10k | 10   | 3   | 73.469mb +0.00%  | 16.379μs +1.66%  | ±2.81% +70.71%  |
| RowsBench               | bench_unique_on_1k         | 2    | 3   | 96.477mb +0.00%  | 186.195ms +0.16% | ±0.83% +160.19% |
+-------------------------+----------------------------+------+-----+------------------+------------------+-----------------+