Added DataFrame::until method

[norberttech]

Change Log

Added

• Added DataFrame::until

Fixed

Changed

Removed

Deprecated

Security

---

Description

The difference between DataFrame::filter and DataFrame::until is that the filter will be executed at all extracted Rows, until on the other hand will send an STOP signal to extractor when a condition is not satisfied.

This is helpful when iterating over sorted datasets without possibility to filter them upfront to avoid reading all rows.

[github-actions]

Flow PHP - Benchmarks

^{_{Results of the benchmarks from this PR are compared with the results from 1.x branch.}}

Extractors

+-----------------------+-------------------+------+-----+------------------+------------------+-----------------+
| benchmark             | subject           | revs | its | mem_peak         | mode             | rstdev          |
+-----------------------+-------------------+------+-----+------------------+------------------+-----------------+
| AvroExtractorBench    | bench_extract_10k | 1    | 3   | 34.747mb +0.00%  | 1.177s +3.54%    | ±1.48% +166.19% |
| CSVExtractorBench     | bench_extract_10k | 1    | 3   | 4.605mb +0.01%   | 302.363ms +0.01% | ±0.53% +272.28% |
| JsonExtractorBench    | bench_extract_10k | 1    | 3   | 4.771mb +0.01%   | 1.396s +0.71%    | ±0.57% -28.45%  |
| ParquetExtractorBench | bench_extract_10k | 1    | 3   | 239.480mb +0.00% | 1.574s -0.08%    | ±0.72% +74.24%  |
| TextExtractorBench    | bench_extract_10k | 1    | 3   | 4.559mb +0.01%   | 24.764ms -0.20%  | ±1.53% -3.99%   |
| XmlExtractorBench     | bench_extract_10k | 1    | 3   | 4.560mb +0.01%   | 402.986ms +0.36% | ±0.24% -86.64%  |
+-----------------------+-------------------+------+-----+------------------+------------------+-----------------+

Transformers

+-----------------------------+--------------------------+------+-----+------------------+-----------------+----------------+
| benchmark                   | subject                  | revs | its | mem_peak         | mode            | rstdev         |
+-----------------------------+--------------------------+------+-----+------------------+-----------------+----------------+
| RenameEntryTransformerBench | bench_transform_10k_rows | 1    | 3   | 110.246mb +0.00% | 67.087ms +2.24% | ±1.16% +12.06% |
+-----------------------------+--------------------------+------+-----+------------------+-----------------+----------------+

Loaders

+--------------------+----------------+------+-----+------------------+------------------+-----------------+
| benchmark          | subject        | revs | its | mem_peak         | mode             | rstdev          |
+--------------------+----------------+------+-----+------------------+------------------+-----------------+
| AvroLoaderBench    | bench_load_10k | 1    | 3   | 94.727mb +0.00%  | 445.433ms -0.17% | ±0.71% -5.36%   |
| CSVLoaderBench     | bench_load_10k | 1    | 3   | 54.711mb +0.00%  | 70.757ms +1.84%  | ±1.06% -46.72%  |
| JsonLoaderBench    | bench_load_10k | 1    | 3   | 105.309mb +0.00% | 55.535ms +1.14%  | ±0.46% -78.07%  |
| ParquetLoaderBench | bench_load_10k | 1    | 3   | 320.787mb +0.00% | 1.462s -0.65%    | ±0.40% +12.19%  |
| TextLoaderBench    | bench_load_10k | 1    | 3   | 17.589mb +0.00%  | 41.479ms -1.80%  | ±0.56% +144.96% |
+--------------------+----------------+------+-----+------------------+------------------+-----------------+

Building Blocks

+-------------------------+----------------------------+------+-----+------------------+------------------+------------------+
| benchmark               | subject                    | revs | its | mem_peak         | mode             | rstdev           |
+-------------------------+----------------------------+------+-----+------------------+------------------+------------------+
| RowsBench               | bench_chunk_10_on_10k      | 2    | 3   | 76.293mb +0.00%  | 2.165ms -0.34%   | ±0.65% -40.32%   |
| RowsBench               | bench_diff_left_1k_on_10k  | 2    | 3   | 96.084mb +0.00%  | 182.867ms +0.87% | ±0.55% -32.32%   |
| RowsBench               | bench_diff_right_1k_on_10k | 2    | 3   | 74.609mb +0.00%  | 17.977ms +0.14%  | ±0.44% +25.43%   |
| RowsBench               | bench_drop_1k_on_10k       | 2    | 3   | 75.432mb +0.00%  | 1.636ms -1.76%   | ±2.04% +213.22%  |
| RowsBench               | bench_drop_right_1k_on_10k | 2    | 3   | 75.432mb +0.00%  | 1.640ms -2.97%   | ±0.22% -86.28%   |
| RowsBench               | bench_entries_on_10k       | 2    | 3   | 74.645mb +0.00%  | 2.491ms -5.36%   | ±0.69% -74.13%   |
| RowsBench               | bench_filter_on_10k        | 2    | 3   | 75.174mb +0.00%  | 16.605ms +17.59% | ±0.54% +123.59%  |
| RowsBench               | bench_find_on_10k          | 2    | 3   | 75.174mb +0.00%  | 16.619ms +16.37% | ±0.54% +23.89%   |
| RowsBench               | bench_find_one_on_10k      | 10   | 3   | 73.077mb +0.00%  | 1.800μs +0.34%   | ±0.00% -100.00%  |
| RowsBench               | bench_first_on_10k         | 10   | 3   | 73.077mb +0.00%  | 0.300μs 0.00%    | ±0.00% 0.00%     |
| RowsBench               | bench_flat_map_on_1k       | 2    | 3   | 86.634mb +0.00%  | 12.698ms -2.57%  | ±1.13% -32.75%   |
| RowsBench               | bench_map_on_10k           | 2    | 3   | 115.993mb +0.00% | 63.730ms -0.53%  | ±1.47% +21.33%   |
| RowsBench               | bench_merge_1k_on_10k      | 2    | 3   | 75.695mb +0.00%  | 1.800ms -3.59%   | ±0.37% -78.95%   |
| RowsBench               | bench_partition_by_on_10k  | 2    | 3   | 77.962mb +0.00%  | 36.759ms +12.95% | ±0.38% -50.88%   |
| RowsBench               | bench_remove_on_10k        | 2    | 3   | 77.795mb +0.00%  | 4.327ms -7.72%   | ±0.38% -88.05%   |
| RowsBench               | bench_sort_asc_on_1k       | 2    | 3   | 73.220mb +0.00%  | 38.743ms +0.84%  | ±0.92% +72.58%   |
| RowsBench               | bench_sort_by_on_1k        | 2    | 3   | 73.221mb +0.00%  | 38.642ms +0.03%  | ±0.66% -16.64%   |
| RowsBench               | bench_sort_desc_on_1k      | 2    | 3   | 73.220mb +0.00%  | 38.843ms +0.23%  | ±2.10% +1080.61% |
| RowsBench               | bench_sort_entries_on_1k   | 2    | 3   | 75.519mb +0.00%  | 7.395ms -0.01%   | ±0.73% +7.12%    |
| RowsBench               | bench_sort_on_1k           | 2    | 3   | 73.077mb +0.00%  | 28.479ms -0.15%  | ±0.10% -78.92%   |
| RowsBench               | bench_take_1k_on_10k       | 10   | 3   | 73.077mb +0.00%  | 12.766μs -0.36%  | ±1.60% +118.51%  |
| RowsBench               | bench_take_right_1k_on_10k | 10   | 3   | 73.077mb +0.00%  | 15.588μs -1.38%  | ±0.61% +103.86%  |
| RowsBench               | bench_unique_on_1k         | 2    | 3   | 96.085mb +0.00%  | 190.118ms +3.04% | ±0.61% +81.49%   |
| TypeDetectorBench       | bench_type_detector        | 1    | 3   | 98.214mb +0.00%  | 970.909ms +1.02% | ±0.07% -86.39%   |
| TypeDetectorBench       | bench_type_detector        | 1    | 3   | 21.905mb +0.00%  | 190.576ms +0.46% | ±1.06% +27.70%   |
| NativeEntryFactoryBench | bench_entry_factory        | 1    | 3   | 115.837mb +0.00% | 769.798ms +0.76% | ±0.97% +46.93%   |
| NativeEntryFactoryBench | bench_entry_factory        | 1    | 3   | 59.555mb +0.00%  | 383.676ms -0.98% | ±0.72% -25.28%   |
| NativeEntryFactoryBench | bench_entry_factory        | 1    | 3   | 14.678mb +0.00%  | 78.935ms +0.72%  | ±0.38% -63.82%   |
+-------------------------+----------------------------+------+-----+------------------+------------------+------------------+