Added ArrayExtractor

[norberttech]

Change Log

Added

• Added ArrayExtractor

Fixed

Changed

Removed

Deprecated

Security

---

Description

The main difference between ArrayExtractor and MemoryExtractor is that ArrayExtractor accepts iterables which means it can receive data from Generators

[github-actions]

Flow PHP - Benchmarks

^{_{Results of the benchmarks from this PR are compared with the results from 1.x branch.}}

Extractors

+-----------------------+-------------------+------+-----+------------------+------------------+-----------------+
| benchmark             | subject           | revs | its | mem_peak         | mode             | rstdev          |
+-----------------------+-------------------+------+-----+------------------+------------------+-----------------+
| AvroExtractorBench    | bench_extract_10k | 1    | 3   | 35.122mb +0.00%  | 688.366ms -3.82% | ±1.76% +39.56%  |
| CSVExtractorBench     | bench_extract_10k | 1    | 3   | 4.821mb +0.03%   | 303.359ms +0.16% | ±0.19% -91.38%  |
| JsonExtractorBench    | bench_extract_10k | 1    | 3   | 4.919mb +0.03%   | 920.716ms -1.66% | ±3.46% +222.47% |
| ParquetExtractorBench | bench_extract_10k | 1    | 3   | 239.625mb +0.00% | 1.135s +2.38%    | ±0.67% +6.60%   |
| TextExtractorBench    | bench_extract_10k | 1    | 3   | 4.698mb +0.03%   | 25.212ms +3.14%  | ±1.05% +39.13%  |
| XmlExtractorBench     | bench_extract_10k | 1    | 3   | 4.699mb +0.03%   | 408.796ms +0.72% | ±0.78% +101.22% |
+-----------------------+-------------------+------+-----+------------------+------------------+-----------------+

Transformers

+-----------------------------+--------------------------+------+-----+------------------+-----------------+----------------+
| benchmark                   | subject                  | revs | its | mem_peak         | mode            | rstdev         |
+-----------------------------+--------------------------+------+-----+------------------+-----------------+----------------+
| RenameEntryTransformerBench | bench_transform_10k_rows | 1    | 3   | 110.385mb +0.00% | 68.017ms +6.63% | ±0.36% -84.96% |
+-----------------------------+--------------------------+------+-----+------------------+-----------------+----------------+

Loaders

+--------------------+----------------+------+-----+------------------+------------------+-----------------+
| benchmark          | subject        | revs | its | mem_peak         | mode             | rstdev          |
+--------------------+----------------+------+-----+------------------+------------------+-----------------+
| AvroLoaderBench    | bench_load_10k | 1    | 3   | 94.783mb +0.00%  | 443.551ms +0.98% | ±0.19% -79.16%  |
| CSVLoaderBench     | bench_load_10k | 1    | 3   | 54.861mb +0.00%  | 72.045ms +0.82%  | ±1.26% +122.14% |
| JsonLoaderBench    | bench_load_10k | 1    | 3   | 105.444mb +0.00% | 58.278ms +5.59%  | ±0.81% -57.71%  |
| ParquetLoaderBench | bench_load_10k | 1    | 3   | 320.653mb +0.00% | 1.542s +4.67%    | ±1.51% +107.62% |
| TextLoaderBench    | bench_load_10k | 1    | 3   | 17.738mb +0.01%  | 42.058ms +0.67%  | ±0.57% +78.29%  |
+--------------------+----------------+------+-----+------------------+------------------+-----------------+

Building Blocks

+-------------------------+----------------------------+------+-----+------------------+------------------+-----------------+
| benchmark               | subject                    | revs | its | mem_peak         | mode             | rstdev          |
+-------------------------+----------------------------+------+-----+------------------+------------------+-----------------+
| RowsBench               | bench_chunk_10_on_10k      | 2    | 3   | 76.435mb +0.00%  | 3.549ms +21.34%  | ±2.56% +4.56%   |
| RowsBench               | bench_diff_left_1k_on_10k  | 2    | 3   | 96.228mb +0.00%  | 184.936ms +1.80% | ±0.49% -37.93%  |
| RowsBench               | bench_diff_right_1k_on_10k | 2    | 3   | 74.754mb +0.00%  | 18.837ms +3.20%  | ±3.09% +341.36% |
| RowsBench               | bench_drop_1k_on_10k       | 2    | 3   | 77.675mb +0.00%  | 1.988ms +14.20%  | ±3.10% +49.81%  |
| RowsBench               | bench_drop_right_1k_on_10k | 2    | 3   | 77.675mb +0.00%  | 1.831ms +3.40%   | ±1.60% -41.68%  |
| RowsBench               | bench_entries_on_10k       | 2    | 3   | 74.788mb +0.00%  | 3.015ms +12.56%  | ±1.01% +0.62%   |
| RowsBench               | bench_filter_on_10k        | 2    | 3   | 75.316mb +0.00%  | 15.081ms -0.12%  | ±0.14% -91.44%  |
| RowsBench               | bench_find_on_10k          | 2    | 3   | 75.316mb +0.00%  | 15.146ms +2.96%  | ±1.09% +3.35%   |
| RowsBench               | bench_find_one_on_10k      | 10   | 3   | 73.219mb +0.00%  | 2.094μs +15.94%  | ±2.28% -11.29%  |
| RowsBench               | bench_first_on_10k         | 10   | 3   | 73.219mb +0.00%  | 0.500μs +25.00%  | ±0.00% -100.00% |
| RowsBench               | bench_flat_map_on_1k       | 2    | 3   | 86.776mb +0.00%  | 14.914ms +18.92% | ±1.85% +0.09%   |
| RowsBench               | bench_map_on_10k           | 2    | 3   | 116.136mb +0.00% | 65.336ms +3.80%  | ±0.81% -67.14%  |
| RowsBench               | bench_merge_1k_on_10k      | 2    | 3   | 75.837mb +0.00%  | 2.069ms +13.33%  | ±3.00% +10.49%  |
| RowsBench               | bench_partition_by_on_10k  | 2    | 3   | 78.110mb +0.00%  | 35.014ms +1.23%  | ±0.95% -30.35%  |
| RowsBench               | bench_remove_on_10k        | 2    | 3   | 77.937mb +0.00%  | 4.184ms +5.51%   | ±2.11% +424.40% |
| RowsBench               | bench_sort_asc_on_1k       | 2    | 3   | 73.365mb +0.00%  | 41.055ms +3.56%  | ±0.65% +70.53%  |
| RowsBench               | bench_sort_by_on_1k        | 2    | 3   | 73.365mb +0.00%  | 42.627ms +6.74%  | ±0.73% +168.54% |
| RowsBench               | bench_sort_desc_on_1k      | 2    | 3   | 73.365mb +0.00%  | 40.789ms +0.43%  | ±2.39% +194.29% |
| RowsBench               | bench_sort_entries_on_1k   | 2    | 3   | 75.662mb +0.00%  | 7.686ms +4.55%   | ±3.69% +246.31% |
| RowsBench               | bench_sort_on_1k           | 2    | 3   | 73.219mb +0.00%  | 30.686ms +6.56%  | ±2.24% +511.04% |
| RowsBench               | bench_take_1k_on_10k       | 10   | 3   | 73.219mb +0.00%  | 14.037μs +4.76%  | ±3.25% +78.03%  |
| RowsBench               | bench_take_right_1k_on_10k | 10   | 3   | 73.219mb +0.00%  | 16.888μs +6.97%  | ±0.56% -66.00%  |
| RowsBench               | bench_unique_on_1k         | 2    | 3   | 96.229mb +0.00%  | 182.653ms +0.32% | ±0.16% -76.54%  |
| TypeDetectorBench       | bench_type_detector        | 1    | 3   | 59.380mb +0.00%  | 336.694ms +1.68% | ±0.26% -58.55%  |
| TypeDetectorBench       | bench_type_detector        | 1    | 3   | 14.303mb +0.01%  | 67.043ms +2.23%  | ±1.79% +463.82% |
| NativeEntryFactoryBench | bench_entry_factory        | 1    | 3   | 115.978mb +0.00% | 395.852ms +3.57% | ±1.21% -52.38%  |
| NativeEntryFactoryBench | bench_entry_factory        | 1    | 3   | 59.696mb +0.00%  | 196.074ms +1.65% | ±3.40% +234.81% |
| NativeEntryFactoryBench | bench_entry_factory        | 1    | 3   | 14.821mb +0.01%  | 42.117ms +5.13%  | ±1.90% -35.29%  |
+-------------------------+----------------------------+------+-----+------------------+------------------+-----------------+