Parquet statistics reading

[norberttech]

Change Log

Added

• Parquet statistics reader

Fixed

Changed

Removed

Deprecated

Security

---

Refs: #686

Description

[github-actions]

Flow PHP - Benchmarks

^{_{Results of the benchmarks from this PR are compared with the results from 1.x branch.}}

Extractors

+-----------------------+-------------------+------+-----+------------------+------------------+-----------------+
| benchmark             | subject           | revs | its | mem_peak         | mode             | rstdev          |
+-----------------------+-------------------+------+-----+------------------+------------------+-----------------+
| AvroExtractorBench    | bench_extract_10k | 1    | 3   | 35.139mb -0.00%  | 355.941ms -0.56% | ±0.41% -26.03%  |
| CSVExtractorBench     | bench_extract_10k | 1    | 3   | 4.789mb +0.02%   | 270.159ms -1.01% | ±1.84% +5.54%   |
| JsonExtractorBench    | bench_extract_10k | 1    | 3   | 4.937mb +0.02%   | 582.020ms +1.27% | ±0.54% +239.01% |
| ParquetExtractorBench | bench_extract_10k | 1    | 3   | 233.514mb +0.01% | 772.229ms -1.12% | ±0.50% +0.61%   |
| TextExtractorBench    | bench_extract_10k | 1    | 3   | 4.783mb +0.02%   | 19.389ms +0.46%  | ±2.47% +117.59% |
| XmlExtractorBench     | bench_extract_10k | 1    | 3   | 4.783mb +0.02%   | 399.876ms -0.55% | ±0.34% +8.90%   |
+-----------------------+-------------------+------+-----+------------------+------------------+-----------------+

Transformers

+-----------------------------+--------------------------+------+-----+-----------------+-----------------+----------------+
| benchmark                   | subject                  | revs | its | mem_peak        | mode            | rstdev         |
+-----------------------------+--------------------------+------+-----+-----------------+-----------------+----------------+
| RenameEntryTransformerBench | bench_transform_10k_rows | 1    | 3   | 87.049mb +0.00% | 48.917ms +0.44% | ±0.17% +19.80% |
+-----------------------------+--------------------------+------+-----+-----------------+-----------------+----------------+

Loaders

+--------------------+----------------+------+-----+------------------+------------------+-----------------+
| benchmark          | subject        | revs | its | mem_peak         | mode             | rstdev          |
+--------------------+----------------+------+-----+------------------+------------------+-----------------+
| AvroLoaderBench    | bench_load_10k | 1    | 3   | 93.289mb +0.00%  | 571.405ms +1.27% | ±0.80% +86.92%  |
| CSVLoaderBench     | bench_load_10k | 1    | 3   | 46.066mb +0.00%  | 68.451ms -0.13%  | ±1.00% +10.58%  |
| JsonLoaderBench    | bench_load_10k | 1    | 3   | 88.568mb +0.00%  | 60.770ms -2.07%  | ±1.01% +9.27%   |
| ParquetLoaderBench | bench_load_10k | 1    | 3   | 284.027mb +0.00% | 1.200s +2.04%    | ±1.20% +316.29% |
| TextLoaderBench    | bench_load_10k | 1    | 3   | 16.559mb +0.01%  | 41.151ms +1.11%  | ±0.59% +42.07%  |
+--------------------+----------------+------+-----+------------------+------------------+-----------------+

Building Blocks

+-------------------------+----------------------------+------+-----+-----------------+------------------+------------------+
| benchmark               | subject                    | revs | its | mem_peak        | mode             | rstdev           |
+-------------------------+----------------------------+------+-----+-----------------+------------------+------------------+
| RowsBench               | bench_chunk_10_on_10k      | 2    | 3   | 60.659mb +0.00% | 2.306ms -0.90%   | ±2.12% +14.98%   |
| RowsBench               | bench_diff_left_1k_on_10k  | 2    | 3   | 80.451mb +0.00% | 152.495ms +1.16% | ±0.25% -26.45%   |
| RowsBench               | bench_diff_right_1k_on_10k | 2    | 3   | 58.977mb +0.00% | 15.167ms +0.77%  | ±0.51% +103.70%  |
| RowsBench               | bench_drop_1k_on_10k       | 2    | 3   | 59.797mb +0.00% | 1.869ms -0.74%   | ±1.53% +130.00%  |
| RowsBench               | bench_drop_right_1k_on_10k | 2    | 3   | 59.797mb +0.00% | 1.998ms +18.16%  | ±3.39% +178.65%  |
| RowsBench               | bench_entries_on_10k       | 2    | 3   | 59.011mb +0.00% | 2.633ms +0.07%   | ±2.58% +620.56%  |
| RowsBench               | bench_filter_on_10k        | 2    | 3   | 59.540mb +0.00% | 14.498ms -3.56%  | ±0.55% -74.44%   |
| RowsBench               | bench_find_on_10k          | 2    | 3   | 59.539mb +0.00% | 14.803ms -1.31%  | ±2.41% -9.58%    |
| RowsBench               | bench_find_one_on_10k      | 10   | 3   | 57.610mb +0.00% | 1.694μs -6.21%   | ±2.83% +10.00%   |
| RowsBench               | bench_first_on_10k         | 10   | 3   | 57.610mb +0.00% | 0.400μs 0.00%    | ±0.00% 0.00%     |
| RowsBench               | bench_flat_map_on_1k       | 2    | 3   | 65.844mb +0.00% | 10.429ms +1.29%  | ±0.86% +22.64%   |
| RowsBench               | bench_map_on_10k           | 2    | 3   | 91.364mb +0.00% | 47.841ms +2.00%  | ±0.83% -30.86%   |
| RowsBench               | bench_merge_1k_on_10k      | 2    | 3   | 60.060mb +0.00% | 2.222ms +19.39%  | ±0.90% -37.81%   |
| RowsBench               | bench_partition_by_on_10k  | 2    | 3   | 62.330mb +0.00% | 33.983ms +0.83%  | ±0.27% -59.00%   |
| RowsBench               | bench_remove_on_10k        | 2    | 3   | 62.161mb +0.00% | 5.174ms +7.59%   | ±0.68% -73.61%   |
| RowsBench               | bench_sort_asc_on_1k       | 2    | 3   | 57.610mb +0.00% | 36.956ms +1.15%  | ±2.71% +289.04%  |
| RowsBench               | bench_sort_by_on_1k        | 2    | 3   | 57.610mb +0.00% | 36.929ms -0.04%  | ±0.70% +48.34%   |
| RowsBench               | bench_sort_desc_on_1k      | 2    | 3   | 57.610mb +0.00% | 37.686ms +2.42%  | ±1.23% +70.24%   |
| RowsBench               | bench_sort_entries_on_1k   | 2    | 3   | 59.885mb +0.00% | 7.272ms +0.49%   | ±0.90% -71.20%   |
| RowsBench               | bench_sort_on_1k           | 2    | 3   | 57.610mb +0.00% | 28.583ms +1.19%  | ±1.13% +1319.07% |
| RowsBench               | bench_take_1k_on_10k       | 10   | 3   | 57.610mb +0.00% | 13.357μs +2.48%  | ±2.49% +57.35%   |
| RowsBench               | bench_take_right_1k_on_10k | 10   | 3   | 57.610mb +0.00% | 15.394μs -0.04%  | ±0.31% -42.14%   |
| RowsBench               | bench_unique_on_1k         | 2    | 3   | 80.451mb +0.00% | 157.237ms -0.54% | ±0.75% +16.48%   |
| NativeEntryFactoryBench | bench_entry_factory        | 1    | 3   | 91.706mb +0.03% | 117.259ms +0.09% | ±0.89% -72.58%   |
| NativeEntryFactoryBench | bench_entry_factory        | 1    | 3   | 47.595mb +0.00% | 60.305ms +6.04%  | ±3.12% +1197.80% |
| NativeEntryFactoryBench | bench_entry_factory        | 1    | 3   | 12.392mb -0.04% | 14.114ms +2.43%  | ±1.44% -22.86%   |
+-------------------------+----------------------------+------+-----+-----------------+------------------+------------------+