Parquet - write column chunk statistics

[norberttech]

Change Log

Added

• write column chunk statistics

Fixed

Changed

Removed

Deprecated

Security

---

Refs: #686

Description

[github-actions]

Flow PHP - Benchmarks

^{_{Results of the benchmarks from this PR are compared with the results from 1.x branch.}}

Extractors

+-----------------------+-------------------+------+-----+------------------+------------------+-----------------+
| benchmark             | subject           | revs | its | mem_peak         | mode             | rstdev          |
+-----------------------+-------------------+------+-----+------------------+------------------+-----------------+
| AvroExtractorBench    | bench_extract_10k | 1    | 3   | 35.139mb +0.00%  | 377.569ms +3.99% | ±2.21% +328.41% |
| CSVExtractorBench     | bench_extract_10k | 1    | 3   | 4.789mb +0.02%   | 269.553ms -0.18% | ±0.17% -20.30%  |
| JsonExtractorBench    | bench_extract_10k | 1    | 3   | 4.937mb +0.02%   | 579.606ms +1.32% | ±1.37% +35.38%  |
| ParquetExtractorBench | bench_extract_10k | 1    | 3   | 233.526mb +0.00% | 775.532ms -0.29% | ±0.88% +461.80% |
| TextExtractorBench    | bench_extract_10k | 1    | 3   | 4.783mb +0.02%   | 19.079ms -1.81%  | ±0.89% -42.22%  |
| XmlExtractorBench     | bench_extract_10k | 1    | 3   | 4.783mb +0.02%   | 402.718ms +0.50% | ±0.62% -53.96%  |
+-----------------------+-------------------+------+-----+------------------+------------------+-----------------+

Transformers

+-----------------------------+--------------------------+------+-----+-----------------+-----------------+-----------------+
| benchmark                   | subject                  | revs | its | mem_peak        | mode            | rstdev          |
+-----------------------------+--------------------------+------+-----+-----------------+-----------------+-----------------+
| RenameEntryTransformerBench | bench_transform_10k_rows | 1    | 3   | 87.049mb +0.00% | 48.681ms +0.45% | ±1.10% +140.28% |
+-----------------------------+--------------------------+------+-----+-----------------+-----------------+-----------------+

Loaders

+--------------------+----------------+------+-----+------------------+------------------+-----------------+
| benchmark          | subject        | revs | its | mem_peak         | mode             | rstdev          |
+--------------------+----------------+------+-----+------------------+------------------+-----------------+
| AvroLoaderBench    | bench_load_10k | 1    | 3   | 93.290mb +0.00%  | 573.563ms +2.52% | ±0.97% -36.04%  |
| CSVLoaderBench     | bench_load_10k | 1    | 3   | 46.066mb +0.00%  | 70.511ms -1.59%  | ±0.46% -53.42%  |
| JsonLoaderBench    | bench_load_10k | 1    | 3   | 88.568mb +0.00%  | 61.719ms +0.78%  | ±0.22% -76.30%  |
| ParquetLoaderBench | bench_load_10k | 1    | 3   | 284.039mb +0.00% | 1.228s +4.85%    | ±1.72% +176.86% |
| TextLoaderBench    | bench_load_10k | 1    | 3   | 16.560mb +0.01%  | 40.674ms -1.06%  | ±0.47% +86.39%  |
+--------------------+----------------+------+-----+------------------+------------------+-----------------+

Building Blocks

+-------------------------+----------------------------+------+-----+-----------------+------------------+-----------------+
| benchmark               | subject                    | revs | its | mem_peak        | mode             | rstdev          |
+-------------------------+----------------------------+------+-----+-----------------+------------------+-----------------+
| RowsBench               | bench_chunk_10_on_10k      | 2    | 3   | 60.659mb +0.00% | 2.381ms +6.86%   | ±0.82% -75.62%  |
| RowsBench               | bench_diff_left_1k_on_10k  | 2    | 3   | 80.451mb +0.00% | 154.051ms +4.42% | ±1.14% +207.35% |
| RowsBench               | bench_diff_right_1k_on_10k | 2    | 3   | 58.977mb +0.00% | 14.973ms +1.13%  | ±1.42% +163.53% |
| RowsBench               | bench_drop_1k_on_10k       | 2    | 3   | 59.797mb +0.00% | 1.947ms +4.32%   | ±2.54% -13.85%  |
| RowsBench               | bench_drop_right_1k_on_10k | 2    | 3   | 59.797mb +0.00% | 1.987ms +10.74%  | ±3.82% +726.62% |
| RowsBench               | bench_entries_on_10k       | 2    | 3   | 59.011mb +0.00% | 2.858ms +7.52%   | ±0.53% -60.97%  |
| RowsBench               | bench_filter_on_10k        | 2    | 3   | 59.540mb +0.00% | 14.594ms +4.54%  | ±1.17% +11.91%  |
| RowsBench               | bench_find_on_10k          | 2    | 3   | 59.539mb +0.00% | 14.850ms +6.31%  | ±0.53% -9.69%   |
| RowsBench               | bench_find_one_on_10k      | 10   | 3   | 57.611mb +0.00% | 1.894μs +5.57%   | ±2.53% -5.36%   |
| RowsBench               | bench_first_on_10k         | 10   | 3   | 57.611mb +0.00% | 0.400μs 0.00%    | ±0.00% 0.00%    |
| RowsBench               | bench_flat_map_on_1k       | 2    | 3   | 65.844mb +0.00% | 10.539ms +3.43%  | ±0.54% +40.60%  |
| RowsBench               | bench_map_on_10k           | 2    | 3   | 91.364mb +0.00% | 49.332ms -0.58%  | ±0.55% -46.76%  |
| RowsBench               | bench_merge_1k_on_10k      | 2    | 3   | 60.060mb +0.00% | 2.219ms +14.82%  | ±2.16% +93.53%  |
| RowsBench               | bench_partition_by_on_10k  | 2    | 3   | 62.330mb +0.00% | 33.630ms +1.25%  | ±1.33% -19.54%  |
| RowsBench               | bench_remove_on_10k        | 2    | 3   | 62.161mb +0.00% | 4.884ms +2.81%   | ±0.99% -45.45%  |
| RowsBench               | bench_sort_asc_on_1k       | 2    | 3   | 57.611mb +0.00% | 37.543ms +1.65%  | ±0.85% +118.34% |
| RowsBench               | bench_sort_by_on_1k        | 2    | 3   | 57.611mb +0.00% | 36.851ms -0.20%  | ±1.62% +441.66% |
| RowsBench               | bench_sort_desc_on_1k      | 2    | 3   | 57.611mb +0.00% | 37.856ms +1.94%  | ±0.92% +56.22%  |
| RowsBench               | bench_sort_entries_on_1k   | 2    | 3   | 59.885mb +0.00% | 7.263ms -0.14%   | ±0.99% -58.07%  |
| RowsBench               | bench_sort_on_1k           | 2    | 3   | 57.610mb +0.00% | 28.199ms -1.65%  | ±0.10% -87.60%  |
| RowsBench               | bench_take_1k_on_10k       | 10   | 3   | 57.611mb +0.00% | 12.880μs +0.77%  | ±1.31% +17.71%  |
| RowsBench               | bench_take_right_1k_on_10k | 10   | 3   | 57.611mb +0.00% | 15.837μs +2.71%  | ±2.89% +216.72% |
| RowsBench               | bench_unique_on_1k         | 2    | 3   | 80.452mb +0.00% | 157.499ms +1.53% | ±0.92% +25.74%  |
| NativeEntryFactoryBench | bench_entry_factory        | 1    | 3   | 91.719mb -0.01% | 116.691ms +0.39% | ±0.51% -61.55%  |
| NativeEntryFactoryBench | bench_entry_factory        | 1    | 3   | 47.602mb +0.01% | 59.785ms -0.13%  | ±1.03% +302.85% |
| NativeEntryFactoryBench | bench_entry_factory        | 1    | 3   | 12.392mb +0.00% | 13.857ms -0.07%  | ±0.97% -66.39%  |
+-------------------------+----------------------------+------+-----+-----------------+------------------+-----------------+