Parquet - writing & reading nullable structures with nullable fields

[norberttech]

Change Log

Added

• Parquet - writing & reading nullable structures with nullable fields

Fixed

Changed

Removed

Deprecated

Security

---

Description

Ref: #575

[github-actions]

Flow PHP - Benchmarks

^{_{Results of the benchmarks from this PR are compared with the results from 1.x branch.}}

Extractors

+-----------------------+-------------------+------+-----+------------------+-------------------+
| benchmark             | subject           | revs | its | mem_peak         | mode              |
+-----------------------+-------------------+------+-----+------------------+-------------------+
| AvroExtractorBench    | bench_extract_10k | 5    | 3   | 44.121mb +0.00%  | 419.470ms -7.08%  |
| CSVExtractorBench     | bench_extract_10k | 5    | 3   | 14.005mb +0.00%  | 340.107ms -13.21% |
| JsonExtractorBench    | bench_extract_10k | 5    | 3   | 18.667mb +0.00%  | 668.453ms -15.86% |
| ParquetExtractorBench | bench_extract_10k | 5    | 3   | 237.791mb +0.00% | 920.264ms -13.59% |
| TextExtractorBench    | bench_extract_10k | 5    | 3   | 7.275mb +0.00%   | 13.861ms -2.49%   |
| XmlExtractorBench     | bench_extract_10k | 5    | 3   | 7.622mb +0.00%   | 557.368ms -17.03% |
+-----------------------+-------------------+------+-----+------------------+-------------------+

Transformers

+-----------------------------+--------------------------+------+-----+-----------------+------------------+
| benchmark                   | subject                  | revs | its | mem_peak        | mode             |
+-----------------------------+--------------------------+------+-----+-----------------+------------------+
| RenameEntryTransformerBench | bench_transform_10k_rows | 5    | 5   | 87.060mb +0.00% | 57.811ms -14.43% |
+-----------------------------+--------------------------+------+-----+-----------------+------------------+

Loaders

+--------------------+----------------+------+-----+------------------+-------------------+
| benchmark          | subject        | revs | its | mem_peak         | mode              |
+--------------------+----------------+------+-----+------------------+-------------------+
| AvroLoaderBench    | bench_load_10k | 5    | 3   | 219.954mb +0.00% | 661.083ms -10.95% |
| CSVLoaderBench     | bench_load_10k | 5    | 3   | 47.138mb +0.00%  | 69.471ms -6.84%   |
| JsonLoaderBench    | bench_load_10k | 5    | 3   | 88.578mb +0.00%  | 64.152ms -21.32%  |
| ParquetLoaderBench | bench_load_10k | 5    | 3   | 281.198mb +0.00% | 954.224ms -9.25%  |
| TextLoaderBench    | bench_load_10k | 5    | 3   | 16.564mb +0.00%  | 35.724ms -6.80%   |
+--------------------+----------------+------+-----+------------------+-------------------+

Building Blocks

+-------------------------+----------------------------+------+-----+-----------------+-------------------+
| benchmark               | subject                    | revs | its | mem_peak        | mode              |
+-------------------------+----------------------------+------+-----+-----------------+-------------------+
| RowsBench               | bench_chunk_10_on_10k      | 5    | 5   | 60.692mb +0.00% | 3.011ms -27.15%   |
| RowsBench               | bench_diff_left_1k_on_10k  | 5    | 5   | 80.484mb +0.00% | 166.147ms -7.83%  |
| RowsBench               | bench_diff_right_1k_on_10k | 5    | 5   | 59.009mb +0.00% | 16.609ms -7.71%   |
| RowsBench               | bench_drop_1k_on_10k       | 5    | 5   | 59.831mb +0.00% | 2.632ms -11.63%   |
| RowsBench               | bench_drop_right_1k_on_10k | 5    | 5   | 61.933mb +0.00% | 2.705ms -11.23%   |
| RowsBench               | bench_entries_on_10k       | 5    | 5   | 59.045mb +0.00% | 3.459ms -8.20%    |
| RowsBench               | bench_filter_on_10k        | 5    | 5   | 59.574mb +0.00% | 17.276ms -27.98%  |
| RowsBench               | bench_find_on_10k          | 5    | 5   | 59.574mb +0.00% | 17.510ms -25.17%  |
| RowsBench               | bench_find_one_on_10k      | 5    | 5   | 57.644mb +0.00% | 4.000μs +0.16%    |
| RowsBench               | bench_first_on_10k         | 5    | 5   | 57.644mb +0.00% | 0.800μs 0.00%     |
| RowsBench               | bench_flat_map_on_1k       | 5    | 5   | 65.878mb +0.00% | 13.139ms -4.97%   |
| RowsBench               | bench_map_on_10k           | 5    | 5   | 91.398mb +0.00% | 55.119ms -17.71%  |
| RowsBench               | bench_merge_1k_on_10k      | 5    | 5   | 60.094mb +0.00% | 2.857ms -17.02%   |
| RowsBench               | bench_partition_by_on_10k  | 5    | 5   | 62.363mb +0.00% | 37.138ms -36.91%  |
| RowsBench               | bench_remove_on_10k        | 5    | 5   | 62.195mb +0.00% | 6.457ms -22.11%   |
| RowsBench               | bench_sort_asc_on_1k       | 5    | 5   | 57.644mb +0.00% | 45.419ms -23.93%  |
| RowsBench               | bench_sort_by_on_1k        | 5    | 5   | 57.644mb +0.00% | 45.656ms -23.94%  |
| RowsBench               | bench_sort_desc_on_1k      | 5    | 5   | 57.644mb +0.00% | 45.758ms -22.74%  |
| RowsBench               | bench_sort_entries_on_1k   | 5    | 5   | 59.919mb +0.00% | 9.809ms -7.22%    |
| RowsBench               | bench_sort_on_1k           | 5    | 5   | 57.644mb +0.00% | 36.009ms -18.23%  |
| RowsBench               | bench_take_1k_on_10k       | 5    | 5   | 57.644mb +0.00% | 21.678μs -17.57%  |
| RowsBench               | bench_take_right_1k_on_10k | 5    | 5   | 57.645mb +0.00% | 27.105μs -15.76%  |
| RowsBench               | bench_unique_on_1k         | 5    | 5   | 80.484mb +0.00% | 168.085ms -11.42% |
| NativeEntryFactoryBench | bench_10k_rows             | 5    | 5   | 83.892mb +0.02% | 133.581ms -11.34% |
| NativeEntryFactoryBench | bench_1k_rows              | 5    | 5   | 45.426mb +0.00% | 13.593ms -11.50%  |
| NativeEntryFactoryBench | bench_5k_rows              | 5    | 5   | 62.718mb -0.02% | 67.001ms -15.36%  |
+-------------------------+----------------------------+------+-----+-----------------+-------------------+