Expanded Parquet schema converter in order to support deeply nested data types

[norberttech]

Change Log

Added

Fixed

Changed

• Expanded Parquet schema converter in order to support deeply nested data types

Removed

Deprecated

Security

---

Description

Closes: #756

[github-actions]

Flow PHP - Benchmarks

^{_{Results of the benchmarks from this PR are compared with the results from 1.x branch.}}

Extractors

+-----------------------+-------------------+------+-----+------------------+------------------+-----------------+
| benchmark             | subject           | revs | its | mem_peak         | mode             | rstdev          |
+-----------------------+-------------------+------+-----+------------------+------------------+-----------------+
| AvroExtractorBench    | bench_extract_10k | 1    | 3   | 34.749mb -0.00%  | 1.162s +4.05%    | ±1.12% +679.36% |
| CSVExtractorBench     | bench_extract_10k | 1    | 3   | 4.605mb +0.00%   | 302.345ms -0.55% | ±0.60% -83.60%  |
| JsonExtractorBench    | bench_extract_10k | 1    | 3   | 4.772mb -0.01%   | 1.376s +0.62%    | ±0.14% -41.21%  |
| ParquetExtractorBench | bench_extract_10k | 1    | 3   | 239.477mb -0.00% | 1.572s +1.91%    | ±0.48% -52.64%  |
| TextExtractorBench    | bench_extract_10k | 1    | 3   | 4.558mb -0.00%   | 25.026ms +6.31%  | ±1.92% +165.92% |
| XmlExtractorBench     | bench_extract_10k | 1    | 3   | 4.558mb -0.00%   | 405.792ms +1.89% | ±2.75% -5.82%   |
+-----------------------+-------------------+------+-----+------------------+------------------+-----------------+

Transformers

+-----------------------------+--------------------------+------+-----+------------------+-----------------+----------------+
| benchmark                   | subject                  | revs | its | mem_peak         | mode            | rstdev         |
+-----------------------------+--------------------------+------+-----+------------------+-----------------+----------------+
| RenameEntryTransformerBench | bench_transform_10k_rows | 1    | 3   | 110.249mb +0.00% | 63.528ms +0.47% | ±0.87% -68.30% |
+-----------------------------+--------------------------+------+-----+------------------+-----------------+----------------+

Loaders

+--------------------+----------------+------+-----+------------------+------------------+----------------+
| benchmark          | subject        | revs | its | mem_peak         | mode             | rstdev         |
+--------------------+----------------+------+-----+------------------+------------------+----------------+
| AvroLoaderBench    | bench_load_10k | 1    | 3   | 94.729mb -0.00%  | 446.529ms +5.96% | ±1.63% -12.55% |
| CSVLoaderBench     | bench_load_10k | 1    | 3   | 54.711mb +0.00%  | 70.907ms +10.81% | ±0.87% -54.76% |
| JsonLoaderBench    | bench_load_10k | 1    | 3   | 105.312mb -0.00% | 54.588ms +3.38%  | ±0.34% -84.07% |
| ParquetLoaderBench | bench_load_10k | 1    | 3   | 320.784mb -0.00% | 1.419s +5.05%    | ±0.14% -87.57% |
| TextLoaderBench    | bench_load_10k | 1    | 3   | 17.589mb +0.00%  | 39.867ms +5.69%  | ±0.54% -73.27% |
+--------------------+----------------+------+-----+------------------+------------------+----------------+

Building Blocks

+-------------------------+----------------------------+------+-----+------------------+------------------+-----------------+
| benchmark               | subject                    | revs | its | mem_peak         | mode             | rstdev          |
+-------------------------+----------------------------+------+-----+------------------+------------------+-----------------+
| RowsBench               | bench_chunk_10_on_10k      | 2    | 3   | 76.296mb +0.00%  | 2.409ms +17.55%  | ±3.48% +71.02%  |
| RowsBench               | bench_diff_left_1k_on_10k  | 2    | 3   | 96.086mb +0.00%  | 181.754ms +5.03% | ±1.05% -42.87%  |
| RowsBench               | bench_diff_right_1k_on_10k | 2    | 3   | 74.612mb +0.00%  | 18.002ms +4.89%  | ±0.46% -84.68%  |
| RowsBench               | bench_drop_1k_on_10k       | 2    | 3   | 75.434mb +0.00%  | 1.606ms +5.85%   | ±1.35% +47.73%  |
| RowsBench               | bench_drop_right_1k_on_10k | 2    | 3   | 75.434mb +0.00%  | 1.632ms +10.01%  | ±3.33% +111.00% |
| RowsBench               | bench_entries_on_10k       | 2    | 3   | 74.648mb +0.00%  | 2.551ms +10.75%  | ±2.27% -30.57%  |
| RowsBench               | bench_filter_on_10k        | 2    | 3   | 75.177mb +0.00%  | 13.719ms +3.18%  | ±0.68% -38.38%  |
| RowsBench               | bench_find_on_10k          | 2    | 3   | 75.176mb +0.00%  | 13.967ms +4.73%  | ±1.08% -16.55%  |
| RowsBench               | bench_find_one_on_10k      | 10   | 3   | 73.080mb +0.00%  | 1.606μs -5.19%   | ±2.89% +2.04%   |
| RowsBench               | bench_first_on_10k         | 10   | 3   | 73.080mb +0.00%  | 0.300μs -25.00%  | ±0.00% -100.00% |
| RowsBench               | bench_flat_map_on_1k       | 2    | 3   | 86.638mb +0.00%  | 12.873ms +1.00%  | ±1.03% -56.89%  |
| RowsBench               | bench_map_on_10k           | 2    | 3   | 115.995mb +0.00% | 63.712ms -0.71%  | ±0.67% +25.68%  |
| RowsBench               | bench_merge_1k_on_10k      | 2    | 3   | 75.697mb +0.00%  | 1.835ms +8.25%   | ±2.61% +68.66%  |
| RowsBench               | bench_partition_by_on_10k  | 2    | 3   | 77.965mb +0.00%  | 34.777ms +7.52%  | ±0.90% -26.13%  |
| RowsBench               | bench_remove_on_10k        | 2    | 3   | 77.798mb +0.00%  | 4.348ms +1.81%   | ±1.16% -59.01%  |
| RowsBench               | bench_sort_asc_on_1k       | 2    | 3   | 73.223mb +0.00%  | 38.629ms -0.77%  | ±1.55% +371.63% |
| RowsBench               | bench_sort_by_on_1k        | 2    | 3   | 73.223mb +0.00%  | 38.970ms -0.21%  | ±0.55% -47.88%  |
| RowsBench               | bench_sort_desc_on_1k      | 2    | 3   | 73.223mb +0.00%  | 38.686ms -2.93%  | ±1.56% +159.66% |
| RowsBench               | bench_sort_entries_on_1k   | 2    | 3   | 75.522mb +0.00%  | 7.328ms +0.96%   | ±0.40% -7.94%   |
| RowsBench               | bench_sort_on_1k           | 2    | 3   | 73.079mb +0.00%  | 28.074ms -3.09%  | ±2.17% +5.42%   |
| RowsBench               | bench_take_1k_on_10k       | 10   | 3   | 73.080mb +0.00%  | 12.806μs -0.74%  | ±0.37% -86.89%  |
| RowsBench               | bench_take_right_1k_on_10k | 10   | 3   | 73.080mb +0.00%  | 15.424μs +0.55%  | ±1.21% -61.20%  |
| RowsBench               | bench_unique_on_1k         | 2    | 3   | 96.087mb +0.00%  | 180.052ms +2.75% | ±0.78% +139.45% |
| NativeEntryFactoryBench | bench_entry_factory        | 1    | 3   | 115.842mb +0.00% | 775.827ms +0.75% | ±1.55% +46.61%  |
| NativeEntryFactoryBench | bench_entry_factory        | 1    | 3   | 59.560mb +0.00%  | 382.029ms +1.98% | ±1.03% -5.43%   |
| NativeEntryFactoryBench | bench_entry_factory        | 1    | 3   | 14.683mb +0.00%  | 78.313ms -0.38%  | ±0.69% -16.78%  |
+-------------------------+----------------------------+------+-----+------------------+------------------+-----------------+