Bug/parquet empty nested structures

[norberttech]

Change Log

Added

Fixed

• Parquet flattened for empty nested structures
• Parquet RLEHybrid bitWidth calculation

Changed

• Parquet - reading byte arrays now returns strings when option byte string as array is set

Removed

• BytesToString data converted as it became redundant

Deprecated

Security

---

Description

[norberttech]

values for those tests were generated with broken shredding logic, they are no longer valid and useful as it's pretty much impossible to understand whats going on here.

[github-actions]

Flow PHP - Benchmarks

^{_{Results of the benchmarks from this PR are compared with the results from 1.x branch.}}

Extractors

+-----------------------+-------------------+------+-----+-------------------+-------------------+-----------------+
| benchmark             | subject           | revs | its | mem_peak          | mode              | rstdev          |
+-----------------------+-------------------+------+-----+-------------------+-------------------+-----------------+
| CSVExtractorBench     | bench_extract_10k | 1    | 3   | 3.914mb +0.03%    | 511.371ms +0.01%  | ±2.40% +12.73%  |
| JsonExtractorBench    | bench_extract_10k | 1    | 3   | 3.947mb +0.03%    | 1.072s -0.23%     | ±0.97% +6.66%   |
| ParquetExtractorBench | bench_extract_10k | 1    | 3   | 135.385mb -78.94% | 428.823ms -42.89% | ±0.71% +139.13% |
| TextExtractorBench    | bench_extract_10k | 1    | 3   | 3.674mb +0.03%    | 33.940ms -0.17%   | ±0.20% -56.42%  |
| XmlExtractorBench     | bench_extract_10k | 1    | 3   | 3.621mb +0.03%    | 437.850ms -0.02%  | ±0.83% -66.28%  |
+-----------------------+-------------------+------+-----+-------------------+-------------------+-----------------+

Transformers

+-----------------------------+--------------------------+------+-----+------------------+-----------------+----------------+
| benchmark                   | subject                  | revs | its | mem_peak         | mode            | rstdev         |
+-----------------------------+--------------------------+------+-----+------------------+-----------------+----------------+
| RenameEntryTransformerBench | bench_transform_10k_rows | 1    | 3   | 115.966mb +0.00% | 60.742ms -0.00% | ±0.97% +50.54% |
+-----------------------------+--------------------------+------+-----+------------------+-----------------+----------------+

Loaders

+--------------------+----------------+------+-----+-------------------+-----------------+----------------+
| benchmark          | subject        | revs | its | mem_peak          | mode            | rstdev         |
+--------------------+----------------+------+-----+-------------------+-----------------+----------------+
| CSVLoaderBench     | bench_load_10k | 1    | 3   | 54.069mb +0.00%   | 85.567ms -0.10% | ±1.20% -17.47% |
| JsonLoaderBench    | bench_load_10k | 1    | 3   | 106.502mb +0.00%  | 53.170ms +0.46% | ±0.68% -32.18% |
| ParquetLoaderBench | bench_load_10k | 1    | 3   | 225.834mb -45.19% | 1.252s -17.14%  | ±0.38% -74.56% |
| TextLoaderBench    | bench_load_10k | 1    | 3   | 16.863mb +0.01%   | 44.680ms +0.65% | ±0.27% -17.11% |
+--------------------+----------------+------+-----+-------------------+-----------------+----------------+

Building Blocks

+-------------------------+----------------------------+------+-----+------------------+------------------+-----------------+
| benchmark               | subject                    | revs | its | mem_peak         | mode             | rstdev          |
+-------------------------+----------------------------+------+-----+------------------+------------------+-----------------+
| TypeDetectorBench       | bench_type_detector        | 1    | 3   | 59.695mb +0.00%  | 436.050ms -0.30% | ±0.83% -74.67%  |
| TypeDetectorBench       | bench_type_detector        | 1    | 3   | 14.234mb +0.01%  | 87.922ms +0.73%  | ±2.56% +265.96% |
| NativeEntryFactoryBench | bench_entry_factory        | 1    | 3   | 116.517mb +0.00% | 493.933ms -1.56% | ±0.32% -89.78%  |
| NativeEntryFactoryBench | bench_entry_factory        | 1    | 3   | 59.995mb +0.00%  | 251.460ms -0.60% | ±1.73% +40.78%  |
| NativeEntryFactoryBench | bench_entry_factory        | 1    | 3   | 14.928mb +0.01%  | 53.834ms -1.08%  | ±0.79% +708.66% |
| RowsBench               | bench_chunk_10_on_10k      | 2    | 3   | 86.786mb +0.00%  | 3.692ms +1.98%   | ±1.40% -38.44%  |
| RowsBench               | bench_diff_left_1k_on_10k  | 2    | 3   | 102.385mb +0.00% | 189.468ms -1.33% | ±0.95% -15.59%  |
| RowsBench               | bench_diff_right_1k_on_10k | 2    | 3   | 85.105mb +0.00%  | 19.457ms +1.01%  | ±0.54% -41.52%  |
| RowsBench               | bench_drop_1k_on_10k       | 2    | 3   | 88.026mb +0.00%  | 2.067ms -6.04%   | ±1.48% +230.29% |
| RowsBench               | bench_drop_right_1k_on_10k | 2    | 3   | 88.026mb +0.00%  | 2.010ms -9.51%   | ±2.69% +11.02%  |
| RowsBench               | bench_entries_on_10k       | 2    | 3   | 85.139mb +0.00%  | 3.166ms +7.23%   | ±2.64% -3.39%   |
| RowsBench               | bench_filter_on_10k        | 2    | 3   | 85.667mb +0.00%  | 16.843ms +9.27%  | ±0.49% -28.04%  |
| RowsBench               | bench_find_on_10k          | 2    | 3   | 85.667mb +0.00%  | 16.782ms +7.30%  | ±2.29% +61.54%  |
| RowsBench               | bench_find_one_on_10k      | 10   | 3   | 83.572mb +0.00%  | 2.006μs +0.61%   | ±2.32% -3.28%   |
| RowsBench               | bench_first_on_10k         | 10   | 3   | 83.572mb +0.00%  | 0.400μs +33.33%  | ±0.00% +0.00%   |
| RowsBench               | bench_flat_map_on_1k       | 2    | 3   | 92.922mb +0.00%  | 13.342ms +3.17%  | ±0.98% -67.04%  |
| RowsBench               | bench_map_on_10k           | 2    | 3   | 122.292mb +0.00% | 61.864ms -0.57%  | ±1.16% +21.00%  |
| RowsBench               | bench_merge_1k_on_10k      | 2    | 3   | 86.187mb +0.00%  | 1.670ms -8.06%   | ±3.51% +185.43% |
| RowsBench               | bench_partition_by_on_10k  | 2    | 3   | 89.534mb +0.00%  | 66.415ms +0.31%  | ±0.13% -86.17%  |
| RowsBench               | bench_remove_on_10k        | 2    | 3   | 88.289mb +0.00%  | 4.892ms +9.17%   | ±3.43% +39.02%  |
| RowsBench               | bench_sort_asc_on_1k       | 2    | 3   | 83.715mb +0.00%  | 42.548ms +4.25%  | ±0.12% -93.49%  |
| RowsBench               | bench_sort_by_on_1k        | 2    | 3   | 83.716mb +0.00%  | 41.696ms +2.59%  | ±1.00% -22.85%  |
| RowsBench               | bench_sort_desc_on_1k      | 2    | 3   | 83.715mb +0.00%  | 42.334ms +3.76%  | ±1.03% -59.67%  |
| RowsBench               | bench_sort_entries_on_1k   | 2    | 3   | 86.013mb +0.00%  | 7.667ms +1.35%   | ±1.76% +17.12%  |
| RowsBench               | bench_sort_on_1k           | 2    | 3   | 83.572mb +0.00%  | 30.243ms +1.09%  | ±1.40% -42.95%  |
| RowsBench               | bench_take_1k_on_10k       | 10   | 3   | 83.572mb +0.00%  | 14.225μs -9.98%  | ±3.43% +50.16%  |
| RowsBench               | bench_take_right_1k_on_10k | 10   | 3   | 83.572mb +0.00%  | 16.400μs -1.10%  | ±0.00% -100.00% |
| RowsBench               | bench_unique_on_1k         | 2    | 3   | 102.386mb +0.00% | 194.901ms +1.12% | ±0.81% +22.51%  |
+-------------------------+----------------------------+------+-----+------------------+------------------+-----------------+