Covered more parquet edge cases

[norberttech]

Change Log

Added

Fixed

• Covered additional parquet edge cases

Changed

Removed

Deprecated

Security

---

Description

These example files are coming from https://github.com/apache/parquet-testing

Whatever our reader can't read (except unsupported encodings/compressions) was moved to the repository and fixed.

[github-actions]

Flow PHP - Benchmarks

^{_{Results of the benchmarks from this PR are compared with the results from 1.x branch.}}

Extractors

+-----------------------+-------------------+------+-----+------------------+------------------+-----------------+
| benchmark             | subject           | revs | its | mem_peak         | mode             | rstdev          |
+-----------------------+-------------------+------+-----+------------------+------------------+-----------------+
| AvroExtractorBench    | bench_extract_10k | 1    | 3   | 35.271mb +0.00%  | 359.797ms -3.01% | ±0.09% -96.58%  |
| CSVExtractorBench     | bench_extract_10k | 1    | 3   | 4.837mb +0.01%   | 268.052ms -0.93% | ±1.58% +675.46% |
| JsonExtractorBench    | bench_extract_10k | 1    | 3   | 5.008mb +0.01%   | 578.304ms -1.27% | ±1.00% +387.47% |
| ParquetExtractorBench | bench_extract_10k | 1    | 3   | 239.704mb +0.00% | 729.855ms -2.19% | ±1.87% +3.01%   |
| TextExtractorBench    | bench_extract_10k | 1    | 3   | 4.831mb +0.01%   | 18.961ms -1.64%  | ±2.38% +95.18%  |
| XmlExtractorBench     | bench_extract_10k | 1    | 3   | 4.831mb +0.01%   | 403.956ms +0.89% | ±0.36% -35.02%  |
+-----------------------+-------------------+------+-----+------------------+------------------+-----------------+

Transformers

+-----------------------------+--------------------------+------+-----+-----------------+-----------------+-----------------+
| benchmark                   | subject                  | revs | its | mem_peak        | mode            | rstdev          |
+-----------------------------+--------------------------+------+-----+-----------------+-----------------+-----------------+
| RenameEntryTransformerBench | bench_transform_10k_rows | 1    | 3   | 87.098mb +0.00% | 47.878ms -3.32% | ±1.26% +200.09% |
+-----------------------------+--------------------------+------+-----+-----------------+-----------------+-----------------+

Loaders

+--------------------+----------------+------+-----+------------------+------------------+-----------------+
| benchmark          | subject        | revs | its | mem_peak         | mode             | rstdev          |
+--------------------+----------------+------+-----+------------------+------------------+-----------------+
| AvroLoaderBench    | bench_load_10k | 1    | 3   | 95.281mb +0.00%  | 578.960ms +0.12% | ±0.39% -45.60%  |
| CSVLoaderBench     | bench_load_10k | 1    | 3   | 46.114mb +0.00%  | 69.702ms +0.13%  | ±0.56% -68.43%  |
| JsonLoaderBench    | bench_load_10k | 1    | 3   | 90.559mb +0.00%  | 64.562ms +1.56%  | ±1.98% +404.70% |
| ParquetLoaderBench | bench_load_10k | 1    | 3   | 292.402mb +0.00% | 1.534s -0.26%    | ±0.31% -67.16%  |
| TextLoaderBench    | bench_load_10k | 1    | 3   | 16.608mb +0.00%  | 40.834ms +0.24%  | ±0.12% -73.18%  |
+--------------------+----------------+------+-----+------------------+------------------+-----------------+

Building Blocks

+-------------------------+----------------------------+------+-----+-----------------+------------------+-----------------+
| benchmark               | subject                    | revs | its | mem_peak        | mode             | rstdev          |
+-------------------------+----------------------------+------+-----+-----------------+------------------+-----------------+
| RowsBench               | bench_chunk_10_on_10k      | 2    | 3   | 60.707mb +0.00% | 2.455ms +8.08%   | ±1.69% +347.99% |
| RowsBench               | bench_diff_left_1k_on_10k  | 2    | 3   | 80.500mb +0.00% | 150.187ms -0.78% | ±0.86% +36.37%  |
| RowsBench               | bench_diff_right_1k_on_10k | 2    | 3   | 59.025mb +0.00% | 15.348ms +2.27%  | ±0.70% +1.55%   |
| RowsBench               | bench_drop_1k_on_10k       | 2    | 3   | 59.846mb +0.00% | 1.806ms +0.70%   | ±3.67% +50.46%  |
| RowsBench               | bench_drop_right_1k_on_10k | 2    | 3   | 59.846mb +0.00% | 1.604ms -9.33%   | ±2.28% +280.22% |
| RowsBench               | bench_entries_on_10k       | 2    | 3   | 59.059mb +0.00% | 2.874ms +7.73%   | ±2.43% +16.33%  |
| RowsBench               | bench_filter_on_10k        | 2    | 3   | 59.588mb +0.00% | 16.829ms +19.42% | ±0.26% -87.00%  |
| RowsBench               | bench_find_on_10k          | 2    | 3   | 59.588mb +0.00% | 16.486ms +13.83% | ±1.22% +8.21%   |
| RowsBench               | bench_find_one_on_10k      | 10   | 3   | 57.659mb +0.00% | 1.906μs +5.54%   | ±2.44% -5.17%   |
| RowsBench               | bench_first_on_10k         | 10   | 3   | 57.659mb +0.00% | 0.400μs 0.00%    | ±0.00% 0.00%    |
| RowsBench               | bench_flat_map_on_1k       | 2    | 3   | 65.893mb +0.00% | 10.329ms +1.07%  | ±1.33% -0.10%   |
| RowsBench               | bench_map_on_10k           | 2    | 3   | 91.412mb +0.00% | 49.428ms +3.04%  | ±1.42% +159.37% |
| RowsBench               | bench_merge_1k_on_10k      | 2    | 3   | 60.109mb +0.00% | 1.992ms +1.89%   | ±1.21% -61.75%  |
| RowsBench               | bench_partition_by_on_10k  | 2    | 3   | 62.379mb +0.00% | 38.373ms +12.81% | ±0.31% -76.30%  |
| RowsBench               | bench_remove_on_10k        | 2    | 3   | 62.209mb +0.00% | 4.576ms -4.87%   | ±3.23% +114.33% |
| RowsBench               | bench_sort_asc_on_1k       | 2    | 3   | 57.659mb +0.00% | 37.693ms -0.32%  | ±1.66% +98.02%  |
| RowsBench               | bench_sort_by_on_1k        | 2    | 3   | 57.659mb +0.00% | 38.958ms +2.84%  | ±2.91% +295.72% |
| RowsBench               | bench_sort_desc_on_1k      | 2    | 3   | 57.659mb +0.00% | 38.042ms +0.57%  | ±2.59% +28.15%  |
| RowsBench               | bench_sort_entries_on_1k   | 2    | 3   | 59.934mb +0.00% | 7.225ms -0.42%   | ±0.89% -13.59%  |
| RowsBench               | bench_sort_on_1k           | 2    | 3   | 57.659mb +0.00% | 28.620ms +1.82%  | ±0.33% -39.33%  |
| RowsBench               | bench_take_1k_on_10k       | 10   | 3   | 57.659mb +0.00% | 12.906μs -0.88%  | ±0.36% -62.11%  |
| RowsBench               | bench_take_right_1k_on_10k | 10   | 3   | 57.659mb +0.00% | 15.806μs +2.60%  | ±0.30% -2.53%   |
| RowsBench               | bench_unique_on_1k         | 2    | 3   | 80.500mb +0.00% | 156.000ms +0.15% | ±0.36% -86.29%  |
| NativeEntryFactoryBench | bench_entry_factory        | 1    | 3   | 93.723mb +0.00% | 122.849ms -0.37% | ±1.06% -14.36%  |
| NativeEntryFactoryBench | bench_entry_factory        | 1    | 3   | 48.638mb +0.00% | 63.219ms +0.19%  | ±1.38% +42.98%  |
| NativeEntryFactoryBench | bench_entry_factory        | 1    | 3   | 12.654mb -0.05% | 14.712ms -1.17%  | ±1.10% +39.38%  |
+-------------------------+----------------------------+------+-----+-----------------+------------------+-----------------+