Added pagination to reader

[norberttech]

Change Log

Added

• Parquet Pagination

Fixed

Changed

• Tuned default parquet options

Removed

Deprecated

Security

---

Description

Resolve #934

[github-actions]

Flow PHP - Benchmarks

^{_{Results of the benchmarks from this PR are compared with the results from 1.x branch.}}

Extractors

+-----------------------+-------------------+------+-----+------------------+------------------+-----------------+
| benchmark             | subject           | revs | its | mem_peak         | mode             | rstdev          |
+-----------------------+-------------------+------+-----+------------------+------------------+-----------------+
| AvroExtractorBench    | bench_extract_10k | 1    | 3   | 35.260mb +0.00%  | 827.006ms +4.93% | ±0.33% -11.26%  |
| CSVExtractorBench     | bench_extract_10k | 1    | 3   | 4.977mb +0.02%   | 341.295ms +2.63% | ±0.43% -56.45%  |
| JsonExtractorBench    | bench_extract_10k | 1    | 3   | 5.129mb +0.02%   | 1.044s +1.71%    | ±1.34% +7.13%   |
| ParquetExtractorBench | bench_extract_10k | 1    | 3   | 239.828mb +0.00% | 1.258s +3.35%    | ±1.11% +239.54% |
| TextExtractorBench    | bench_extract_10k | 1    | 3   | 4.898mb +0.02%   | 35.243ms +4.45%  | ±0.55% -74.89%  |
| XmlExtractorBench     | bench_extract_10k | 1    | 3   | 4.900mb +0.02%   | 434.585ms +1.84% | ±0.70% -51.21%  |
+-----------------------+-------------------+------+-----+------------------+------------------+-----------------+

Transformers

+-----------------------------+--------------------------+------+-----+------------------+-----------------+----------------+
| benchmark                   | subject                  | revs | its | mem_peak         | mode            | rstdev         |
+-----------------------------+--------------------------+------+-----+------------------+-----------------+----------------+
| RenameEntryTransformerBench | bench_transform_10k_rows | 1    | 3   | 110.600mb +0.00% | 64.355ms +6.94% | ±0.39% -64.01% |
+-----------------------------+--------------------------+------+-----+------------------+-----------------+----------------+

Loaders

+--------------------+----------------+------+-----+------------------+------------------+-----------------+
| benchmark          | subject        | revs | its | mem_peak         | mode             | rstdev          |
+--------------------+----------------+------+-----+------------------+------------------+-----------------+
| AvroLoaderBench    | bench_load_10k | 1    | 3   | 95.640mb +0.00%  | 457.477ms +2.84% | ±0.35% -77.77%  |
| CSVLoaderBench     | bench_load_10k | 1    | 3   | 54.114mb +0.00%  | 71.942ms +0.06%  | ±0.68% -57.93%  |
| JsonLoaderBench    | bench_load_10k | 1    | 3   | 106.545mb +0.00% | 52.756ms +2.31%  | ±0.74% +304.79% |
| ParquetLoaderBench | bench_load_10k | 1    | 3   | 321.750mb +0.00% | 1.470s +0.71%    | ±0.44% -33.84%  |
| TextLoaderBench    | bench_load_10k | 1    | 3   | 17.938mb +0.00%  | 40.996ms +0.59%  | ±0.74% -74.84%  |
+--------------------+----------------+------+-----+------------------+------------------+-----------------+

Building Blocks

+-------------------------+----------------------------+------+-----+------------------+------------------+-----------------+
| benchmark               | subject                    | revs | its | mem_peak         | mode             | rstdev          |
+-------------------------+----------------------------+------+-----+------------------+------------------+-----------------+
| RowsBench               | bench_chunk_10_on_10k      | 2    | 3   | 76.674mb +0.00%  | 3.389ms +0.42%   | ±3.08% -21.05%  |
| RowsBench               | bench_diff_left_1k_on_10k  | 2    | 3   | 96.401mb +0.00%  | 182.671ms +1.59% | ±1.22% +74.77%  |
| RowsBench               | bench_diff_right_1k_on_10k | 2    | 3   | 74.927mb +0.00%  | 18.106ms +2.30%  | ±0.57% -67.54%  |
| RowsBench               | bench_drop_1k_on_10k       | 2    | 3   | 77.914mb +0.00%  | 1.761ms +6.94%   | ±0.19% -92.22%  |
| RowsBench               | bench_drop_right_1k_on_10k | 2    | 3   | 77.914mb +0.00%  | 1.741ms +11.13%  | ±1.50% -30.50%  |
| RowsBench               | bench_entries_on_10k       | 2    | 3   | 75.027mb +0.00%  | 2.587ms -2.58%   | ±2.99% +258.84% |
| RowsBench               | bench_filter_on_10k        | 2    | 3   | 75.555mb +0.00%  | 14.811ms -1.40%  | ±1.39% +853.36% |
| RowsBench               | bench_find_on_10k          | 2    | 3   | 75.555mb +0.00%  | 14.938ms -0.10%  | ±1.19% +41.47%  |
| RowsBench               | bench_find_one_on_10k      | 10   | 3   | 73.460mb +0.00%  | 1.794μs +11.70%  | ±2.67% -7.55%   |
| RowsBench               | bench_first_on_10k         | 10   | 3   | 73.460mb +0.00%  | 0.400μs +33.33%  | ±0.00% +0.00%   |
| RowsBench               | bench_flat_map_on_1k       | 2    | 3   | 87.014mb +0.00%  | 12.940ms +3.70%  | ±0.07% -93.69%  |
| RowsBench               | bench_map_on_10k           | 2    | 3   | 116.375mb +0.00% | 64.475ms +4.77%  | ±0.55% +28.60%  |
| RowsBench               | bench_merge_1k_on_10k      | 2    | 3   | 76.075mb +0.00%  | 1.225ms +6.52%   | ±0.55% -83.88%  |
| RowsBench               | bench_partition_by_on_10k  | 2    | 3   | 79.422mb +0.00%  | 58.890ms +6.62%  | ±1.16% -13.55%  |
| RowsBench               | bench_remove_on_10k        | 2    | 3   | 78.177mb +0.00%  | 3.838ms +6.10%   | ±1.43% +88.05%  |
| RowsBench               | bench_sort_asc_on_1k       | 2    | 3   | 73.538mb +0.00%  | 39.271ms +3.59%  | ±1.13% -46.75%  |
| RowsBench               | bench_sort_by_on_1k        | 2    | 3   | 73.538mb +0.00%  | 41.569ms +4.91%  | ±2.05% -17.17%  |
| RowsBench               | bench_sort_desc_on_1k      | 2    | 3   | 73.538mb +0.00%  | 40.005ms +2.70%  | ±1.03% -55.08%  |
| RowsBench               | bench_sort_entries_on_1k   | 2    | 3   | 75.901mb +0.00%  | 7.304ms -1.40%   | ±1.02% -27.81%  |
| RowsBench               | bench_sort_on_1k           | 2    | 3   | 73.460mb +0.00%  | 29.149ms +1.66%  | ±0.84% -54.40%  |
| RowsBench               | bench_take_1k_on_10k       | 10   | 3   | 73.460mb +0.00%  | 13.304μs -0.58%  | ±2.98% +179.89% |
| RowsBench               | bench_take_right_1k_on_10k | 10   | 3   | 73.460mb +0.00%  | 15.880μs +0.37%  | ±1.06% +34.57%  |
| RowsBench               | bench_unique_on_1k         | 2    | 3   | 96.402mb +0.00%  | 183.282ms -0.13% | ±0.40% -11.97%  |
| NativeEntryFactoryBench | bench_entry_factory        | 1    | 3   | 116.706mb +0.00% | 490.572ms +3.76% | ±0.94% +96.61%  |
| NativeEntryFactoryBench | bench_entry_factory        | 1    | 3   | 60.184mb +0.00%  | 247.209ms +4.93% | ±1.92% +25.24%  |
| NativeEntryFactoryBench | bench_entry_factory        | 1    | 3   | 15.118mb +0.01%  | 52.336ms +2.71%  | ±0.87% +9.55%   |
| TypeDetectorBench       | bench_type_detector        | 1    | 3   | 59.957mb +0.00%  | 432.242ms +1.55% | ±0.36% -42.51%  |
| TypeDetectorBench       | bench_type_detector        | 1    | 3   | 14.496mb +0.01%  | 86.231ms +3.12%  | ±0.43% -42.10%  |
+-------------------------+----------------------------+------+-----+------------------+------------------+-----------------+