Skip to content

Conversation

@norberttech
Copy link
Member

Change Log

Added

Fixed

  • writing nullable parquet values that are marked in the schema as optional

Changed

  • updated dependencies

Removed

Deprecated

Security


Description

@github-actions
Copy link
Contributor

github-actions bot commented Mar 28, 2024

Flow PHP - Benchmarks

Results of the benchmarks from this PR are compared with the results from 1.x branch.

Extractors
+-----------------------+-------------------+------+-----+------------------+------------------+----------------+
| benchmark             | subject           | revs | its | mem_peak         | mode             | rstdev         |
+-----------------------+-------------------+------+-----+------------------+------------------+----------------+
| AvroExtractorBench    | bench_extract_10k | 1    | 3   | 35.280mb +0.00%  | 807.948ms -2.12% | ±0.44% +6.29%  |
| CSVExtractorBench     | bench_extract_10k | 1    | 3   | 5.003mb +0.01%   | 345.004ms +0.81% | ±0.64% +58.26% |
| JsonExtractorBench    | bench_extract_10k | 1    | 3   | 5.153mb +0.00%   | 1.048s +0.32%    | ±0.13% -79.72% |
| ParquetExtractorBench | bench_extract_10k | 1    | 3   | 135.825mb +0.00% | 903.630ms -0.69% | ±0.26% -83.53% |
| TextExtractorBench    | bench_extract_10k | 1    | 3   | 4.910mb +0.01%   | 39.058ms +10.47% | ±2.26% +71.56% |
| XmlExtractorBench     | bench_extract_10k | 1    | 3   | 4.916mb +0.01%   | 431.263ms -1.01% | ±0.42% -37.99% |
+-----------------------+-------------------+------+-----+------------------+------------------+----------------+
Transformers
+-----------------------------+--------------------------+------+-----+------------------+-----------------+----------------+
| benchmark                   | subject                  | revs | its | mem_peak         | mode            | rstdev         |
+-----------------------------+--------------------------+------+-----+------------------+-----------------+----------------+
| RenameEntryTransformerBench | bench_transform_10k_rows | 1    | 3   | 110.616mb +0.00% | 64.192ms -0.28% | ±0.66% -15.72% |
+-----------------------------+--------------------------+------+-----+------------------+-----------------+----------------+
Loaders
+--------------------+----------------+------+-----+------------------+------------------+----------------+
| benchmark          | subject        | revs | its | mem_peak         | mode             | rstdev         |
+--------------------+----------------+------+-----+------------------+------------------+----------------+
| AvroLoaderBench    | bench_load_10k | 1    | 3   | 95.660mb +0.00%  | 470.798ms +1.15% | ±0.05% -93.39% |
| CSVLoaderBench     | bench_load_10k | 1    | 3   | 54.142mb +0.00%  | 71.440ms -3.47%  | ±0.43% -40.66% |
| JsonLoaderBench    | bench_load_10k | 1    | 3   | 106.567mb +0.00% | 51.669ms -0.74%  | ±0.69% -9.68%  |
| ParquetLoaderBench | bench_load_10k | 1    | 3   | 224.388mb +0.00% | 1.440s +1.69%    | ±0.74% -65.58% |
| TextLoaderBench    | bench_load_10k | 1    | 3   | 17.957mb +0.00%  | 40.183ms +0.97%  | ±0.30% -54.94% |
+--------------------+----------------+------+-----+------------------+------------------+----------------+
Building Blocks
+-------------------------+----------------------------+------+-----+------------------+------------------+-----------------+
| benchmark               | subject                    | revs | its | mem_peak         | mode             | rstdev          |
+-------------------------+----------------------------+------+-----+------------------+------------------+-----------------+
| RowsBench               | bench_chunk_10_on_10k      | 2    | 3   | 76.683mb +0.00%  | 3.400ms +0.71%   | ±2.55% +67.99%  |
| RowsBench               | bench_diff_left_1k_on_10k  | 2    | 3   | 96.410mb +0.00%  | 182.939ms +1.09% | ±1.17% +89.91%  |
| RowsBench               | bench_diff_right_1k_on_10k | 2    | 3   | 74.936mb +0.00%  | 18.458ms +2.47%  | ±0.34% -72.32%  |
| RowsBench               | bench_drop_1k_on_10k       | 2    | 3   | 77.923mb +0.00%  | 1.697ms -1.49%   | ±1.18% -59.21%  |
| RowsBench               | bench_drop_right_1k_on_10k | 2    | 3   | 77.923mb +0.00%  | 1.887ms +11.57%  | ±2.32% +96.40%  |
| RowsBench               | bench_entries_on_10k       | 2    | 3   | 75.035mb +0.00%  | 2.603ms +1.69%   | ±0.94% -41.35%  |
| RowsBench               | bench_filter_on_10k        | 2    | 3   | 75.564mb +0.00%  | 15.174ms +2.73%  | ±1.88% +79.00%  |
| RowsBench               | bench_find_on_10k          | 2    | 3   | 75.564mb +0.00%  | 15.207ms +5.03%  | ±1.06% -52.55%  |
| RowsBench               | bench_find_one_on_10k      | 10   | 3   | 73.468mb +0.00%  | 1.794μs +5.15%   | ±2.67% -1.89%   |
| RowsBench               | bench_first_on_10k         | 10   | 3   | 73.468mb +0.00%  | 0.400μs +33.33%  | ±0.00% +0.00%   |
| RowsBench               | bench_flat_map_on_1k       | 2    | 3   | 87.023mb +0.00%  | 13.273ms +3.02%  | ±2.41% -31.72%  |
| RowsBench               | bench_map_on_10k           | 2    | 3   | 116.383mb +0.00% | 66.412ms +3.96%  | ±0.49% -54.38%  |
| RowsBench               | bench_merge_1k_on_10k      | 2    | 3   | 76.084mb +0.00%  | 1.382ms +11.84%  | ±3.22% +2.54%   |
| RowsBench               | bench_partition_by_on_10k  | 2    | 3   | 79.431mb +0.00%  | 59.683ms +2.09%  | ±2.84% +755.43% |
| RowsBench               | bench_remove_on_10k        | 2    | 3   | 78.185mb +0.00%  | 3.860ms -9.19%   | ±2.45% +18.01%  |
| RowsBench               | bench_sort_asc_on_1k       | 2    | 3   | 73.546mb +0.00%  | 40.968ms +1.00%  | ±0.38% -74.69%  |
| RowsBench               | bench_sort_by_on_1k        | 2    | 3   | 73.547mb +0.00%  | 42.729ms +4.92%  | ±2.24% +108.97% |
| RowsBench               | bench_sort_desc_on_1k      | 2    | 3   | 73.546mb +0.00%  | 41.048ms +1.11%  | ±2.18% +229.16% |
| RowsBench               | bench_sort_entries_on_1k   | 2    | 3   | 75.909mb +0.00%  | 7.447ms +1.89%   | ±1.26% -35.98%  |
| RowsBench               | bench_sort_on_1k           | 2    | 3   | 73.468mb +0.00%  | 29.404ms -1.94%  | ±0.30% -85.52%  |
| RowsBench               | bench_take_1k_on_10k       | 10   | 3   | 73.468mb +0.00%  | 13.312μs -2.88%  | ±0.71% -69.12%  |
| RowsBench               | bench_take_right_1k_on_10k | 10   | 3   | 73.468mb +0.00%  | 15.800μs +0.04%  | ±0.52% -73.31%  |
| RowsBench               | bench_unique_on_1k         | 2    | 3   | 96.477mb +0.00%  | 186.186ms -0.42% | ±0.88% +11.87%  |
| NativeEntryFactoryBench | bench_entry_factory        | 1    | 3   | 116.715mb +0.00% | 499.216ms +0.66% | ±1.65% +24.03%  |
| NativeEntryFactoryBench | bench_entry_factory        | 1    | 3   | 60.193mb +0.00%  | 246.117ms -0.85% | ±0.67% -30.50%  |
| NativeEntryFactoryBench | bench_entry_factory        | 1    | 3   | 15.128mb +0.00%  | 52.898ms +1.45%  | ±1.84% -17.91%  |
| TypeDetectorBench       | bench_type_detector        | 1    | 3   | 59.959mb +0.00%  | 434.770ms +0.27% | ±0.35% -42.72%  |
| TypeDetectorBench       | bench_type_detector        | 1    | 3   | 14.498mb +0.00%  | 85.823ms +0.30%  | ±0.59% -63.64%  |
+-------------------------+----------------------------+------+-----+------------------+------------------+-----------------+

@norberttech norberttech merged commit 37a630b into flow-php:1.x Mar 28, 2024
@norberttech norberttech deleted the bug/parquet-nullable-rows branch May 9, 2024 08:55
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant