Skip to content

Conversation

@norberttech
Copy link
Member

Change Log

Added

  • output on landing page examples
  • writing to output.txt directly from examples

Fixed

  • when writing directly to file stream, loader is not closing stream after each chunk of rows

Changed

Removed

Deprecated

Security


Description

image

@github-actions
Copy link
Contributor

Flow PHP - Benchmarks

Results of the benchmarks from this PR are compared with the results from 1.x branch.

Extractors
+-----------------------+-------------------+------+-----+------------------+------------------+-----------------+
| benchmark             | subject           | revs | its | mem_peak         | mode             | rstdev          |
+-----------------------+-------------------+------+-----+------------------+------------------+-----------------+
| AvroExtractorBench    | bench_extract_10k | 1    | 3   | 35.279mb +0.00%  | 824.330ms +0.38% | ±0.74% +69.76%  |
| CSVExtractorBench     | bench_extract_10k | 1    | 3   | 5.002mb +0.01%   | 343.149ms +0.18% | ±0.22% -70.15%  |
| JsonExtractorBench    | bench_extract_10k | 1    | 3   | 5.152mb +0.01%   | 1.048s -0.50%    | ±0.87% +331.90% |
| ParquetExtractorBench | bench_extract_10k | 1    | 3   | 135.823mb +0.00% | 896.813ms +0.04% | ±0.47% +10.85%  |
| TextExtractorBench    | bench_extract_10k | 1    | 3   | 4.910mb +0.01%   | 35.404ms -0.11%  | ±0.35% -63.83%  |
| XmlExtractorBench     | bench_extract_10k | 1    | 3   | 4.916mb +0.01%   | 427.368ms -1.37% | ±0.86% +32.80%  |
+-----------------------+-------------------+------+-----+------------------+------------------+-----------------+
Transformers
+-----------------------------+--------------------------+------+-----+------------------+-----------------+----------------+
| benchmark                   | subject                  | revs | its | mem_peak         | mode            | rstdev         |
+-----------------------------+--------------------------+------+-----+------------------+-----------------+----------------+
| RenameEntryTransformerBench | bench_transform_10k_rows | 1    | 3   | 110.616mb +0.00% | 64.385ms -0.96% | ±0.84% -30.60% |
+-----------------------------+--------------------------+------+-----+------------------+-----------------+----------------+
Loaders
+--------------------+----------------+------+-----+------------------+------------------+------------------+
| benchmark          | subject        | revs | its | mem_peak         | mode             | rstdev           |
+--------------------+----------------+------+-----+------------------+------------------+------------------+
| AvroLoaderBench    | bench_load_10k | 1    | 3   | 95.659mb +0.00%  | 466.607ms +1.10% | ±1.03% +1777.63% |
| CSVLoaderBench     | bench_load_10k | 1    | 3   | 54.141mb +0.00%  | 72.156ms +1.60%  | ±0.64% +140.51%  |
| JsonLoaderBench    | bench_load_10k | 1    | 3   | 106.567mb +0.00% | 52.736ms +1.86%  | ±0.94% +157.59%  |
| ParquetLoaderBench | bench_load_10k | 1    | 3   | 224.386mb +0.00% | 1.416s +0.05%    | ±0.74% +110.23%  |
| TextLoaderBench    | bench_load_10k | 1    | 3   | 17.957mb +0.00%  | 39.709ms -0.33%  | ±0.72% +241.26%  |
+--------------------+----------------+------+-----+------------------+------------------+------------------+
Building Blocks
+-------------------------+----------------------------+------+-----+------------------+------------------+-------------------------------+
| benchmark               | subject                    | revs | its | mem_peak         | mode             | rstdev                        |
+-------------------------+----------------------------+------+-----+------------------+------------------+-------------------------------+
| RowsBench               | bench_chunk_10_on_10k      | 2    | 3   | 76.682mb +0.00%  | 3.392ms +1.78%   | ±1.78% +415.78%               |
| RowsBench               | bench_diff_left_1k_on_10k  | 2    | 3   | 96.409mb +0.00%  | 179.432ms -0.90% | ±1.32% -19.36%                |
| RowsBench               | bench_diff_right_1k_on_10k | 2    | 3   | 74.935mb +0.00%  | 18.399ms +0.71%  | ±0.85% -22.40%                |
| RowsBench               | bench_drop_1k_on_10k       | 2    | 3   | 77.922mb +0.00%  | 1.719ms +1.30%   | ±0.77% -67.98%                |
| RowsBench               | bench_drop_right_1k_on_10k | 2    | 3   | 77.922mb +0.00%  | 1.730ms -1.46%   | ±1.09% -39.83%                |
| RowsBench               | bench_entries_on_10k       | 2    | 3   | 75.035mb +0.00%  | 2.583ms -10.30%  | ±1.19% -53.65%                |
| RowsBench               | bench_filter_on_10k        | 2    | 3   | 75.564mb +0.00%  | 16.646ms +10.36% | ±2.68% +76.95%                |
| RowsBench               | bench_find_on_10k          | 2    | 3   | 75.564mb +0.00%  | 16.452ms +11.87% | ±0.73% -28.36%                |
| RowsBench               | bench_find_one_on_10k      | 10   | 3   | 73.468mb +0.00%  | 1.794μs -5.58%   | ±2.67% +22832449392073000.00% |
| RowsBench               | bench_first_on_10k         | 10   | 3   | 73.468mb +0.00%  | 0.300μs -25.00%  | ±0.00% -100.00%               |
| RowsBench               | bench_flat_map_on_1k       | 2    | 3   | 87.022mb +0.00%  | 13.002ms -1.11%  | ±0.38% -65.26%                |
| RowsBench               | bench_map_on_10k           | 2    | 3   | 116.383mb +0.00% | 65.517ms -1.05%  | ±0.54% +12.97%                |
| RowsBench               | bench_merge_1k_on_10k      | 2    | 3   | 76.083mb +0.00%  | 1.276ms -1.97%   | ±1.34% -21.39%                |
| RowsBench               | bench_partition_by_on_10k  | 2    | 3   | 79.430mb +0.00%  | 59.295ms +1.10%  | ±1.55% +67.93%                |
| RowsBench               | bench_remove_on_10k        | 2    | 3   | 78.185mb +0.00%  | 3.863ms -5.90%   | ±0.82% -77.55%                |
| RowsBench               | bench_sort_asc_on_1k       | 2    | 3   | 73.546mb +0.00%  | 39.818ms -1.26%  | ±3.09% +129.92%               |
| RowsBench               | bench_sort_by_on_1k        | 2    | 3   | 73.546mb +0.00%  | 39.848ms -0.42%  | ±0.58% -42.45%                |
| RowsBench               | bench_sort_desc_on_1k      | 2    | 3   | 73.546mb +0.00%  | 40.591ms -0.42%  | ±0.28% -56.18%                |
| RowsBench               | bench_sort_entries_on_1k   | 2    | 3   | 75.909mb +0.00%  | 7.295ms -1.96%   | ±0.75% +176.98%               |
| RowsBench               | bench_sort_on_1k           | 2    | 3   | 73.468mb +0.00%  | 28.739ms -1.05%  | ±0.47% -28.04%                |
| RowsBench               | bench_take_1k_on_10k       | 10   | 3   | 73.468mb +0.00%  | 13.206μs -4.22%  | ±0.36% -81.09%                |
| RowsBench               | bench_take_right_1k_on_10k | 10   | 3   | 73.468mb +0.00%  | 15.906μs -1.69%  | ±0.30% -61.49%                |
| RowsBench               | bench_unique_on_1k         | 2    | 3   | 96.476mb +0.00%  | 188.472ms +1.86% | ±0.75% +116.76%               |
| NativeEntryFactoryBench | bench_entry_factory        | 1    | 3   | 116.714mb +0.00% | 501.823ms -0.68% | ±0.55% -59.34%                |
| NativeEntryFactoryBench | bench_entry_factory        | 1    | 3   | 60.192mb +0.00%  | 252.834ms +1.67% | ±1.32% +105.64%               |
| NativeEntryFactoryBench | bench_entry_factory        | 1    | 3   | 15.127mb +0.00%  | 53.012ms -0.38%  | ±0.86% +393.90%               |
| TypeDetectorBench       | bench_type_detector        | 1    | 3   | 59.958mb +0.00%  | 431.720ms +0.11% | ±0.55% -41.14%                |
| TypeDetectorBench       | bench_type_detector        | 1    | 3   | 14.497mb +0.00%  | 85.805ms -1.77%  | ±0.53% -39.56%                |
+-------------------------+----------------------------+------+-----+------------------+------------------+-------------------------------+

@norberttech norberttech merged commit 3c45ad3 into flow-php:1.x Mar 23, 2024
@norberttech norberttech deleted the feature/landing-code-samples branch May 9, 2024 08:55
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant