Added column statistics to pipeline execution report

[norberttech]

Change Log

Added

• Column statistics to pipeline execution report

Fixed

Changed

Removed

Deprecated

Security

---

Description

Resolves #1525

[github-actions]

Flow PHP - Benchmarks

^{_{Results of the benchmarks from this PR are compared with the results from 1.x branch.}}

Extractors

+-----------------------+-------------------+------+-----+-----------------+------------------+-----------------+
| benchmark             | subject           | revs | its | mem_peak        | mode             | rstdev          |
+-----------------------+-------------------+------+-----+-----------------+------------------+-----------------+
| CSVExtractorBench     | bench_extract_10k | 1    | 3   | 4.857mb +0.05%  | 557.864ms -0.13% | ±0.32% -68.21%  |
| JsonExtractorBench    | bench_extract_10k | 1    | 3   | 4.930mb +0.05%  | 1.082s +1.05%    | ±0.81% +91.83%  |
| ParquetExtractorBench | bench_extract_10k | 1    | 3   | 86.384mb +0.00% | 925.806ms +4.38% | ±3.02% +739.54% |
| TextExtractorBench    | bench_extract_10k | 1    | 3   | 4.587mb +0.05%  | 36.454ms +0.95%  | ±0.23% -71.13%  |
| XmlExtractorBench     | bench_extract_10k | 1    | 3   | 4.562mb +0.05%  | 605.994ms +0.22% | ±0.56% +183.91% |
+-----------------------+-------------------+------+-----+-----------------+------------------+-----------------+

Transformers

+-----------------------------+--------------------------+------+-----+------------------+-----------------+-----------------+
| benchmark                   | subject                  | revs | its | mem_peak         | mode            | rstdev          |
+-----------------------------+--------------------------+------+-----+------------------+-----------------+-----------------+
| RenameEntryTransformerBench | bench_transform_10k_rows | 1    | 3   | 127.384mb +0.00% | 74.536ms +4.09% | ±1.21% +126.14% |
+-----------------------------+--------------------------+------+-----+------------------+-----------------+-----------------+

Loaders

+--------------------+----------------+------+-----+------------------+------------------+-----------------+
| benchmark          | subject        | revs | its | mem_peak         | mode             | rstdev          |
+--------------------+----------------+------+-----+------------------+------------------+-----------------+
| CSVLoaderBench     | bench_load_10k | 1    | 3   | 64.056mb +0.00%  | 107.561ms +4.44% | ±0.66% -57.97%  |
| JsonLoaderBench    | bench_load_10k | 1    | 3   | 84.404mb +0.00%  | 100.168ms +4.57% | ±0.29% -91.66%  |
| ParquetLoaderBench | bench_load_10k | 1    | 3   | 161.255mb +0.00% | 21.315s +4.01%   | ±0.68% +210.32% |
| TextLoaderBench    | bench_load_10k | 1    | 3   | 18.119mb +0.01%  | 31.802ms +1.90%  | ±1.89% -32.37%  |
+--------------------+----------------+------+-----+------------------+------------------+-----------------+

Building Blocks

+-------------------+----------------------------+------+-----+------------------+------------------+-----------------+
| benchmark         | subject                    | revs | its | mem_peak         | mode             | rstdev          |
+-------------------+----------------------------+------+-----+------------------+------------------+-----------------+
| TypeDetectorBench | bench_type_detector        | 1    | 3   | 43.900mb +0.01%  | 373.384ms +1.80% | ±0.22% -59.73%  |
| TypeDetectorBench | bench_type_detector        | 1    | 3   | 11.710mb +0.02%  | 74.640ms +0.21%  | ±0.54% +204.75% |
| EntryFactoryBench | bench_entry_factory        | 1    | 3   | 106.003mb +0.00% | 469.760ms +2.26% | ±0.58% -43.67%  |
| EntryFactoryBench | bench_entry_factory        | 1    | 3   | 55.193mb +0.00%  | 235.807ms +1.55% | ±0.53% -26.79%  |
| EntryFactoryBench | bench_entry_factory        | 1    | 3   | 14.715mb +0.02%  | 52.297ms +3.11%  | ±1.21% +11.18%  |
| RowsBench         | bench_chunk_10_on_10k      | 2    | 3   | 97.046mb +0.00%  | 3.840ms +21.90%  | ±1.14% -45.31%  |
| RowsBench         | bench_diff_left_1k_on_10k  | 2    | 3   | 114.336mb +0.00% | 190.466ms +2.67% | ±0.49% -22.85%  |
| RowsBench         | bench_diff_right_1k_on_10k | 2    | 3   | 97.056mb +0.00%  | 19.195ms +3.16%  | ±0.23% -82.85%  |
| RowsBench         | bench_drop_1k_on_10k       | 2    | 3   | 97.921mb +0.00%  | 1.987ms +25.90%  | ±0.83% -79.12%  |
| RowsBench         | bench_drop_right_1k_on_10k | 2    | 3   | 97.921mb +0.00%  | 2.031ms +35.67%  | ±1.91% -15.14%  |
| RowsBench         | bench_entries_on_10k       | 2    | 3   | 96.082mb +0.00%  | 5.930ms +32.50%  | ±2.81% -1.36%   |
| RowsBench         | bench_filter_on_10k        | 2    | 3   | 96.611mb +0.00%  | 17.121ms +4.20%  | ±0.36% -42.33%  |
| RowsBench         | bench_find_on_10k          | 2    | 3   | 96.611mb +0.00%  | 17.116ms +3.53%  | ±0.79% +297.40% |
| RowsBench         | bench_find_one_on_10k      | 10   | 3   | 95.302mb +0.00%  | 2.000μs +10.74%  | ±0.00% -100.00% |
| RowsBench         | bench_first_on_10k         | 10   | 3   | 95.302mb +0.00%  | 0.400μs 0.00%    | ±0.00% 0.00%    |
| RowsBench         | bench_flat_map_on_1k       | 2    | 3   | 104.520mb +0.00% | 16.058ms +8.28%  | ±1.34% +5.12%   |
| RowsBench         | bench_map_on_10k           | 2    | 3   | 134.587mb +0.00% | 76.357ms +4.38%  | ±0.76% +44.93%  |
| RowsBench         | bench_merge_1k_on_10k      | 2    | 3   | 97.131mb +0.00%  | 2.047ms +50.15%  | ±2.66% -17.20%  |
| RowsBench         | bench_partition_by_on_10k  | 2    | 3   | 100.501mb +0.00% | 65.861ms +4.85%  | ±0.10% -90.08%  |
| RowsBench         | bench_remove_on_10k        | 2    | 3   | 98.183mb +0.00%  | 4.382ms +16.96%  | ±0.81% +32.08%  |
| RowsBench         | bench_sort_asc_on_1k       | 2    | 3   | 95.598mb +0.00%  | 44.835ms +5.73%  | ±2.03% +614.98% |
| RowsBench         | bench_sort_by_on_1k        | 2    | 3   | 95.598mb +0.00%  | 46.418ms +10.85% | ±3.75% +300.96% |
| RowsBench         | bench_sort_desc_on_1k      | 2    | 3   | 95.598mb +0.00%  | 45.634ms +7.16%  | ±1.62% +132.46% |
| RowsBench         | bench_sort_entries_on_1k   | 2    | 3   | 97.742mb +0.00%  | 8.462ms +4.02%   | ±1.91% +431.89% |
| RowsBench         | bench_sort_on_1k           | 2    | 3   | 95.492mb +0.00%  | 30.734ms +4.72%  | ±2.99% +459.45% |
| RowsBench         | bench_take_1k_on_10k       | 10   | 3   | 95.302mb +0.00%  | 14.806μs +10.99% | ±0.32% -85.30%  |
| RowsBench         | bench_take_right_1k_on_10k | 10   | 3   | 95.302mb +0.00%  | 17.583μs +14.92% | ±1.22% +128.70% |
| RowsBench         | bench_unique_on_1k         | 2    | 3   | 114.337mb +0.00% | 192.516ms +2.09% | ±0.36% -59.56%  |
+-------------------+----------------------------+------+-----+------------------+------------------+-----------------+

[codecov]

Codecov Report

Attention: Patch coverage is 96.87500% with 5 lines in your changes missing coverage. Please review.

Project coverage is 83.27%. Comparing base (eb80e18) to head (5e34068).
Report is 4 commits behind head on 1.x.

✅ All tests successful. No failed tests found.

Additional details and impacted files

@@            Coverage Diff             @@
##              1.x    #1527      +/-   ##
==========================================
+ Coverage   83.15%   83.27%   +0.12%     
==========================================
  Files         681      687       +6     
  Lines       18512    18635     +123     
==========================================
+ Hits        15393    15518     +125     
+ Misses       3119     3117       -2     

Components	Coverage Δ
etl	86.66% <95.45%> (+0.17%)	⬆️
cli	84.35% <98.61%> (+0.62%)	⬆️
lib-array-dot	94.53% <ø> (ø)
lib-azure-sdk	62.56% <ø> (ø)
lib-doctrine-dbal-bulk	90.11% <ø> (ø)
lib-filesystem	78.02% <ø> (ø)
lib-parquet	84.33% <ø> (ø)
lib-parquet-viewer	82.02% <ø> (ø)
lib-snappy	91.16% <ø> (+0.46%)	⬆️
bridge-filesystem-async-aws	90.38% <ø> (ø)
bridge-filesystem-azure	89.92% <ø> (ø)
bridge-monolog-http	96.38% <ø> (ø)
symfony-http-foundation	74.41% <ø> (ø)
adapter-chartjs	86.45% <ø> (ø)
adapter-csv	89.57% <ø> (ø)
adapter-doctrine	89.14% <ø> (ø)
adapter-elasticsearch	97.19% <ø> (ø)
adapter-google-sheet	78.04% <ø> (ø)
adapter-http	59.15% <ø> (ø)
adapter-json	90.62% <ø> (ø)
adapter-logger	53.84% <ø> (ø)
adapter-meilisearch	97.75% <ø> (ø)
adapter-parquet	80.85% <ø> (ø)
adapter-text	84.44% <ø> (ø)
adapter-xml	83.15% <ø> (ø)

🚀 New features to boost your workflow:

• 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.