RetryLoader

[norberttech]

Resolves: #1842

Change Log

---

Added

• RetryLoader
• Retry mechanism building blocks

Fixed

Changed

Removed

Deprecated

Security

[codecov]

Codecov Report

❌ Patch coverage is 96.09375% with 5 lines in your changes missing coverage. Please review.
✅ Project coverage is 82.41%. Comparing base (be1db54) to head (627c17d).
⚠️ Report is 2 commits behind head on 1.x.
✅ All tests successful. No failed tests found.

Additional details and impacted files

@@            Coverage Diff             @@
##              1.x    #1845      +/-   ##
==========================================
+ Coverage   82.32%   82.41%   +0.08%     
==========================================
  Files         754      768      +14     
  Lines       21351    21479     +128     
==========================================
+ Hits        17577    17701     +124     
- Misses       3774     3778       +4     

Components	Coverage Δ
etl	89.25% <96.09%> (+0.12%)	⬆️
cli	85.91% <ø> (ø)
lib-array-dot	94.56% <ø> (ø)
lib-azure-sdk	61.35% <ø> (ø)
lib-doctrine-dbal-bulk	95.59% <ø> (ø)
lib-filesystem	78.02% <ø> (ø)
lib-types	53.55% <ø> (ø)
lib-parquet	85.50% <ø> (ø)
lib-parquet-viewer	83.11% <ø> (ø)
lib-snappy	90.23% <ø> (+0.46%)	⬆️
bridge-filesystem-async-aws	90.38% <ø> (ø)
bridge-filesystem-azure	89.92% <ø> (ø)
bridge-monolog-http	97.04% <ø> (ø)
bridge-openapi-specification	94.52% <ø> (ø)
symfony-http-foundation	74.41% <ø> (ø)
adapter-chartjs	86.70% <ø> (ø)
adapter-csv	88.85% <ø> (ø)
adapter-doctrine	91.21% <ø> (ø)
adapter-elasticsearch	97.23% <ø> (ø)
adapter-google-sheet	84.49% <ø> (ø)
adapter-http	58.10% <ø> (ø)
adapter-json	87.98% <ø> (ø)
adapter-logger	53.84% <ø> (ø)
adapter-meilisearch	97.95% <ø> (ø)
adapter-parquet	78.92% <ø> (ø)
adapter-text	84.44% <ø> (ø)
adapter-xml	82.73% <ø> (ø)

🚀 New features to boost your workflow:

• 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

[github-actions]

Flow PHP - Benchmarks

^{_{Results of the benchmarks from this PR are compared with the results from 1.x branch.}}

Extractors

+-----------------------+------------------------+------+-----+-----------------+------------------+------------------+
| benchmark             | subject                | revs | its | mem_peak        | mode             | rstdev           |
+-----------------------+------------------------+------+-----+-----------------+------------------+------------------+
| CSVExtractorBench     | bench_extract_10k      | 1    | 3   | 4.904mb +0.44%  | 439.947ms +1.52% | ±0.25% -35.65%   |
| ExcelExtractorBench   | bench_extract_10k_ods  | 1    | 3   | 65.599mb +0.03% | 1.079s +2.28%    | ±2.33% +1550.34% |
| ExcelExtractorBench   | bench_extract_10k_xlsx | 1    | 3   | 67.711mb +0.03% | 1.664s -1.16%    | ±0.82% +4.56%    |
| JsonExtractorBench    | bench_extract_10k      | 1    | 3   | 5.495mb -0.17%  | 1.133s +0.80%    | ±0.88% -19.60%   |
| ParquetExtractorBench | bench_extract_10k      | 1    | 3   | 10.722mb +0.03% | 9.211s -19.84%   | ±0.65% +75.34%   |
| TextExtractorBench    | bench_extract_10k      | 1    | 3   | 4.630mb +0.47%  | 42.145ms +0.20%  | ±0.77% -13.59%   |
| XmlExtractorBench     | bench_extract_10k      | 1    | 3   | 4.612mb +0.47%  | 597.065ms +0.92% | ±0.07% -90.44%   |
+-----------------------+------------------------+------+-----+-----------------+------------------+------------------+

Transformers

+---------------------------------+--------------------------+------+-----+------------------+-----------------+-----------------+
| benchmark                       | subject                  | revs | its | mem_peak         | mode            | rstdev          |
+---------------------------------+--------------------------+------+-----+------------------+-----------------+-----------------+
| RenameEachEntryTransformerBench | bench_transform_10k_rows | 1    | 3   | 18.623mb +0.12%  | 72.877ms +0.96% | ±0.54% +44.54%  |
| RenameEntryTransformerBench     | bench_transform_10k_rows | 1    | 3   | 123.427mb +0.02% | 65.697ms +0.49% | ±1.34% +535.90% |
+---------------------------------+--------------------------+------+-----+------------------+-----------------+-----------------+

Loaders

+--------------------+----------------+------+-----+------------------+------------------+----------------+
| benchmark          | subject        | revs | its | mem_peak         | mode             | rstdev         |
+--------------------+----------------+------+-----+------------------+------------------+----------------+
| CSVLoaderBench     | bench_load_10k | 1    | 3   | 62.565mb +0.03%  | 84.354ms +1.31%  | ±1.40% -15.33% |
| JsonLoaderBench    | bench_load_10k | 1    | 3   | 80.646mb +0.03%  | 102.707ms +1.84% | ±0.61% +8.85%  |
| ParquetLoaderBench | bench_load_10k | 1    | 3   | 819.325mb +0.05% | 20.056s -26.13%  | ±0.39% -46.53% |
| TextLoaderBench    | bench_load_10k | 1    | 3   | 17.929mb +0.12%  | 29.571ms -0.71%  | ±0.47% -39.37% |
+--------------------+----------------+------+-----+------------------+------------------+----------------+

Building Blocks

+-------------------+----------------------------+------+-----+------------------+------------------+------------------+
| benchmark         | subject                    | revs | its | mem_peak         | mode             | rstdev           |
+-------------------+----------------------------+------+-----+------------------+------------------+------------------+
| TypeDetectorBench | bench_type_detector        | 1    | 3   | 42.569mb +0.05%  | 413.517ms +1.46% | ±1.06% +146.35%  |
| TypeDetectorBench | bench_type_detector        | 1    | 3   | 11.626mb +0.19%  | 82.120ms +2.20%  | ±2.25% +321.26%  |
| EntryFactoryBench | bench_entry_factory        | 1    | 3   | 106.043mb +0.02% | 666.153ms +3.11% | ±2.51% +74.80%   |
| EntryFactoryBench | bench_entry_factory        | 1    | 3   | 55.321mb +0.04%  | 328.651ms +1.11% | ±1.12% -13.32%   |
| EntryFactoryBench | bench_entry_factory        | 1    | 3   | 14.907mb +0.15%  | 70.284ms -0.62%  | ±1.04% -46.47%   |
| RowsBench         | bench_chunk_10_on_10k      | 2    | 3   | 93.514mb +0.02%  | 3.790ms +17.21%  | ±2.07% +84.40%   |
| RowsBench         | bench_diff_left_1k_on_10k  | 2    | 3   | 110.904mb +0.02% | 235.684ms -0.15% | ±0.75% +164.60%  |
| RowsBench         | bench_diff_right_1k_on_10k | 2    | 3   | 93.624mb +0.02%  | 23.975ms -0.30%  | ±0.57% +4.37%    |
| RowsBench         | bench_drop_1k_on_10k       | 2    | 3   | 94.389mb +0.02%  | 1.556ms +16.42%  | ±3.01% +231.92%  |
| RowsBench         | bench_drop_right_1k_on_10k | 2    | 3   | 94.389mb +0.02%  | 1.505ms +18.35%  | ±1.77% +103.94%  |
| RowsBench         | bench_entries_on_10k       | 2    | 3   | 92.550mb +0.02%  | 3.581ms +2.18%   | ±0.98% -53.82%   |
| RowsBench         | bench_filter_on_10k        | 2    | 3   | 93.079mb +0.02%  | 17.058ms +12.90% | ±1.28% +467.14%  |
| RowsBench         | bench_find_on_10k          | 2    | 3   | 93.079mb +0.02%  | 17.868ms +16.96% | ±0.50% -67.58%   |
| RowsBench         | bench_find_one_on_10k      | 10   | 3   | 91.767mb +0.02%  | 1.894μs +11.41%  | ±2.53% +0.00%    |
| RowsBench         | bench_first_on_10k         | 10   | 3   | 91.767mb +0.02%  | 0.400μs 0.00%    | ±0.00% 0.00%     |
| RowsBench         | bench_flat_map_on_1k       | 2    | 3   | 100.828mb +0.02% | 14.442ms +1.75%  | ±1.24% +18.22%   |
| RowsBench         | bench_map_on_10k           | 2    | 3   | 130.255mb +0.02% | 67.254ms -1.42%  | ±0.55% -35.18%   |
| RowsBench         | bench_merge_1k_on_10k      | 2    | 3   | 93.598mb +0.02%  | 1.394ms +17.55%  | ±3.38% +162.54%  |
| RowsBench         | bench_partition_by_on_10k  | 2    | 3   | 96.986mb +0.02%  | 63.958ms +3.99%  | ±0.93% +147.42%  |
| RowsBench         | bench_remove_on_10k        | 2    | 3   | 94.651mb +0.02%  | 3.594ms +3.68%   | ±1.09% +57.05%   |
| RowsBench         | bench_sort_asc_on_1k       | 2    | 3   | 92.148mb +0.02%  | 40.791ms +2.57%  | ±1.09% +63.88%   |
| RowsBench         | bench_sort_by_on_1k        | 2    | 3   | 92.148mb +0.02%  | 40.001ms +1.19%  | ±1.08% -24.97%   |
| RowsBench         | bench_sort_desc_on_1k      | 2    | 3   | 92.148mb +0.02%  | 40.309ms +0.20%  | ±1.24% +1193.61% |
| RowsBench         | bench_sort_entries_on_1k   | 2    | 3   | 94.210mb +0.02%  | 8.052ms +2.52%   | ±0.90% +31.33%   |
| RowsBench         | bench_sort_on_1k           | 2    | 3   | 91.960mb +0.02%  | 29.006ms -0.20%  | ±0.68% -37.97%   |
| RowsBench         | bench_take_1k_on_10k       | 10   | 3   | 91.767mb +0.02%  | 14.210μs +2.22%  | ±3.11% +164.59%  |
| RowsBench         | bench_take_right_1k_on_10k | 10   | 3   | 91.767mb +0.02%  | 16.458μs +4.21%  | ±3.44% +1049.81% |
| RowsBench         | bench_unique_on_1k         | 2    | 3   | 110.904mb +0.02% | 243.168ms +2.26% | ±0.76% +236.94%  |
+-------------------+----------------------------+------+-----+------------------+------------------+------------------+

Parquet Library

+--------------------+---------------------------------+------+-----+------------------+-------------------+-----------------+
| benchmark          | subject                         | revs | its | mem_peak         | mode              | rstdev          |
+--------------------+---------------------------------+------+-----+------------------+-------------------+-----------------+
| ParquetReaderBench | bench_page_headers              | 1    | 3   | 6.696mb +1.30%   | 3.294s -3.54%     | ±0.70% -28.09%  |
| ParquetReaderBench | bench_read_metadata             | 1    | 3   | 5.381mb +0.40%   | 18.050ms -7.25%   | ±0.38% -52.31%  |
| ParquetReaderBench | bench_read_schema               | 1    | 3   | 5.381mb +0.40%   | 18.364ms -4.41%   | ±0.64% -39.25%  |
| ParquetReaderBench | bench_read_values_all_columns   | 1    | 3   | 9.131mb +0.03%   | 5.648s -27.80%    | ±0.29% -31.84%  |
| ParquetReaderBench | bench_read_values_single_column | 1    | 3   | 6.428mb +0.04%   | 233.263ms -49.64% | ±1.03% +396.40% |
| ParquetReaderBench | bench_read_values_with_limit    | 1    | 3   | 6.958mb -0.14%   | 29.183ms -16.09%  | ±0.20% -83.75%  |
| ParquetWriterBench | bench_write_batch               | 1    | 3   | 11.748mb -14.23% | 193.426ms -13.38% | ±0.39% -15.70%  |
| ParquetWriterBench | bench_write_gzip                | 1    | 3   | 10.373mb +0.22%  | 217.939ms -0.40%  | ±0.28% +87.46%  |
| ParquetWriterBench | bench_write_row_by_row          | 1    | 3   | 11.748mb -14.23% | 191.413ms -14.34% | ±0.25% -39.49%  |
| ParquetWriterBench | bench_write_snappy              | 1    | 3   | 11.748mb -14.23% | 191.079ms -13.58% | ±0.42% -28.59%  |
| ParquetWriterBench | bench_write_uncompressed        | 1    | 3   | 10.049mb +0.23%  | 191.337ms -0.25%  | ±0.53% +126.35% |
+--------------------+---------------------------------+------+-----+------------------+-------------------+-----------------+