Added a way to generate a Flow schema from Database in php

[norberttech]

Change Log

Added

• Databas to Flow schema converter
• Database table list in CLI
• Database table schema in CLI
• Schema conversion in CLI

Fixed

• Too generic names of DbalMetadata keys

Changed

Removed

Deprecated

Security

---

Description

[github-actions]

Flow PHP - Benchmarks

^{_{Results of the benchmarks from this PR are compared with the results from 1.x branch.}}

Extractors

+-----------------------+-------------------+------+-----+-----------------+------------------+-----------------+
| benchmark             | subject           | revs | its | mem_peak        | mode             | rstdev          |
+-----------------------+-------------------+------+-----+-----------------+------------------+-----------------+
| CSVExtractorBench     | bench_extract_10k | 1    | 3   | 4.850mb +0.28%  | 556.370ms -0.62% | ±0.90% -9.61%   |
| JsonExtractorBench    | bench_extract_10k | 1    | 3   | 4.923mb +0.28%  | 1.077s +1.04%    | ±0.50% +18.97%  |
| ParquetExtractorBench | bench_extract_10k | 1    | 3   | 86.377mb +0.02% | 908.130ms +1.93% | ±1.83% +237.66% |
| TextExtractorBench    | bench_extract_10k | 1    | 3   | 4.580mb +0.30%  | 36.215ms +2.70%  | ±0.11% -34.07%  |
| XmlExtractorBench     | bench_extract_10k | 1    | 3   | 4.554mb +0.30%  | 604.585ms -0.27% | ±0.12% +38.95%  |
+-----------------------+-------------------+------+-----+-----------------+------------------+-----------------+

Transformers

+-----------------------------+--------------------------+------+-----+------------------+-----------------+----------------+
| benchmark                   | subject                  | revs | its | mem_peak         | mode            | rstdev         |
+-----------------------------+--------------------------+------+-----+------------------+-----------------+----------------+
| RenameEntryTransformerBench | bench_transform_10k_rows | 1    | 3   | 127.376mb +0.01% | 74.673ms +5.74% | ±0.59% +91.13% |
+-----------------------------+--------------------------+------+-----+------------------+-----------------+----------------+

Loaders

+--------------------+----------------+------+-----+------------------+------------------+-----------------+
| benchmark          | subject        | revs | its | mem_peak         | mode             | rstdev          |
+--------------------+----------------+------+-----+------------------+------------------+-----------------+
| CSVLoaderBench     | bench_load_10k | 1    | 3   | 64.049mb +0.02%  | 104.772ms +3.65% | ±1.33% +786.87% |
| JsonLoaderBench    | bench_load_10k | 1    | 3   | 84.396mb +0.02%  | 98.719ms +3.44%  | ±0.33% -65.86%  |
| ParquetLoaderBench | bench_load_10k | 1    | 3   | 161.247mb +0.01% | 21.222s +3.25%   | ±2.53% +906.88% |
| TextLoaderBench    | bench_load_10k | 1    | 3   | 18.112mb +0.07%  | 32.436ms +3.41%  | ±0.56% -2.94%   |
+--------------------+----------------+------+-----+------------------+------------------+-----------------+

Building Blocks

+-------------------+----------------------------+------+-----+------------------+------------------+------------------+
| benchmark         | subject                    | revs | its | mem_peak         | mode             | rstdev           |
+-------------------+----------------------------+------+-----+------------------+------------------+------------------+
| TypeDetectorBench | bench_type_detector        | 1    | 3   | 43.896mb +0.02%  | 369.984ms +3.47% | ±0.66% +1.75%    |
| TypeDetectorBench | bench_type_detector        | 1    | 3   | 11.705mb +0.09%  | 75.097ms +3.71%  | ±0.37% -50.21%   |
| EntryFactoryBench | bench_entry_factory        | 1    | 3   | 105.995mb +0.01% | 461.879ms +2.65% | ±0.76% -36.50%   |
| EntryFactoryBench | bench_entry_factory        | 1    | 3   | 55.186mb +0.02%  | 237.886ms +3.42% | ±0.34% -54.69%   |
| EntryFactoryBench | bench_entry_factory        | 1    | 3   | 14.708mb +0.09%  | 51.467ms +1.86%  | ±0.75% -31.62%   |
| RowsBench         | bench_chunk_10_on_10k      | 2    | 3   | 97.039mb +0.01%  | 3.792ms +8.88%   | ±0.88% +293.95%  |
| RowsBench         | bench_diff_left_1k_on_10k  | 2    | 3   | 114.329mb +0.01% | 184.022ms -1.15% | ±0.63% -51.79%   |
| RowsBench         | bench_diff_right_1k_on_10k | 2    | 3   | 97.049mb +0.01%  | 18.752ms +0.95%  | ±0.70% -26.05%   |
| RowsBench         | bench_drop_1k_on_10k       | 2    | 3   | 97.914mb +0.01%  | 2.020ms +22.72%  | ±2.60% +58.09%   |
| RowsBench         | bench_drop_right_1k_on_10k | 2    | 3   | 97.914mb +0.01%  | 1.962ms +6.75%   | ±1.58% -55.88%   |
| RowsBench         | bench_entries_on_10k       | 2    | 3   | 96.074mb +0.01%  | 5.334ms +12.74%  | ±0.63% -24.00%   |
| RowsBench         | bench_filter_on_10k        | 2    | 3   | 96.603mb +0.01%  | 17.055ms +3.92%  | ±0.52% -50.82%   |
| RowsBench         | bench_find_on_10k          | 2    | 3   | 96.603mb +0.01%  | 17.047ms +3.73%  | ±0.57% -16.29%   |
| RowsBench         | bench_find_one_on_10k      | 10   | 3   | 95.295mb +0.01%  | 2.006μs +18.00%  | ±2.32% +0.00%    |
| RowsBench         | bench_first_on_10k         | 10   | 3   | 95.295mb +0.01%  | 0.400μs 0.00%    | ±0.00% 0.00%     |
| RowsBench         | bench_flat_map_on_1k       | 2    | 3   | 104.513mb +0.01% | 17.452ms +22.19% | ±2.23% +269.79%  |
| RowsBench         | bench_map_on_10k           | 2    | 3   | 134.580mb +0.01% | 75.458ms +7.87%  | ±1.02% +149.38%  |
| RowsBench         | bench_merge_1k_on_10k      | 2    | 3   | 97.123mb +0.01%  | 2.086ms +40.14%  | ±2.05% -31.90%   |
| RowsBench         | bench_partition_by_on_10k  | 2    | 3   | 100.428mb +0.08% | 66.642ms +1.84%  | ±1.54% +253.82%  |
| RowsBench         | bench_remove_on_10k        | 2    | 3   | 98.176mb +0.01%  | 4.427ms +17.63%  | ±1.83% +250.60%  |
| RowsBench         | bench_sort_asc_on_1k       | 2    | 3   | 95.590mb +0.01%  | 45.965ms +8.20%  | ±0.39% +158.92%  |
| RowsBench         | bench_sort_by_on_1k        | 2    | 3   | 95.591mb +0.01%  | 45.290ms +1.61%  | ±0.51% -85.72%   |
| RowsBench         | bench_sort_desc_on_1k      | 2    | 3   | 95.590mb +0.01%  | 44.937ms +2.00%  | ±1.31% +1287.10% |
| RowsBench         | bench_sort_entries_on_1k   | 2    | 3   | 97.735mb +0.01%  | 8.790ms +8.25%   | ±0.99% +15.73%   |
| RowsBench         | bench_sort_on_1k           | 2    | 3   | 95.485mb +0.01%  | 31.021ms +4.94%  | ±0.40% +11.49%   |
| RowsBench         | bench_take_1k_on_10k       | 10   | 3   | 95.295mb +0.01%  | 14.712μs +11.35% | ±0.64% -10.16%   |
| RowsBench         | bench_take_right_1k_on_10k | 10   | 3   | 95.295mb +0.01%  | 16.894μs +9.55%  | ±0.28% -65.56%   |
| RowsBench         | bench_unique_on_1k         | 2    | 3   | 114.330mb +0.01% | 185.252ms -1.92% | ±0.18% -81.13%   |
+-------------------+----------------------------+------+-----+------------------+------------------+------------------+

[codecov]

Codecov Report

Attention: Patch coverage is 86.76471% with 63 lines in your changes missing coverage. Please review.

Project coverage is 83.18%. Comparing base (8bfb453) to head (8c86c4a).
Report is 2 commits behind head on 1.x.

Additional details and impacted files

@@            Coverage Diff             @@
##              1.x    #1511      +/-   ##
==========================================
+ Coverage   83.05%   83.18%   +0.12%     
==========================================
  Files         685      696      +11     
  Lines       18315    18755     +440     
==========================================
+ Hits        15212    15601     +389     
- Misses       3103     3154      +51     

Components	Coverage Δ
etl	86.52% <98.38%> (+0.43%)	⬆️
cli	82.88% <70.41%> (-3.86%)	⬇️
lib-array-dot	94.53% <ø> (ø)
lib-azure-sdk	62.56% <ø> (ø)
lib-doctrine-dbal-bulk	90.11% <ø> (ø)
lib-filesystem	78.02% <ø> (ø)
lib-parquet	84.33% <ø> (ø)
lib-parquet-viewer	82.02% <ø> (ø)
lib-rdsl	87.09% <ø> (ø)
lib-snappy	91.16% <ø> (ø)
bridge-filesystem-async-aws	90.38% <ø> (ø)
bridge-filesystem-azure	89.92% <ø> (ø)
bridge-monolog-http	96.38% <ø> (ø)
symfony-http-foundation	74.41% <ø> (ø)
adapter-chartjs	86.45% <ø> (ø)
adapter-csv	89.57% <ø> (ø)
adapter-doctrine	88.94% <85.00%> (+0.86%)	⬆️
adapter-elasticsearch	97.19% <ø> (ø)
adapter-google-sheet	78.04% <ø> (ø)
adapter-http	59.15% <ø> (ø)
adapter-json	90.62% <ø> (ø)
adapter-logger	53.84% <ø> (ø)
adapter-meilisearch	97.75% <ø> (ø)
adapter-parquet	80.85% <ø> (ø)
adapter-text	84.44% <ø> (ø)
adapter-xml	83.15% <ø> (ø)