simplify primary key metadata

[norberttech]

Change Log

Added

Fixed

• Setting up the offset for reading remote parquet files

Changed

• Allow to not pass primary key name to dbal metadata since it's not used in some cases

Removed

Deprecated

Security

---

Description

[github-actions]

Flow PHP - Benchmarks

^{_{Results of the benchmarks from this PR are compared with the results from 1.x branch.}}

Extractors

+-----------------------+-------------------+------+-----+-----------------+------------------+----------------+
| benchmark             | subject           | revs | its | mem_peak        | mode             | rstdev         |
+-----------------------+-------------------+------+-----+-----------------+------------------+----------------+
| CSVExtractorBench     | bench_extract_10k | 1    | 3   | 4.786mb +0.01%  | 554.700ms -0.30% | ±0.68% -80.59% |
| JsonExtractorBench    | bench_extract_10k | 1    | 3   | 4.860mb +0.01%  | 1.065s +0.65%    | ±0.67% -57.14% |
| ParquetExtractorBench | bench_extract_10k | 1    | 3   | 86.300mb +0.00% | 897.131ms -0.03% | ±0.41% +80.98% |
| TextExtractorBench    | bench_extract_10k | 1    | 3   | 4.517mb +0.01%  | 35.634ms +1.40%  | ±0.64% -36.37% |
| XmlExtractorBench     | bench_extract_10k | 1    | 3   | 4.491mb +0.01%  | 605.163ms +0.92% | ±0.45% -46.90% |
+-----------------------+-------------------+------+-----+-----------------+------------------+----------------+

Transformers

+-----------------------------+--------------------------+------+-----+------------------+-----------------+----------------+
| benchmark                   | subject                  | revs | its | mem_peak         | mode            | rstdev         |
+-----------------------------+--------------------------+------+-----+------------------+-----------------+----------------+
| RenameEntryTransformerBench | bench_transform_10k_rows | 1    | 3   | 127.313mb +0.00% | 71.189ms -0.88% | ±0.61% -36.20% |
+-----------------------------+--------------------------+------+-----+------------------+-----------------+----------------+

Loaders

+--------------------+----------------+------+-----+------------------+------------------+-----------------+
| benchmark          | subject        | revs | its | mem_peak         | mode             | rstdev          |
+--------------------+----------------+------+-----+------------------+------------------+-----------------+
| CSVLoaderBench     | bench_load_10k | 1    | 3   | 63.984mb +0.00%  | 103.204ms -0.81% | ±0.97% +155.43% |
| JsonLoaderBench    | bench_load_10k | 1    | 3   | 84.331mb +0.00%  | 99.137ms +0.52%  | ±0.24% -54.53%  |
| ParquetLoaderBench | bench_load_10k | 1    | 3   | 161.172mb +0.00% | 20.686s +0.62%   | ±0.09% -76.51%  |
| TextLoaderBench    | bench_load_10k | 1    | 3   | 17.983mb +0.00%  | 31.310ms +0.67%  | ±0.66% +25.68%  |
+--------------------+----------------+------+-----+------------------+------------------+-----------------+

Building Blocks

+-------------------+----------------------------+------+-----+------------------+------------------+-----------------+
| benchmark         | subject                    | revs | its | mem_peak         | mode             | rstdev          |
+-------------------+----------------------------+------+-----+------------------+------------------+-----------------+
| EntryFactoryBench | bench_entry_factory        | 1    | 3   | 105.956mb +0.00% | 453.425ms +0.51% | ±0.50% -45.60%  |
| EntryFactoryBench | bench_entry_factory        | 1    | 3   | 55.146mb +0.00%  | 231.123ms +0.21% | ±0.13% -81.23%  |
| EntryFactoryBench | bench_entry_factory        | 1    | 3   | 14.668mb +0.00%  | 50.285ms +0.81%  | ±0.66% -54.15%  |
| RowsBench         | bench_chunk_10_on_10k      | 2    | 3   | 96.999mb +0.00%  | 3.257ms -2.23%   | ±1.47% -58.43%  |
| RowsBench         | bench_diff_left_1k_on_10k  | 2    | 3   | 114.281mb +0.00% | 184.180ms -1.15% | ±0.92% +301.43% |
| RowsBench         | bench_diff_right_1k_on_10k | 2    | 3   | 97.001mb +0.00%  | 18.301ms -3.31%  | ±0.34% -77.08%  |
| RowsBench         | bench_drop_1k_on_10k       | 2    | 3   | 97.874mb +0.00%  | 1.593ms -1.01%   | ±1.88% -12.08%  |
| RowsBench         | bench_drop_right_1k_on_10k | 2    | 3   | 97.874mb +0.00%  | 1.524ms -9.40%   | ±1.75% -50.39%  |
| RowsBench         | bench_entries_on_10k       | 2    | 3   | 96.035mb +0.00%  | 4.538ms -12.92%  | ±2.19% +30.20%  |
| RowsBench         | bench_filter_on_10k        | 2    | 3   | 96.564mb +0.00%  | 16.430ms -1.94%  | ±0.20% -54.48%  |
| RowsBench         | bench_find_on_10k          | 2    | 3   | 96.564mb +0.00%  | 16.335ms -1.30%  | ±0.98% +14.71%  |
| RowsBench         | bench_find_one_on_10k      | 10   | 3   | 95.255mb +0.00%  | 1.906μs -4.41%   | ±2.44% +1.72%   |
| RowsBench         | bench_first_on_10k         | 10   | 3   | 95.255mb +0.00%  | 0.400μs 0.00%    | ±0.00% 0.00%    |
| RowsBench         | bench_flat_map_on_1k       | 2    | 3   | 104.473mb +0.00% | 14.495ms -2.64%  | ±1.30% +96.28%  |
| RowsBench         | bench_map_on_10k           | 2    | 3   | 134.540mb +0.00% | 71.949ms -2.64%  | ±0.57% +16.58%  |
| RowsBench         | bench_merge_1k_on_10k      | 2    | 3   | 97.083mb +0.00%  | 1.394ms -15.61%  | ±2.50% +81.16%  |
| RowsBench         | bench_partition_by_on_10k  | 2    | 3   | 100.381mb +0.00% | 65.775ms +1.67%  | ±0.99% +40.34%  |
| RowsBench         | bench_remove_on_10k        | 2    | 3   | 98.136mb +0.00%  | 3.787ms -3.02%   | ±2.52% +130.71% |
| RowsBench         | bench_sort_asc_on_1k       | 2    | 3   | 95.543mb +0.00%  | 41.815ms -0.36%  | ±0.83% +179.28% |
| RowsBench         | bench_sort_by_on_1k        | 2    | 3   | 95.543mb +0.00%  | 43.258ms -0.53%  | ±1.48% +254.01% |
| RowsBench         | bench_sort_desc_on_1k      | 2    | 3   | 95.543mb +0.00%  | 42.449ms -1.88%  | ±0.70% -43.00%  |
| RowsBench         | bench_sort_entries_on_1k   | 2    | 3   | 97.695mb +0.00%  | 8.462ms +1.80%   | ±2.19% +511.06% |
| RowsBench         | bench_sort_on_1k           | 2    | 3   | 95.445mb +0.00%  | 29.444ms -0.31%  | ±0.74% -50.04%  |
| RowsBench         | bench_take_1k_on_10k       | 10   | 3   | 95.255mb +0.00%  | 13.388μs +0.52%  | ±1.94% +83.76%  |
| RowsBench         | bench_take_right_1k_on_10k | 10   | 3   | 95.255mb +0.00%  | 15.300μs -2.13%  | ±0.00% -100.00% |
| RowsBench         | bench_unique_on_1k         | 2    | 3   | 114.282mb +0.00% | 189.685ms +0.86% | ±0.36% -13.27%  |
| TypeDetectorBench | bench_type_detector        | 1    | 3   | 43.791mb +0.00%  | 360.353ms +0.04% | ±1.10% +388.73% |
| TypeDetectorBench | bench_type_detector        | 1    | 3   | 11.601mb +0.00%  | 73.013ms -0.79%  | ±0.42% +3.01%   |
+-------------------+----------------------------+------+-----+------------------+------------------+-----------------+

[codecov]

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 82.94%. Comparing base (c70f44c) to head (2d410e7).
Report is 2 commits behind head on 1.x.

Additional details and impacted files

@@            Coverage Diff             @@
##              1.x    #1437      +/-   ##
==========================================
+ Coverage   82.91%   82.94%   +0.02%     
==========================================
  Files         657      657              
  Lines       17631    17632       +1     
==========================================
+ Hits        14619    14624       +5     
+ Misses       3012     3008       -4     

Components	Coverage Δ
etl	85.75% <ø> (ø)
cli	85.17% <ø> (ø)
lib-array-dot	94.53% <ø> (ø)
lib-azure-sdk	62.56% <ø> (ø)
lib-doctrine-dbal-bulk	97.36% <ø> (ø)
lib-filesystem	76.75% <ø> (+0.51%)	⬆️
lib-parquet	84.33% <100.00%> (+<0.01%)	⬆️
lib-parquet-viewer	82.02% <ø> (ø)
lib-rdsl	87.09% <ø> (ø)
lib-snappy	91.16% <ø> (ø)
bridge-filesystem-async-aws	90.38% <ø> (ø)
bridge-filesystem-azure	89.92% <ø> (ø)
bridge-monolog-http	96.38% <ø> (ø)
symfony-http-foundation	77.10% <ø> (ø)
adapter-chartjs	86.45% <ø> (ø)
adapter-csv	89.49% <ø> (ø)
adapter-doctrine	88.54% <100.00%> (ø)
adapter-elasticsearch	97.19% <ø> (ø)
adapter-google-sheet	78.04% <ø> (ø)
adapter-http	59.15% <ø> (ø)
adapter-json	92.85% <ø> (ø)
adapter-logger	53.84% <ø> (ø)
adapter-meilisearch	97.75% <ø> (ø)
adapter-parquet	80.85% <ø> (ø)
adapter-text	84.44% <ø> (ø)
adapter-xml	83.15% <ø> (ø)