Automate normalization of rows in JsonLoader

[norberttech]

Change Log

Added

Fixed

Changed

• Automate normalization of rows in JsonLoader

Removed

Deprecated

Security

---

Description

Resolves: #1179

[github-actions]

Flow PHP - Benchmarks

^{_{Results of the benchmarks from this PR are compared with the results from 1.x branch.}}

Extractors

+-----------------------+-------------------+------+-----+-----------------+------------------+-----------------+
| benchmark             | subject           | revs | its | mem_peak        | mode             | rstdev          |
+-----------------------+-------------------+------+-----+-----------------+------------------+-----------------+
| CSVExtractorBench     | bench_extract_10k | 1    | 3   | 3.949mb +0.03%  | 513.717ms +0.82% | ±0.58% +235.04% |
| JsonExtractorBench    | bench_extract_10k | 1    | 3   | 4.082mb +0.03%  | 1.066s -0.68%    | ±0.86% +46.44%  |
| ParquetExtractorBench | bench_extract_10k | 1    | 3   | 28.544mb +0.00% | 429.516ms -0.88% | ±0.41% -47.64%  |
| TextExtractorBench    | bench_extract_10k | 1    | 3   | 3.709mb +0.03%  | 33.669ms -1.12%  | ±1.67% -22.66%  |
| XmlExtractorBench     | bench_extract_10k | 1    | 3   | 3.698mb +0.03%  | 759.387ms -3.42% | ±1.41% +88.15%  |
+-----------------------+-------------------+------+-----+-----------------+------------------+-----------------+

Transformers

+-----------------------------+--------------------------+------+-----+------------------+-----------------+----------------+
| benchmark                   | subject                  | revs | its | mem_peak         | mode            | rstdev         |
+-----------------------------+--------------------------+------+-----+------------------+-----------------+----------------+
| RenameEntryTransformerBench | bench_transform_10k_rows | 1    | 3   | 116.050mb +0.00% | 60.106ms -5.12% | ±0.48% -49.98% |
+-----------------------------+--------------------------+------+-----+------------------+-----------------+----------------+

Loaders

+--------------------+----------------+------+-----+-------------------+--------------------+-----------------+
| benchmark          | subject        | revs | its | mem_peak          | mode               | rstdev          |
+--------------------+----------------+------+-----+-------------------+--------------------+-----------------+
| CSVLoaderBench     | bench_load_10k | 1    | 3   | 54.171mb +0.00%   | 85.566ms -1.57%    | ±1.40% +117.24% |
| JsonLoaderBench    | bench_load_10k | 1    | 3   | 102.493mb -12.36% | 116.654ms +120.89% | ±0.91% -31.30%  |
| ParquetLoaderBench | bench_load_10k | 1    | 3   | 123.822mb +0.00%  | 1.224s -0.06%      | ±0.08% -80.49%  |
| TextLoaderBench    | bench_load_10k | 1    | 3   | 16.966mb +0.01%   | 43.940ms +0.27%    | ±0.62% +131.43% |
+--------------------+----------------+------+-----+-------------------+--------------------+-----------------+

Building Blocks

+-------------------------+----------------------------+------+-----+------------------+------------------+------------------+
| benchmark               | subject                    | revs | its | mem_peak         | mode             | rstdev           |
+-------------------------+----------------------------+------+-----+------------------+------------------+------------------+
| TypeDetectorBench       | bench_type_detector        | 1    | 3   | 52.631mb +0.00%  | 414.820ms +6.43% | ±3.00% +100.87%  |
| TypeDetectorBench       | bench_type_detector        | 1    | 3   | 12.897mb +0.01%  | 78.074ms -1.49%  | ±0.42% -84.63%   |
| NativeEntryFactoryBench | bench_entry_factory        | 1    | 3   | 106.824mb +0.00% | 471.594ms +0.61% | ±0.90% -45.03%   |
| NativeEntryFactoryBench | bench_entry_factory        | 1    | 3   | 55.182mb +0.00%  | 243.053ms +2.44% | ±1.37% +38.77%   |
| NativeEntryFactoryBench | bench_entry_factory        | 1    | 3   | 14.020mb +0.01%  | 50.656ms +0.22%  | ±2.31% +36.52%   |
| RowsBench               | bench_chunk_10_on_10k      | 2    | 3   | 86.807mb +0.00%  | 3.238ms -3.47%   | ±2.68% +147.88%  |
| RowsBench               | bench_diff_left_1k_on_10k  | 2    | 3   | 102.410mb +0.00% | 189.680ms +0.83% | ±1.17% +131.54%  |
| RowsBench               | bench_diff_right_1k_on_10k | 2    | 3   | 85.130mb +0.00%  | 18.439ms -1.75%  | ±0.78% -62.28%   |
| RowsBench               | bench_drop_1k_on_10k       | 2    | 3   | 88.047mb +0.00%  | 1.700ms -13.88%  | ±0.47% -74.98%   |
| RowsBench               | bench_drop_right_1k_on_10k | 2    | 3   | 88.047mb +0.00%  | 1.703ms -8.04%   | ±1.09% -37.51%   |
| RowsBench               | bench_entries_on_10k       | 2    | 3   | 85.159mb +0.00%  | 2.603ms -6.24%   | ±2.20% -17.31%   |
| RowsBench               | bench_filter_on_10k        | 2    | 3   | 85.688mb +0.00%  | 15.926ms -13.25% | ±1.73% -13.04%   |
| RowsBench               | bench_find_on_10k          | 2    | 3   | 85.688mb +0.00%  | 16.271ms -10.31% | ±1.39% -59.45%   |
| RowsBench               | bench_find_one_on_10k      | 10   | 3   | 83.591mb +0.00%  | 1.794μs 0.00%    | ±2.67% 0.00%     |
| RowsBench               | bench_first_on_10k         | 10   | 3   | 83.591mb +0.00%  | 0.300μs +50.00%  | ±0.00% -100.00%  |
| RowsBench               | bench_flat_map_on_1k       | 2    | 3   | 92.942mb +0.00%  | 12.054ms -5.74%  | ±0.62% -56.98%   |
| RowsBench               | bench_map_on_10k           | 2    | 3   | 122.313mb +0.00% | 60.121ms -5.96%  | ±0.47% -52.76%   |
| RowsBench               | bench_merge_1k_on_10k      | 2    | 3   | 86.207mb +0.00%  | 1.289ms -7.33%   | ±1.78% +102.67%  |
| RowsBench               | bench_partition_by_on_10k  | 2    | 3   | 89.559mb +0.00%  | 63.458ms +0.27%  | ±0.85% +22.64%   |
| RowsBench               | bench_remove_on_10k        | 2    | 3   | 88.309mb +0.00%  | 3.880ms -2.63%   | ±0.88% +141.05%  |
| RowsBench               | bench_sort_asc_on_1k       | 2    | 3   | 83.741mb +0.00%  | 39.342ms +0.66%  | ±0.41% -71.66%   |
| RowsBench               | bench_sort_by_on_1k        | 2    | 3   | 83.742mb +0.00%  | 39.581ms +1.05%  | ±0.19% -47.28%   |
| RowsBench               | bench_sort_desc_on_1k      | 2    | 3   | 83.741mb +0.00%  | 38.866ms +0.14%  | ±1.66% +297.11%  |
| RowsBench               | bench_sort_entries_on_1k   | 2    | 3   | 86.033mb +0.00%  | 7.402ms +1.63%   | ±0.66% +1367.06% |
| RowsBench               | bench_sort_on_1k           | 2    | 3   | 83.591mb +0.00%  | 28.541ms -9.99%  | ±0.53% +107.45%  |
| RowsBench               | bench_take_1k_on_10k       | 10   | 3   | 83.591mb +0.00%  | 13.394μs +0.38%  | ±0.35% -85.61%   |
| RowsBench               | bench_take_right_1k_on_10k | 10   | 3   | 83.591mb +0.00%  | 15.883μs -0.47%  | ±1.35% -26.16%   |
| RowsBench               | bench_unique_on_1k         | 2    | 3   | 102.411mb +0.00% | 191.429ms -1.46% | ±0.64% +98.76%   |
+-------------------------+----------------------------+------+-----+------------------+------------------+------------------+