Make parquet rows normalizer to respect given schema

[norberttech]

Change Log

Added

Fixed

• Make parquet rows normalizer to respect given schema

Changed

Removed

Deprecated

Security

---

Description

Currently even if we decide to tell parquet by passing schema "save id as integer" it would still save the value as is.
This change takes "id" type from schema (which is either provided or inferred) and it's casting given value the type defined by that schema.

[github-actions]

Flow PHP - Benchmarks

^{_{Results of the benchmarks from this PR are compared with the results from 1.x branch.}}

Extractors

+-----------------------+-------------------+------+-----+------------------+------------------+-----------------+
| benchmark             | subject           | revs | its | mem_peak         | mode             | rstdev          |
+-----------------------+-------------------+------+-----+------------------+------------------+-----------------+
| AvroExtractorBench    | bench_extract_10k | 1    | 3   | 35.238mb +0.00%  | 813.999ms -2.06% | ±2.52% +386.98% |
| CSVExtractorBench     | bench_extract_10k | 1    | 3   | 4.964mb +0.01%   | 341.806ms -0.53% | ±0.48% -71.22%  |
| JsonExtractorBench    | bench_extract_10k | 1    | 3   | 5.107mb +0.01%   | 1.047s -0.55%    | ±0.13% -78.16%  |
| ParquetExtractorBench | bench_extract_10k | 1    | 3   | 239.806mb +0.00% | 1.250s -0.21%    | ±1.39% -4.03%   |
| TextExtractorBench    | bench_extract_10k | 1    | 3   | 4.885mb +0.00%   | 34.687ms +0.49%  | ±1.39% +134.80% |
| XmlExtractorBench     | bench_extract_10k | 1    | 3   | 4.887mb +0.00%   | 431.769ms +0.22% | ±0.65% +92.78%  |
+-----------------------+-------------------+------+-----+------------------+------------------+-----------------+

Transformers

+-----------------------------+--------------------------+------+-----+------------------+-----------------+-----------------+
| benchmark                   | subject                  | revs | its | mem_peak         | mode            | rstdev          |
+-----------------------------+--------------------------+------+-----+------------------+-----------------+-----------------+
| RenameEntryTransformerBench | bench_transform_10k_rows | 1    | 3   | 110.587mb +0.00% | 66.050ms +5.09% | ±1.30% +157.97% |
+-----------------------------+--------------------------+------+-----+------------------+-----------------+-----------------+

Loaders

+--------------------+----------------+------+-----+------------------+------------------+-----------------+
| benchmark          | subject        | revs | its | mem_peak         | mode             | rstdev          |
+--------------------+----------------+------+-----+------------------+------------------+-----------------+
| AvroLoaderBench    | bench_load_10k | 1    | 3   | 95.618mb +0.00%  | 482.043ms +0.41% | ±2.83% +244.51% |
| CSVLoaderBench     | bench_load_10k | 1    | 3   | 54.101mb +0.00%  | 72.463ms +1.55%  | ±0.26% -44.36%  |
| JsonLoaderBench    | bench_load_10k | 1    | 3   | 106.523mb +0.00% | 52.257ms -0.80%  | ±0.25% -87.04%  |
| ParquetLoaderBench | bench_load_10k | 1    | 3   | 321.728mb +0.00% | 1.511s +16.26%   | ±0.22% -76.03%  |
| TextLoaderBench    | bench_load_10k | 1    | 3   | 17.925mb +0.00%  | 41.509ms -0.29%  | ±1.68% -8.01%   |
+--------------------+----------------+------+-----+------------------+------------------+-----------------+

Building Blocks

+-------------------------+----------------------------+------+-----+------------------+------------------+------------------+
| benchmark               | subject                    | revs | its | mem_peak         | mode             | rstdev           |
+-------------------------+----------------------------+------+-----+------------------+------------------+------------------+
| RowsBench               | bench_chunk_10_on_10k      | 2    | 3   | 76.662mb +0.00%  | 3.524ms +3.35%   | ±0.26% -86.84%   |
| RowsBench               | bench_diff_left_1k_on_10k  | 2    | 3   | 96.389mb +0.00%  | 180.384ms -2.48% | ±0.71% +153.38%  |
| RowsBench               | bench_diff_right_1k_on_10k | 2    | 3   | 74.914mb +0.00%  | 17.985ms -2.90%  | ±3.06% +2037.62% |
| RowsBench               | bench_drop_1k_on_10k       | 2    | 3   | 77.902mb +0.00%  | 1.795ms +4.71%   | ±1.38% -34.32%   |
| RowsBench               | bench_drop_right_1k_on_10k | 2    | 3   | 77.902mb +0.00%  | 1.833ms +6.49%   | ±2.02% +67.95%   |
| RowsBench               | bench_entries_on_10k       | 2    | 3   | 75.014mb +0.00%  | 2.684ms +3.91%   | ±0.47% -66.24%   |
| RowsBench               | bench_filter_on_10k        | 2    | 3   | 75.543mb +0.00%  | 15.141ms +3.31%  | ±3.59% +11.62%   |
| RowsBench               | bench_find_on_10k          | 2    | 3   | 75.543mb +0.00%  | 15.000ms +1.00%  | ±0.06% -95.89%   |
| RowsBench               | bench_find_one_on_10k      | 10   | 3   | 73.447mb +0.00%  | 1.894μs +11.81%  | ±2.53% -10.71%   |
| RowsBench               | bench_first_on_10k         | 10   | 3   | 73.447mb +0.00%  | 0.400μs 0.00%    | ±0.00% 0.00%     |
| RowsBench               | bench_flat_map_on_1k       | 2    | 3   | 87.001mb +0.00%  | 12.848ms -0.39%  | ±0.34% -69.67%   |
| RowsBench               | bench_map_on_10k           | 2    | 3   | 116.362mb +0.00% | 66.249ms +2.74%  | ±0.74% +80.72%   |
| RowsBench               | bench_merge_1k_on_10k      | 2    | 3   | 76.062mb +0.00%  | 1.402ms +12.69%  | ±2.39% +127.84%  |
| RowsBench               | bench_partition_by_on_10k  | 2    | 3   | 79.343mb +0.00%  | 59.081ms +3.45%  | ±1.80% +210.79%  |
| RowsBench               | bench_remove_on_10k        | 2    | 3   | 78.164mb +0.00%  | 4.145ms +7.33%   | ±3.59% +58.85%   |
| RowsBench               | bench_sort_asc_on_1k       | 2    | 3   | 73.525mb +0.00%  | 39.899ms -2.72%  | ±0.53% -67.25%   |
| RowsBench               | bench_sort_by_on_1k        | 2    | 3   | 73.526mb +0.00%  | 39.927ms -4.70%  | ±1.38% +169.34%  |
| RowsBench               | bench_sort_desc_on_1k      | 2    | 3   | 73.525mb +0.00%  | 40.906ms -1.23%  | ±1.55% +219.65%  |
| RowsBench               | bench_sort_entries_on_1k   | 2    | 3   | 75.888mb +0.00%  | 7.258ms -4.42%   | ±0.56% -25.81%   |
| RowsBench               | bench_sort_on_1k           | 2    | 3   | 73.447mb +0.00%  | 28.860ms -0.22%  | ±1.02% -6.98%    |
| RowsBench               | bench_take_1k_on_10k       | 10   | 3   | 73.447mb +0.00%  | 13.306μs -1.19%  | ±0.35% -76.71%   |
| RowsBench               | bench_take_right_1k_on_10k | 10   | 3   | 73.447mb +0.00%  | 15.883μs -4.31%  | ±1.35% -8.50%    |
| RowsBench               | bench_unique_on_1k         | 2    | 3   | 96.390mb +0.00%  | 181.462ms -2.80% | ±0.82% +2.95%    |
| NativeEntryFactoryBench | bench_entry_factory        | 1    | 3   | 116.688mb +0.00% | 492.530ms -1.42% | ±1.93% +986.11%  |
| NativeEntryFactoryBench | bench_entry_factory        | 1    | 3   | 60.166mb +0.00%  | 251.021ms +0.78% | ±0.62% -52.70%   |
| NativeEntryFactoryBench | bench_entry_factory        | 1    | 3   | 15.099mb +0.00%  | 52.871ms +2.09%  | ±1.93% +18.29%   |
| TypeDetectorBench       | bench_type_detector        | 1    | 3   | 59.943mb +0.00%  | 434.679ms +0.95% | ±0.81% -10.84%   |
| TypeDetectorBench       | bench_type_detector        | 1    | 3   | 14.482mb +0.00%  | 87.194ms +2.18%  | ±0.95% +353.92%  |
+-------------------------+----------------------------+------+-----+------------------+------------------+------------------+