Entry factory schema

[norberttech]

Change Log

Added

• Caster mechanism that works on top of Flow\PHP\Type
• AutoCaster that works on top of Caster
• print_schema function
• print_rows function
• possibility to pass predefined schema to CSVExtractor
• possibility to pass predefined schema to JsonExtractor

Fixed

• NativeEntryFactory fully respecting schema definitions

Changed

Removed

Deprecated

Security

---

Description

This is probably one of the most important changes in 0.6.0
Thanks to the new Caster mechanism, EntryFactory will cast values into their native/logical types defined by schema.

Why this is so important?
Because once we identify the schema of our dataset, processing the same file can be way more efficient:

With provided schema to CSV Extractor
Without provided schema and autoCast

[github-actions]

Flow PHP - Benchmarks

^{_{Results of the benchmarks from this PR are compared with the results from 1.x branch.}}

Extractors

+-----------------------+-------------------+------+-----+------------------+------------------+-----------------+
| benchmark             | subject           | revs | its | mem_peak         | mode             | rstdev          |
+-----------------------+-------------------+------+-----+------------------+------------------+-----------------+
| AvroExtractorBench    | bench_extract_10k | 1    | 3   | 35.136mb +0.29%  | 823.023ms +1.09% | ±0.53% -72.49%  |
| CSVExtractorBench     | bench_extract_10k | 1    | 3   | 4.853mb +2.30%   | 344.783ms +1.58% | ±0.20% +17.08%  |
| JsonExtractorBench    | bench_extract_10k | 1    | 3   | 4.933mb +3.53%   | 1.040s +0.13%    | ±1.15% +104.80% |
| ParquetExtractorBench | bench_extract_10k | 1    | 3   | 239.639mb +0.07% | 1.261s +2.85%    | ±2.51% +40.51%  |
| TextExtractorBench    | bench_extract_10k | 1    | 3   | 4.675mb +4.49%   | 35.380ms +0.38%  | ±0.45% -48.09%  |
| XmlExtractorBench     | bench_extract_10k | 1    | 3   | 4.677mb +4.49%   | 432.539ms -0.47% | ±0.52% +125.00% |
+-----------------------+-------------------+------+-----+------------------+------------------+-----------------+

Transformers

+-----------------------------+--------------------------+------+-----+------------------+-----------------+-----------------+
| benchmark                   | subject                  | revs | its | mem_peak         | mode            | rstdev          |
+-----------------------------+--------------------------+------+-----+------------------+-----------------+-----------------+
| RenameEntryTransformerBench | bench_transform_10k_rows | 1    | 3   | 110.394mb +0.17% | 66.408ms +5.00% | ±1.30% +198.10% |
+-----------------------------+--------------------------+------+-----+------------------+-----------------+-----------------+

Loaders

+--------------------+----------------+------+-----+------------------+------------------+-----------------+
| benchmark          | subject        | revs | its | mem_peak         | mode             | rstdev          |
+--------------------+----------------+------+-----+------------------+------------------+-----------------+
| AvroLoaderBench    | bench_load_10k | 1    | 3   | 95.516mb +0.11%  | 478.347ms +2.56% | ±0.15% -20.49%  |
| CSVLoaderBench     | bench_load_10k | 1    | 3   | 53.926mb +0.32%  | 73.995ms +3.23%  | ±1.02% +100.09% |
| JsonLoaderBench    | bench_load_10k | 1    | 3   | 106.414mb +0.10% | 53.344ms +3.25%  | ±1.12% -25.71%  |
| ParquetLoaderBench | bench_load_10k | 1    | 3   | 321.626mb +0.03% | 1.316s +3.72%    | ±1.22% +311.95% |
| TextLoaderBench    | bench_load_10k | 1    | 3   | 17.742mb +1.03%  | 41.118ms +2.38%  | ±0.48% -5.39%   |
+--------------------+----------------+------+-----+------------------+------------------+-----------------+

Building Blocks

+-------------------------+----------------------------+------+-----+------------------+------------------+------------------+
| benchmark               | subject                    | revs | its | mem_peak         | mode             | rstdev           |
+-------------------------+----------------------------+------+-----+------------------+------------------+------------------+
| NativeEntryFactoryBench | bench_entry_factory        | 1    | 3   | 116.521mb +0.14% | 503.274ms +1.58% | ±0.13% -60.76%   |
| NativeEntryFactoryBench | bench_entry_factory        | 1    | 3   | 59.999mb +0.28%  | 249.560ms +2.24% | ±1.41% +98.03%   |
| NativeEntryFactoryBench | bench_entry_factory        | 1    | 3   | 14.932mb +1.12%  | 54.868ms +8.80%  | ±1.63% +16.44%   |
| TypeDetectorBench       | bench_type_detector        | 1    | 3   | 59.928mb +0.03%  | 436.873ms +1.80% | ±0.68% -25.14%   |
| TypeDetectorBench       | bench_type_detector        | 1    | 3   | 14.467mb +0.10%  | 89.153ms +4.70%  | ±0.52% -19.62%   |
| RowsBench               | bench_chunk_10_on_10k      | 2    | 3   | 76.474mb +0.25%  | 3.989ms +22.56%  | ±3.00% +122.60%  |
| RowsBench               | bench_diff_left_1k_on_10k  | 2    | 3   | 96.267mb +0.13%  | 183.675ms -1.03% | ±0.42% -71.95%   |
| RowsBench               | bench_diff_right_1k_on_10k | 2    | 3   | 74.792mb +0.16%  | 18.689ms +0.03%  | ±1.29% +1673.24% |
| RowsBench               | bench_drop_1k_on_10k       | 2    | 3   | 77.714mb +0.24%  | 1.990ms +23.82%  | ±0.82% -42.74%   |
| RowsBench               | bench_drop_right_1k_on_10k | 2    | 3   | 77.714mb +0.24%  | 1.903ms +17.11%  | ±1.12% -60.94%   |
| RowsBench               | bench_entries_on_10k       | 2    | 3   | 74.826mb +0.25%  | 2.857ms +17.39%  | ±2.77% +96.50%   |
| RowsBench               | bench_filter_on_10k        | 2    | 3   | 75.355mb +0.25%  | 15.573ms +6.43%  | ±1.05% +24.68%   |
| RowsBench               | bench_find_on_10k          | 2    | 3   | 75.355mb +0.25%  | 15.617ms +9.27%  | ±0.53% -20.79%   |
| RowsBench               | bench_find_one_on_10k      | 10   | 3   | 73.258mb +0.26%  | 1.994μs +10.77%  | ±2.40% +0.00%    |
| RowsBench               | bench_first_on_10k         | 10   | 3   | 73.258mb +0.26%  | 0.400μs 0.00%    | ±0.00% 0.00%     |
| RowsBench               | bench_flat_map_on_1k       | 2    | 3   | 86.879mb +0.14%  | 15.965ms +22.35% | ±2.79% +177.84%  |
| RowsBench               | bench_map_on_10k           | 2    | 3   | 116.174mb +0.16% | 67.828ms +6.98%  | ±1.02% +34.87%   |
| RowsBench               | bench_merge_1k_on_10k      | 2    | 3   | 75.875mb +0.25%  | 1.669ms +40.72%  | ±2.41% +146.99%  |
| RowsBench               | bench_partition_by_on_10k  | 2    | 3   | 79.221mb +0.15%  | 59.586ms +4.83%  | ±0.54% -25.52%   |
| RowsBench               | bench_remove_on_10k        | 2    | 3   | 77.976mb +0.24%  | 4.226ms +11.84%  | ±2.80% +434.12%  |
| RowsBench               | bench_sort_asc_on_1k       | 2    | 3   | 73.403mb +0.17%  | 42.584ms +5.72%  | ±1.04% -31.00%   |
| RowsBench               | bench_sort_by_on_1k        | 2    | 3   | 73.404mb +0.17%  | 44.283ms +10.07% | ±1.10% -6.89%    |
| RowsBench               | bench_sort_desc_on_1k      | 2    | 3   | 73.403mb +0.17%  | 44.935ms +12.16% | ±1.17% +23.16%   |
| RowsBench               | bench_sort_entries_on_1k   | 2    | 3   | 75.700mb +0.25%  | 7.940ms +8.39%   | ±1.02% +345.70%  |
| RowsBench               | bench_sort_on_1k           | 2    | 3   | 73.258mb +0.26%  | 31.613ms +10.13% | ±0.38% -34.26%   |
| RowsBench               | bench_take_1k_on_10k       | 10   | 3   | 73.258mb +0.26%  | 13.812μs +6.20%  | ±0.68% +87.98%   |
| RowsBench               | bench_take_right_1k_on_10k | 10   | 3   | 73.258mb +0.26%  | 16.946μs +7.33%  | ±2.67% +344.92%  |
| RowsBench               | bench_unique_on_1k         | 2    | 3   | 96.268mb +0.13%  | 187.591ms +0.56% | ±0.42% +10.44%   |
+-------------------------+----------------------------+------+-----+------------------+------------------+------------------+