Unified regex based scalar functions and their behaviors

[norberttech]

Change Log

Added

• Regex Scalar Function
• Missing optional arguments to all regex scalar functions

Fixed

Changed

• Renamed PregMatch into RegexMatch
• Renamed PregMatchAll into RegexMatchAll
• Renamed PregReplex into RegexReplace

Removed

Deprecated

Security

---

resolves: #1159

Description

Usage example:

Input:

<rss xmlns:g="http://base.google.com/ns/1.0">
<channel>
    <item>
        <g:price>123 EUR</g:price>
    </item>
    <item>
        <g:price>154.12 USD</g:price>
    </item>
</channel>
</rss>

Pipeline:

df()
    ->read(from_xml(__DIR__ . '/file.xml', 'rss/channel/item'))
    ->withEntry('price', ref('node')->xpath('g:price')->domElementValue()->regex(lit('/(\d+(?:\.\d+)?)\s+([A-Z]{3})/')))
    ->withEntry('amount', when(ref('price')->isNotNull(), ref('price')->arrayGet(1))->cast('float'))
    ->withEntry('currency', when(ref('price')->isNotNull(), ref('price')->arrayGet(2)))
    ->drop('node', 'price')
    ->write(to_output(false))
    ->run();

output:

+--------+----------+
| amount | currency |
+--------+----------+
| 123.00 |      EUR |
+--------+----------+
1 rows
+--------+----------+
| amount | currency |
+--------+----------+
| 154.12 |      USD |
+--------+----------+
1 rows

[github-actions]

Flow PHP - Benchmarks

^{_{Results of the benchmarks from this PR are compared with the results from 1.x branch.}}

Extractors

+-----------------------+-------------------+------+-----+-----------------+------------------+-----------------+
| benchmark             | subject           | revs | its | mem_peak        | mode             | rstdev          |
+-----------------------+-------------------+------+-----+-----------------+------------------+-----------------+
| CSVExtractorBench     | bench_extract_10k | 1    | 3   | 3.945mb +0.11%  | 508.336ms -0.72% | ±0.29% +21.98%  |
| JsonExtractorBench    | bench_extract_10k | 1    | 3   | 3.977mb +0.11%  | 1.088s -0.50%    | ±0.59% -60.13%  |
| ParquetExtractorBench | bench_extract_10k | 1    | 3   | 28.538mb +0.02% | 433.980ms -1.37% | ±0.99% +188.34% |
| TextExtractorBench    | bench_extract_10k | 1    | 3   | 3.704mb +0.12%  | 33.894ms +0.30%  | ±0.97% -40.19%  |
| XmlExtractorBench     | bench_extract_10k | 1    | 3   | 3.651mb +0.12%  | 436.500ms -2.13% | ±0.80% -63.87%  |
+-----------------------+-------------------+------+-----+-----------------+------------------+-----------------+

Transformers

+-----------------------------+--------------------------+------+-----+------------------+-----------------+----------------+
| benchmark                   | subject                  | revs | its | mem_peak         | mode            | rstdev         |
+-----------------------------+--------------------------+------+-----+------------------+-----------------+----------------+
| RenameEntryTransformerBench | bench_transform_10k_rows | 1    | 3   | 116.046mb +0.00% | 59.838ms -0.76% | ±0.37% -53.40% |
+-----------------------------+--------------------------+------+-----+------------------+-----------------+----------------+

Loaders

+--------------------+----------------+------+-----+------------------+-----------------+----------------+
| benchmark          | subject        | revs | its | mem_peak         | mode            | rstdev         |
+--------------------+----------------+------+-----+------------------+-----------------+----------------+
| CSVLoaderBench     | bench_load_10k | 1    | 3   | 54.167mb +0.01%  | 83.864ms -1.23% | ±1.29% -13.98% |
| JsonLoaderBench    | bench_load_10k | 1    | 3   | 106.534mb +0.00% | 52.125ms -3.65% | ±0.09% -82.41% |
| ParquetLoaderBench | bench_load_10k | 1    | 3   | 123.816mb +0.01% | 1.238s -0.11%   | ±0.35% -57.54% |
| TextLoaderBench    | bench_load_10k | 1    | 3   | 16.961mb +0.03%  | 44.397ms -0.10% | ±0.72% +31.98% |
+--------------------+----------------+------+-----+------------------+-----------------+----------------+

Building Blocks

+-------------------------+----------------------------+------+-----+------------------+------------------+-----------------+
| benchmark               | subject                    | revs | its | mem_peak         | mode             | rstdev          |
+-------------------------+----------------------------+------+-----+------------------+------------------+-----------------+
| TypeDetectorBench       | bench_type_detector        | 1    | 3   | 59.714mb +0.01%  | 437.711ms -0.37% | ±2.98% -15.51%  |
| TypeDetectorBench       | bench_type_detector        | 1    | 3   | 14.254mb +0.03%  | 88.115ms +0.78%  | ±2.77% +201.80% |
| NativeEntryFactoryBench | bench_entry_factory        | 1    | 3   | 116.533mb +0.00% | 506.079ms +0.08% | ±0.99% -25.80%  |
| NativeEntryFactoryBench | bench_entry_factory        | 1    | 3   | 60.011mb +0.01%  | 252.942ms -3.11% | ±0.41% -87.62%  |
| NativeEntryFactoryBench | bench_entry_factory        | 1    | 3   | 14.945mb +0.03%  | 54.555ms -2.90%  | ±0.86% -34.26%  |
| RowsBench               | bench_chunk_10_on_10k      | 2    | 3   | 86.803mb +0.01%  | 3.430ms -8.85%   | ±1.14% -61.91%  |
| RowsBench               | bench_diff_left_1k_on_10k  | 2    | 3   | 102.401mb +0.01% | 193.475ms +3.28% | ±1.10% -17.91%  |
| RowsBench               | bench_diff_right_1k_on_10k | 2    | 3   | 85.121mb +0.01%  | 19.119ms -0.52%  | ±1.45% -15.73%  |
| RowsBench               | bench_drop_1k_on_10k       | 2    | 3   | 88.043mb +0.00%  | 2.007ms +8.08%   | ±3.44% +38.24%  |
| RowsBench               | bench_drop_right_1k_on_10k | 2    | 3   | 88.043mb +0.00%  | 1.868ms -10.66%  | ±2.40% -24.01%  |
| RowsBench               | bench_entries_on_10k       | 2    | 3   | 85.155mb +0.01%  | 2.926ms -0.41%   | ±3.04% -7.94%   |
| RowsBench               | bench_filter_on_10k        | 2    | 3   | 85.684mb +0.01%  | 15.039ms -1.27%  | ±2.89% +209.01% |
| RowsBench               | bench_find_on_10k          | 2    | 3   | 85.684mb +0.01%  | 15.501ms +0.51%  | ±0.01% -98.98%  |
| RowsBench               | bench_find_one_on_10k      | 10   | 3   | 83.588mb +0.01%  | 1.806μs -9.70%   | ±2.57% +0.00%   |
| RowsBench               | bench_first_on_10k         | 10   | 3   | 83.588mb +0.01%  | 0.400μs -20.00%  | ±0.00% +0.00%   |
| RowsBench               | bench_flat_map_on_1k       | 2    | 3   | 92.938mb +0.00%  | 12.219ms -15.91% | ±0.45% -62.89%  |
| RowsBench               | bench_map_on_10k           | 2    | 3   | 122.309mb +0.00% | 61.877ms -1.77%  | ±0.83% +73.59%  |
| RowsBench               | bench_merge_1k_on_10k      | 2    | 3   | 86.204mb +0.01%  | 1.610ms -0.49%   | ±3.72% +72.65%  |
| RowsBench               | bench_partition_by_on_10k  | 2    | 3   | 89.550mb +0.01%  | 68.499ms +5.21%  | ±0.72% -49.01%  |
| RowsBench               | bench_remove_on_10k        | 2    | 3   | 88.305mb +0.00%  | 4.051ms -15.75%  | ±3.09% +257.69% |
| RowsBench               | bench_sort_asc_on_1k       | 2    | 3   | 83.732mb +0.01%  | 39.763ms -6.09%  | ±0.54% -85.24%  |
| RowsBench               | bench_sort_by_on_1k        | 2    | 3   | 83.732mb +0.01%  | 40.270ms -5.05%  | ±1.63% +179.50% |
| RowsBench               | bench_sort_desc_on_1k      | 2    | 3   | 83.732mb +0.01%  | 40.068ms -5.86%  | ±3.20% +54.96%  |
| RowsBench               | bench_sort_entries_on_1k   | 2    | 3   | 86.029mb +0.01%  | 7.490ms -6.01%   | ±0.41% -76.66%  |
| RowsBench               | bench_sort_on_1k           | 2    | 3   | 83.588mb +0.01%  | 29.112ms -7.59%  | ±1.28% -30.30%  |
| RowsBench               | bench_take_1k_on_10k       | 10   | 3   | 83.588mb +0.01%  | 13.542μs -2.56%  | ±2.40% -4.81%   |
| RowsBench               | bench_take_right_1k_on_10k | 10   | 3   | 83.588mb +0.01%  | 16.299μs -1.36%  | ±3.74% +229.56% |
| RowsBench               | bench_unique_on_1k         | 2    | 3   | 102.402mb +0.01% | 194.765ms -0.90% | ±1.40% +28.94%  |
+-------------------------+----------------------------+------+-----+------------------+------------------+-----------------+