Skip to content

Conversation

@stloyd
Copy link
Member

@stloyd stloyd commented Oct 16, 2023

Change Log

Added

Fixed

  • Fix Avro adapter handling nullable values

Changed

Removed

Deprecated

Security


Description

Without this fix adjusted test for Avro fails:

There was 1 error:

1) Flow\ETL\Adapter\Avro\Tests\Integration\AvroTest::test_writing_and_reading_avro_with_all_supported_types
AvroIOTypeException: The datum array (
  'integer' => 10,
  'float' => 1.5,
  'string' => NULL,
  'boolean' => true,
  'datetime' => 1697475815694710,
  'json_object' => '{"id":1,"name":"test"}',
  'json' => '[{"id":1,"name":"test"},{"id":2,"name":"test"}]',
  'list_of_strings' => 
  array (
    0 => 'a',
    1 => 'b',
    2 => 'c',
  ),
  'list_of_datetimes' => 
  array (
    0 => 1697475815694711,
    1 => 1697475815694712,
    2 => 1697475815694712,
  ),
  'address' => 
  array (
    'street' => 'street_10',
    'city' => 'city_10',
    'zip' => 'zip_10',
    'country' => 'country_10',
    'location' => 
    array (
      'lat' => 1.5,
      'lon' => 1.5,
    ),
  ),
) is not an example of schema {"type":"record","name":"row","fields":[{"name":"integer","type":{"type":"int"}},{"name":"float","type":{"type":"float"}},{"name":"string","type":{"type":"string"}},{"name":"boolean","type":{"type":"boolean"}},{"name":"datetime","type":{"type":"long"},"logicalType":"timestamp-micros"},{"name":"json_object","type":{"type":"string"}},{"name":"json","type":{"type":"string"}},{"name":"list_of_strings","type":{"type":"array","items":{"type":"string"}}},{"name":"list_of_datetimes","type":{"type":"array","items":{"type":"long"},"logicalType":"timestamp-micros"}},{"name":"address","type":{"type":"record","name":"Address","fields":[{"name":"street","type":{"type":"string"}},{"name":"city","type":{"type":"string"}},{"name":"zip","type":{"type":"string"}},{"name":"country","type":{"type":"string"}},{"name":"location","type":{"type":"record","name":"Location","fields":[{"name":"lat","type":{"type":"float"}},{"name":"lon","type":{"type":"float"}}]}}]}}]}

/Users/stloyd/Documents/flow/vendor/flix-tech/avro-php/lib/avro/datum.php:101
/Users/stloyd/Documents/flow/vendor/flix-tech/avro-php/lib/avro/datum.php:147
/Users/stloyd/Documents/flow/vendor/flix-tech/avro-php/lib/avro/data_file.php:543
/Users/stloyd/Documents/flow/src/adapter/etl-adapter-avro/src/Flow/ETL/Adapter/Avro/FlixTech/AvroLoader.php:98
/Users/stloyd/Documents/flow/src/core/etl/src/Flow/ETL/Pipeline/SynchronousPipeline.php:81
/Users/stloyd/Documents/flow/src/core/etl/src/Flow/ETL/DataFrame.php:582
/Users/stloyd/Documents/flow/src/adapter/etl-adapter-avro/tests/Flow/ETL/Adapter/Avro/Tests/Integration/AvroTest.php:135

@stloyd stloyd requested a review from norberttech October 16, 2023 17:04
@norberttech norberttech enabled auto-merge (squash) October 16, 2023 17:07
@norberttech norberttech merged commit 651d3a3 into flow-php:1.x Oct 16, 2023
@stloyd stloyd deleted the bugfix/arvo-nullable branch October 16, 2023 17:17
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants