Skip to content

GH-35948: [Go] Only cast int8 and unit8 to float64 when JSON marshaling arrays - #35950

Merged
zeroshade merged 3 commits into
apache:mainfrom
cloudquery:fix/array_dont_cast_int64_unit64
Jun 6, 2023
Merged

GH-35948: [Go] Only cast int8 and unit8 to float64 when JSON marshaling arrays#35950
zeroshade merged 3 commits into
apache:mainfrom
cloudquery:fix/array_dont_cast_int64_unit64

Conversation

@erezrokah

@erezrokah erezrokah commented Jun 6, 2023 •
edited by github-actions Bot
Loading

Copy link
Copy Markdown
Contributor

Rationale for this change

See issue.

Based on

arrow/go/arrow/array/numeric.gen.go.tmpl

Line 120 in 9fb8697

{{else if (eq .Size "1")}}
it seems we should only do the casting for unit8 and int8 types.

What changes are included in this PR?

This PR fixes the numeric template so it only casts int8 and unit8 to float64 when JSON marshaling arrays to avoid losing data for other types.

Are these changes tested?

Yes

Are there any user-facing changes?

The user facing change is that if a user was JSON marshaling arrays that has uint64 or int64 values they can now get different (more accurate) values than before

@erezrokah
erezrokah requested a review from zeroshade as a code owner June 6, 2023 15:56
@zeroshade zeroshade changed the title GH-35948: [GO] Only cast int8 and unit8 to float64 when JSON marshaling arrays GH-35948: [Go] Only cast int8 and unit8 to float64 when JSON marshaling arrays Jun 6, 2023
@erezrokah

erezrokah commented Jun 6, 2023

Copy link
Copy Markdown
Contributor Author

Not sure if the failing test is related to flaky, help appreciated

@zeroshade

zeroshade commented Jun 6, 2023

Copy link
Copy Markdown
Member

@erezrokah The failing test is indeed a flaky test, I haven't yet been able to reproduce it locally in order to fix it which annoys me. But you're fine.

zeroshade
zeroshade approved these changes Jun 6, 2023
View reviewed changes

@zeroshade zeroshade left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM! Thanks much for this!

@zeroshade
zeroshade merged commit 745fa94 into apache:main Jun 6, 2023
@github-actions github-actions Bot added awaiting merge Awaiting merge and removed awaiting review Awaiting review labels Jun 6, 2023
@erezrokah

erezrokah commented Jun 6, 2023

Copy link
Copy Markdown
Contributor Author

Thanks for the quick review and merge!

@erezrokah
erezrokah deleted the fix/array_dont_cast_int64_unit64 branch June 6, 2023 16:39
@ursabot

ursabot commented Jun 7, 2023

Copy link
Copy Markdown

Benchmark runs are scheduled for baseline = 105b9df and contender = 745fa94. 745fa94 is a master commit associated with this PR. Results will be available as each benchmark for each run completes.
Conbench compare runs links:
[Finished ⬇️0.0% ⬆️0.0%] ec2-t3-xlarge-us-east-2
[Finished ⬇️2.04% ⬆️0.03%] test-mac-arm
[Finished ⬇️0.0% ⬆️0.0%] ursa-i9-9960x
[Failed ⬇️0.15% ⬆️0.06%] ursa-thinkcentre-m75q
Buildkite builds:
[Finished] 745fa94f ec2-t3-xlarge-us-east-2
[Finished] 745fa94f test-mac-arm
[Finished] 745fa94f ursa-i9-9960x
[Failed] 745fa94f ursa-thinkcentre-m75q
[Finished] 105b9df0 ec2-t3-xlarge-us-east-2
[Finished] 105b9df0 test-mac-arm
[Finished] 105b9df0 ursa-i9-9960x
[Failed] 105b9df0 ursa-thinkcentre-m75q
Supported benchmarks:
ec2-t3-xlarge-us-east-2: Supported benchmark langs: Python, R. Runs only benchmarks with cloud = True
test-mac-arm: Supported benchmark langs: C++, Python, R
ursa-i9-9960x: Supported benchmark langs: Python, R, JavaScript
ursa-thinkcentre-m75q: Supported benchmark langs: C++, Java

zeroshade pushed a commit to zeroshade/arrow that referenced this pull request Apr 11, 2024
@erezrokah @zeroshade
apacheGH-35948: [Go] Only cast int8 and unit8 to float64 when J…
8ace49b 
…SON marshaling arrays (apache#35950)

### Rationale for this change

See [issue](apache#35948).

Based on https://github.com/apache/arrow/blob/9fb8697dcb442f63317c7d6046393fb74842e0ae/go/arrow/array/numeric.gen.go.tmpl#L120 it seems we should only do the casting for `unit8` and `int8` types.

### What changes are included in this PR?

This PR fixes the numeric template so it only casts `int8` and `unit8` to `float64` when JSON marshaling arrays to avoid losing data for other types.

### Are these changes tested?

Yes

### Are there any user-facing changes?

The user facing change is that if a user was JSON marshaling arrays that has `uint64` or `int64` values they can now get different (more accurate) values than before

* Closes: apache#35948

Authored-by: erezrokah <erezrokah@users.noreply.github.com>
Signed-off-by: Matt Topol <zotthewizard@gmail.com>
(cherry picked from commit 745fa94)
@zeroshade zeroshade mentioned this pull request Apr 11, 2024
Closed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@zeroshade zeroshade zeroshade approved these changes

Assignees

No one assigned

Labels

awaiting merge Awaiting merge Component: Go

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

[Go] Marshaling int64 or unit64 arrays to JSON can cause loss of data

3 participants

@erezrokah @zeroshade @ursabot