[fix](variant)fix top-array data into variant column#54396
Merged
eldenmoon merged 11 commits intoapache:masterfrom Aug 14, 2025
Merged
[fix](variant)fix top-array data into variant column#54396eldenmoon merged 11 commits intoapache:masterfrom
eldenmoon merged 11 commits intoapache:masterfrom
Conversation
Contributor
|
Thank you for your contribution to Apache Doris. Please clearly describe your PR:
|
16 tasks
Contributor
Author
|
run buildall |
TPC-H: Total hot run time: 33652 ms |
TPC-DS: Total hot run time: 170329 ms |
ClickBench: Total hot run time: 33.38 s |
Contributor
Author
|
run buildall |
TPC-H: Total hot run time: 33441 ms |
TPC-DS: Total hot run time: 170544 ms |
ClickBench: Total hot run time: 33.32 s |
Contributor
Author
|
run buildall |
TPC-H: Total hot run time: 33611 ms |
TPC-DS: Total hot run time: 171227 ms |
ClickBench: Total hot run time: 32.6 s |
Contributor
BE UT Coverage ReportIncrement line coverage Increment coverage report
|
Contributor
BE Regression && UT Coverage ReportIncrement line coverage Increment coverage report
|
Contributor
Author
|
run buildall |
TPC-H: Total hot run time: 33681 ms |
TPC-DS: Total hot run time: 169334 ms |
ClickBench: Total hot run time: 33.8 s |
Contributor
BE UT Coverage ReportIncrement line coverage Increment coverage report
|
Contributor
BE Regression && UT Coverage ReportIncrement line coverage Increment coverage report
|
Contributor
Author
|
run buildall |
TPC-H: Total hot run time: 33989 ms |
TPC-DS: Total hot run time: 161506 ms |
Contributor
Author
|
run buildall |
TPC-H: Total hot run time: 34386 ms |
TPC-DS: Total hot run time: 185049 ms |
ClickBench: Total hot run time: 32.79 s |
BE UT Coverage ReportIncrement line coverage Increment coverage report
|
Contributor
BE Regression && UT Coverage ReportIncrement line coverage Increment coverage report
|
1 similar comment
Contributor
BE Regression && UT Coverage ReportIncrement line coverage Increment coverage report
|
github-actions
bot
added
the
approved
Contributor
|
PR approved by at least one committer and no changes requested. |
Contributor
|
PR approved by anyone and no changes requested. |
amorynan
pushed a commit
to amorynan/doris
that referenced
this pull request
Aug 15, 2025
This pr mainly solves the problem when we insert top-level nested array
data into variant columnIssue Number:
like :
```
mysql> insert into sv1 values (1, '[{"a": 1, "c": 1.1}, {"b": "1"}]');
we maintain the association information between Nested in array , in this case , we will keep same offset at a,b,c ,
mysql> select * from sv1;
+------+----------------------------------------------+
| k | v |
+------+----------------------------------------------+
| 1 | {"a":[1,null],"b":[null,"1"],"c":[1.1,null]} |
+------+----------------------------------------------+
```
but in top array we just do flatten array not fill the null value to
maintain the association information between Nested in array
like:
```
mysql> insert into sv1 values (16, '[{"a": 1, "b": 1}, {"b": 2}, {"b": 3}]');
Query OK, 1 row affected (0.15 sec)
{'label':'label_4b3ede8b449a4d9a_b55fdeebe13b741c', 'status':'VISIBLE', 'txnId':'6031'}
mysql> select v from sv1 where k=16;
+-----------------------+
| v |
+-----------------------+
| {"a":[1],"b":[1,2,3]} |
+-----------------------+
1 row in set (0.19 sec)
```
amorynan
pushed a commit
to amorynan/doris
that referenced
this pull request
Aug 16, 2025
This pr mainly solves the problem when we insert top-level nested array
data into variant columnIssue Number:
like :
```
mysql> insert into sv1 values (1, '[{"a": 1, "c": 1.1}, {"b": "1"}]');
we maintain the association information between Nested in array , in this case , we will keep same offset at a,b,c ,
mysql> select * from sv1;
+------+----------------------------------------------+
| k | v |
+------+----------------------------------------------+
| 1 | {"a":[1,null],"b":[null,"1"],"c":[1.1,null]} |
+------+----------------------------------------------+
```
but in top array we just do flatten array not fill the null value to
maintain the association information between Nested in array
like:
```
mysql> insert into sv1 values (16, '[{"a": 1, "b": 1}, {"b": 2}, {"b": 3}]');
Query OK, 1 row affected (0.15 sec)
{'label':'label_4b3ede8b449a4d9a_b55fdeebe13b741c', 'status':'VISIBLE', 'txnId':'6031'}
mysql> select v from sv1 where k=16;
+-----------------------+
| v |
+-----------------------+
| {"a":[1],"b":[1,2,3]} |
+-----------------------+
1 row in set (0.19 sec)
```
dataroaring
pushed a commit
that referenced
this pull request
Aug 18, 2025
### What problem does this PR solve? backport : #54396 This pr mainly solves the problem when we insert top-level nested array data into variant columnIssue Number: like : ``` mysql> insert into sv1 values (1, '[{"a": 1, "c": 1.1}, {"b": "1"}]'); ``` we maintain the association information between Nested in array , in this case , we will keep same offset at a,b,c , ``` mysql> select * from sv1; +------+----------------------------------------------+ | k | v | +------+----------------------------------------------+ | 1 | {"a":[1,null],"b":[null,"1"],"c":[1.1,null]} | +------+----------------------------------------------+ ``` but in top array we just do flatten array not fill the null value to maintain the association information between Nested in array like: ``` mysql> insert into sv1 values (16, '[{"a": 1, "b": 1}, {"b": 2}, {"b": 3}]'); Query OK, 1 row affected (0.15 sec) {'label':'label_4b3ede8b449a4d9a_b55fdeebe13b741c', 'status':'VISIBLE', 'txnId':'6031'} mysql> select v from sv1 where k=16; +-----------------------+ | v | +-----------------------+ | {"a":[1],"b":[1,2,3]} | +-----------------------+ 1 row in set (0.19 sec) ``` close #xxx Related PR: #xxx Problem Summary: ### Release note None ### Check List (For Author) - Test <!-- At least one of them must be included. --> - [ ] Regression test - [ ] Unit Test - [ ] Manual test (add detailed scripts or steps below) - [ ] No need to test or manual test. Explain why: - [ ] This is a refactor/code format and no logic has been changed. - [ ] Previous test can cover this change. - [ ] No code files have been changed. - [ ] Other reason <!-- Add your reason? --> - Behavior changed: - [ ] No. - [ ] Yes. <!-- Explain the behavior change --> - Does this need documentation? - [ ] No. - [ ] Yes. <!-- Add document PR link here. eg: apache/doris-website#1214 --> ### Check List (For Reviewer who merge this PR) - [ ] Confirm the release note - [ ] Confirm test cases - [ ] Confirm document - [ ] Add branch pick label <!-- Add branch pick label that this PR should merge into -->
morrySnow
pushed a commit
that referenced
this pull request
Aug 19, 2025
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
8 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What problem does this PR solve?
This pr mainly solves the problem when we insert top-level nested array data into variant columnIssue Number:
like :
but in top array we just do flatten array not fill the null value to maintain the association information between Nested in array
like:
Issue Number: close #xxx
Related PR: #xxx
Problem Summary:
Release note
None
Check List (For Author)
Test
Behavior changed:
Does this need documentation?
Check List (For Reviewer who merge this PR)