Skip to content

[fix](planner) should always execute projection plan#17885

Merged
morrySnow merged 1 commit intoapache:masterfrom
morrySnow:fix_insert_projection
Mar 22, 2023
Merged

[fix](planner) should always execute projection plan#17885
morrySnow merged 1 commit intoapache:masterfrom
morrySnow:fix_insert_projection

Conversation

@morrySnow
Copy link
Contributor

@morrySnow morrySnow commented Mar 17, 2023

Proposed changes

Issue Number: close #xxx

Problem summary

  1. should always execute projection plan, whatever the statement it is.
  2. should always execute projection plan, since we only have vectorized engine now

Checklist(Required)

  • Does it affect the original behavior
  • Has unit tests been added
  • Has document been added or modified
  • Does it need to update dependencies
  • Is this PR support rollback (If NO, please explain WHY)

Further comments

If this is a relatively large or complex change, kick off the discussion at dev@doris.apache.org by explaining why you chose the solution you did and what alternatives you considered, etc...

@github-actions github-actions bot added the area/planner Issues or PRs related to the query planner label Mar 17, 2023
@morrySnow morrySnow added usercase Important user case type label dev/1.2.3 labels Mar 17, 2023
@morrySnow morrySnow force-pushed the fix_insert_projection branch from 257e5a9 to 8ae8408 Compare March 19, 2023 13:05
@morrySnow
Copy link
Contributor Author

morrySnow commented Mar 19, 2023

run buildall

@hello-stephen
Copy link
Contributor

hello-stephen commented Mar 19, 2023 •
edited
Loading

TeamCity pipeline, clickbench performance test result:
the sum of best hot time: 34.6 seconds
stream load tsv: 465 seconds loaded 74807831229 Bytes, about 153 MB/s
stream load json: 25 seconds loaded 2358488459 Bytes, about 89 MB/s
stream load orc: 74 seconds loaded 1101869774 Bytes, about 14 MB/s
stream load parquet: 33 seconds loaded 861443392 Bytes, about 24 MB/s
https://doris-community-test-1308700295.cos.ap-hongkong.myqcloud.com/tmp/20230321122214_clickbench_pr_117976.html

@morrySnow morrySnow force-pushed the fix_insert_projection branch from 8ae8408 to ac16382 Compare March 21, 2023 05:28
@morrySnow
Copy link
Contributor Author

morrySnow commented Mar 21, 2023

run buildall

@github-actions github-actions bot added the approved Indicates a PR has been approved by one committer. label Mar 22, 2023
@github-actions
Copy link
Contributor

github-actions bot commented Mar 22, 2023

PR approved by at least one committer and no changes requested.

@github-actions
Copy link
Contributor

github-actions bot commented Mar 22, 2023

PR approved by anyone and no changes requested.

@morrySnow morrySnow merged commit a4b151e into apache:master Mar 22, 2023
@morrySnow morrySnow deleted the fix_insert_projection branch March 22, 2023 02:53
morningman pushed a commit that referenced this pull request Mar 28, 2023
@morrySnow @morningman
[fix](planner) should always execute projection plan (#17885)
f5d6201 
1. should always execute projection plan, whatever the statement it is.
2. should always execute projection plan, since we only have vectorized engine now
gnehil pushed a commit to gnehil/doris that referenced this pull request Apr 21, 2023
@morrySnow @gnehil
[fix](planner) should always execute projection plan (apache#17885)
df650e5 
1. should always execute projection plan, whatever the statement it is.
2. should always execute projection plan, since we only have vectorized engine now
mongo360 pushed a commit to mongo360/doris that referenced this pull request Jul 12, 2023
@morrySnow
[fix](planner) should always execute projection plan (apache#17885)
775f743 
1. should always execute projection plan, whatever the statement it is.
2. should always execute projection plan, since we only have vectorized engine now
swjtu-zhanglei pushed a commit to swjtu-zhanglei/incubator-doris that referenced this pull request Jul 25, 2023
@deardeng
[cherry-pick] Pick doris planner fix 17885 and 18042 (apache#1569)
b7ae91a 
commit f5d6201
Author: morrySnow <101034200+morrySnow@users.noreply.github.com>
Date:   Wed Mar 22 10:53:15 2023 +0800

    [fix](planner) should always execute projection plan (apache#17885)

    1. should always execute projection plan, whatever the statement it is.
    2. should always execute projection plan, since we only have vectorized engine now

commit ad61c84
Author: mch_ucchi <41606806+sohardforaname@users.noreply.github.com>
Date:   Mon Mar 27 17:50:52 2023 +0800

    [fix](planner) fix conjunct planned on exchange node (apache#18042)

    sql like:
    select k5, k6, SUM(k3) AS k3
    from (
        select
            k5,
            date_format(k6, '%Y-%m-%d') as k6,
            count(distinct k3) as k3
        from t
        group by k5, k6
    ) AS temp where 1=1
    group by k5, k6;

    will throw exception since conjuncts planned on exchange node, because exchange node cannot handle conjuncts, now we skip exchange node when planning conjuncts, which fixes the bug.
    notice: the bug occurs iff the conjunct is always true like 1=1 above.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@starocean999 starocean999 starocean999 approved these changes

Assignees

No one assigned

Labels

approved Indicates a PR has been approved by one committer. area/planner Issues or PRs related to the query planner dev/1.2.4-merged reviewed usercase Important user case type label

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

4 participants

@morrySnow @hello-stephen @starocean999 @morningman

Comments