Skip to content

Comments

feat(online): union all query#3590

Merged
aceforeverd merged 2 commits into4paradigm:mainfrom
aceforeverd:feat-union-query
Dec 4, 2023
Merged

feat(online): union all query#3590
aceforeverd merged 2 commits into4paradigm:mainfrom
aceforeverd:feat-union-query

Conversation

@aceforeverd
Copy link
Collaborator

@aceforeverd aceforeverd commented Nov 14, 2023

SYNTAX

UNION is one kind of set operation, currently it support UNION only.

set_operation:
  query_expr UNION { DISTINCT | ALL } query_expr

query_expr:
  [ WITH {non_recursive_cte}[, ...] ]
  { select | ( query_expr ) | set_operation }
  [ ORDER BY expression [, ...] ]
  [ LIMIT count ]

select:
  SELECT select_list
  [ FROM from_clause[, ...] ]
  [ WHERE bool_expression ]
  [ GROUP BY group_by_specification ]
  [ HAVING bool_expression ]
  [ WINDOW window_clause ]

Rules

1. ONLY UNION ALL

2. `UNION output result, with ordering undefined by default.

Say SELECT * FROM t1 UNION ALL SELECT * FROM t2, and assume two inputs source for UNION is know as ordered. UNION result may still not ordered.

3. support with other SQL clauses

combine support status
UNION (t1, t2) online preview
LASTJOIN ( UNION ) online preview/request
LEFTJOIN ( UNION ) online preview
LASTJOIN (LEFTJOIN(UNION)) online preview/reqeust
LASTJOIN (UNION (FILTER,..) ) 部分支持, 具体看优化情况
WINDOW (UNION ) NO
GROUP BY ( UNION ) NO

4. Online Rules

PATTERN: LAST JOIN ( UNION ALL (t1, t2) )

  • GUIDE 1: use with LASTJOIN, LASTJOIN should index optimize all producers of UNION query
  • GUIDE 2: use it only if ordering of the UNION query does not matter

@github-actions github-actions bot added execute-engine hybridse sql engine storage-engine openmldb storage engine. nameserver & tablet labels Nov 14, 2023
@codecov
Copy link

codecov bot commented Nov 14, 2023

Codecov Report

Attention: 178 lines in your changes are missing coverage. Please review.

Comparison is base (5731b2b) 40.14% compared to head (6a03397) 74.94%.
Report is 6 commits behind head on main.

Files Patch % Lines
hybridse/src/vm/catalog_wrapper.cc 38.77% 60 Missing ⚠️
hybridse/src/vm/runner.cc 49.12% 29 Missing ⚠️
hybridse/src/vm/catalog_wrapper.h 48.88% 23 Missing ⚠️
...se/src/passes/physical/group_and_sort_optimized.cc 81.66% 22 Missing ⚠️
hybridse/src/vm/physical_op.cc 73.61% 19 Missing ⚠️
hybridse/src/node/sql_node.cc 60.00% 10 Missing ⚠️
hybridse/src/vm/schemas_context.cc 0.00% 6 Missing ⚠️
hybridse/src/vm/mem_catalog.cc 60.00% 4 Missing ⚠️
hybridse/src/vm/runner_builder.cc 87.50% 2 Missing ⚠️
hybridse/include/codec/row.h 0.00% 1 Missing ⚠️
... and 2 more
Additional details and impacted files
@@              Coverage Diff              @@
##               main    #3590       +/-   ##
=============================================
+ Coverage     40.14%   74.94%   +34.80%     
  Complexity      635      635               
=============================================
  Files           191      724      +533     
  Lines         11207   130710   +119503     
  Branches       1317     1317               
=============================================
+ Hits           4499    97966    +93467     
- Misses         6417    32453    +26036     
  Partials        291      291               

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

@aceforeverd aceforeverd marked this pull request as ready for review November 15, 2023 16:01
Re-impl column to source info in physical node, support multiple result
source info, previously it is one result only.

Multiple source result is required for SetOperationNode, like union all,
every input node may produce the target column.
@aceforeverd aceforeverd changed the title feat: union query feat(online): union all query Dec 4, 2023
@aceforeverd aceforeverd merged commit 85926ce into 4paradigm:main Dec 4, 2023
@aceforeverd aceforeverd deleted the feat-union-query branch December 4, 2023 08:06
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

execute-engine hybridse sql engine storage-engine openmldb storage engine. nameserver & tablet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants